Speed is one of those "Quantity has a quality all its own" things. We use very fast tools in a qualitatively different way, even though all that changed was how long it takes in seconds.
It is interesting that nobody was writing these tools in C or in C++. There are obvious ergonomic reasons, but perhaps also it matters that Rust cares a lot more about types than either of those languages.
> It is interesting that nobody was writing these tools in C or in C++
This one's easier to explain. People interested in tooling for a specific language probably want to write that tooling in that language (hence pip, poetry, mypy, jedi, etc). Normally that would be the end if it, if Python wasn't 10-100x slower than a natively compiled language. And going from Python to Rust is an order of magnitude easier than going from Python to C or Python to C++, because the compiler is so good at identifying silly mistakes. Rust is just a friendlier language.
It's just very hard to write such systems in C/C++. Even all these Rust versions are segfaulting and panicking quite a lot. So many corner and edge cases that can be found in Python code, and the memory handling is also hard.
The author of Zuban started writing it back in 2020 or 2021, so it took him more than 4 years to complete it. And he is the author of Jedi, so he had prior experience already.
I get why they'd panic, but why have enough segfaults that you noticed? So, I went and read the code.
Zuban seems to have a bunch of scary "I'm not sure if this is correct" unsafe blocks, which to me would be a red flag. I mean, it's better that there's a comment expressing the doubt, but my experience is that if you're not sure whether it's correct, it's probably not correct.
While acknowledging the risk of causing a pile-on (which I don't want!), would it be possible to have a link to them or a description of what the unsafe blocks accomplish? I'm intrigued if they are for performance or API ergonomics, if they are due to limitations of the borrow checker, the stdlib or crate dependencies.
For anyone reaching for unsafe, there are in many cases either an existing API (split_at_mut comes to mind). For others, using zero-copy or bytemuck instead of unsafe is a good idea too.
None of that is to say "never write unsafe", unsafe existing is pretty much one of the reasons for Rust to be :)
I skimmed these, so this is nothing close to a survey, much less a comprehensive review of the software, however
For example in crates/zuban_python/src/file/diagnostics.rs:
"TODO this unsafe feels very wrong, because a bit lower we might modify the complex/ points."
or crates/zuban_python/src/database.rs:
"Points are guarded by specific logic and if they are overwritten by something that shouldn't it should not be that tragic."
I saw nothing where I was like "ZOMG this is definitely busted" but I definitely did not get the robust "Oh, I see now why this is correct" that I like from a good unsafe rationale comment, and these aren't tiny things like the small unsafe bit twiddling transmutes which are probably either actually correct or in any case will do what you expected at compile time and so any surprises are priced in without a rationale text.
Yes, but why they did not write it in a compiled language? Pyright is pretty slow in large code bases and takes a lot of RAM. Javascript can be faster than python in some cases, but Python is so easily extendable with C,C++, Cython, Rust. They could use Python with one of the compiled language.
It's also horrible for fasle positives unless your project happens to be the exact same setup as the maintainers' - I had to turn off the actual type checking on it. I've since moved wholesale to the Ty alpha and it feels a hell of a lot smarter.
Sometimes dynamic Python idioms are incompatible with typed Python. I personally think that's fine, since I consider static typing a significant improvement overall.
This isn’t. They actually fixed that bug. Then they changed their minds and backed the fix back out again because they don’t think you should write Python that way:
> I think EAFP is a very unfortunate and ill-advised practice.
if "baz" in bar and "qux" in bar["baz"]:
foo = bar["baz"]["qux"]
...
else:
...
If this were a linter then I would accept that it is going to be opinionated. But this is not a linter, it’s a type checker. Their opinions about EAFP are irrelevant. That’s idiomatic Python.
Well I agree with them. The second code is clearly better. Exceptions should be used for error handling and if those keys are actually optional then you should explicitly check if they exist (or use something like `bar.get("baz")`).
You are welcome to that opinion, but type checkers should not be opinionated, especially if they push people to write non-idiomatic Python. If you think this should be written a different way, that’s what lint rules are for.
I'm sorry, I can't take seriously any piece of software which decided to prefix the previous version's name with "based". I'm aware this is a me problem.
Hah. I love the name. It implies that whatever the original “pyright” was doing wasn’t keepin’ it real. This new version, it’s “based” so it must be somehow more “real” and “grounded” and “legit”.
All I know is it is much more strict about stuff than pylance was.
yeah we all heard that story every 3 months with all the previous package managers. until there's adoption by an overwhelming majority of projects, it isn't really settled yet.
I’ve literally never heard that much buzz and excitement about Python tool before. And I’ve seen them all.
All of them had some big issue that prevented it from getting mainstream. Either it was slow, or didn’t work with existing workflow, or had complex configuration, or something that prevented gradual adoption.
uv is universally praised as the second coming Christ in Python world (and for a good reason). So no, I doubt there will be something else. Not only you need to be better than uv, you also need to have community momentum.
- zuban
- ty (from ruff team)
- pyrefly
One year ago, we had none of them, only slow options.