• 0 Posts
  • 15 Comments
Joined 2 years ago
cake
Cake day: June 22nd, 2024

help-circle
  • I write python code for a living. There is no way to sugarcoat it, the new unittests are slop. There already exists a good writeup of why, which I’m going to quote here:

    So, look. One shot rewriting the whole test suite in another language is probably not great to do, but what happened here is so much worse than you are expecting. https://github.com/RsyncProject/rsync/pull/903/
    This does not “translate tests into pytest” or a unit testing framework, it writes its own testing framework where tests are whole python scripts that redefine basic test functions in every script. Surely there would be a single way to “run rsync and get the results” - nope, well, there is, but then every test file will randomly redefine its own _run_and_capture function. So like now rsync needs a test suite for its test suite.
    If instead of telling an LLM to “rewrite the tests in python” you just searched “python testing” you would find the pytest docs. And then you would find examples. And then you could write fixtures to deduplicate all the prior shell script setup and teardown stuff, and so on. But since it was just “rewrite the tests in python” its now worse than before, and the odds of the rewrite actually being a 100% faithful translation are close to 0.

    https://neuromatch.social/@jonny/116666900898570791

    Yes right - and after reading about a dozen of the test scripts I can definitely see why using pytest would be useful here to consolidate some of the behavior that was repetitive and ad-hoc in the original testing scripts. Like the tests need to do repetitive things like set up test directories with different names and structures, run and capture results, setup and teardown a server, parameterize over a range of values. Done right, a pytest suite would have made perfect sense and improved both the existing tests by making them more systematic and uniform, but also made it easier to add new tests over time. However that is not what happened, and what did happen is much worse because it did the opposite of almost all those desirable qualities.

    https://neuromatch.social/@jonny/116671260017373441

    You should read the whole thread, the author goes into more detail, as to why you cannot trust the software any more after the rewrite of the unittests and why you should avoid any new release of rsync since then.













  • I generally agree with your statement, just one thing to keep in mind: Mypy sucks for any library larger than a few thousand lines spread over a couple of files, but pyright is developed by M$ and might be part of their usual Embrace, extend, and extinguish strategy. The other two contenders are pytype (google) and pyre (facebook), so it’s not like there’s a good selection of independent, good and FOSS type checkers out there at the moment.

    Astral - the people behind the ruff linter - are currently developing ty, yet another static type checker for python, with a lot of promise, but it’s going to be a couple of months, maybe a year before it is in any shape to be used in production code.