I just spent days and days fixing years of old code that had incorrect backslash escapes in string literals because they were upgraded from silent deprecationwarnings to very much not silent syntaxwarnings in 3.12 and I couldn’t find a way to shut them up.
Yes I know incorrect string literals are bad and they should be fixed, but this was legacy, well tested code and we were fine with the silent errors. But no: Python forced me to halt what I was doing and fix shit that didn’t need fixing for days, then re-release new versions of all our packages, then warn our customers, write an entry in the company’s website’s support page, blah blah blah… Stupid things that, ya know, takes a truckload of time.
And I also had to rewrite our old installers that used to install Python modules system-wide because, again, 3.12 pip now refuses to install outside of a venv unless you use --break-system-packages, and even if you do, it bitches and moan. I didn’t want to fix our old installers either, but there ya go, same thing: no way to make it revert to the old way and let old code grow old peacefully.
And I had to rewrite our Unix modules that use crypt because inexplicably, for no rational reason, it’s being dropped. WHY??? Yes it’s insecure, but if you need it for backward compatibility reasons, why remove it? Particularly since it’s still there: all you have to do is replace it with passlib.sha512. It’s no biggie, but again, when you have mountains of old code to go through, it takes time to fix it, retest it and re-release it. What’s the point in making things extra-difficult gratuitously?
And before you say “All you had to do was stick to 3.11 or below”, try installing 3.11 and below for all users in Win11: you can’t. Our users getting new Windows machine essentially have to use Python 3.12, meaning I have to fix all our old code so stuff doesn’t break or look like shit.
Sigh…
I love this language. I really do. But the Python people essentially give zero shit about backward-compatibility and every 6 months to 2 years, some small things break that becomes a massive pain in the rear-end.
And as a result, guess what: as long as whatever version of Python I’m currently using works, I am NOT upgrading it, because I don’t like to have to work overtime for nothing. And that’s not great because Python users should want to get the latest and greatest instead of fearing what new things will get broken.
The problem is, our code has to run on a customer install regardless of the install. Typically they downloaded Python and installed it themselves on their machine, and more often than not, they have stuff running on their machines and modules installed for their own stuff, and I can’t go about telling them to install some other version and/or force them to use a certain tool to do so: it’s gonna break their own config and it’s going to piss them off.
Our code has to run clean without fuss on Linux and Windows, regardless of what’s installed and how. All we require is any Python version between 3.6 and (now…) 3.12. We tell them they can use another version if they want but then they’re on their own if there are problems. We guarantee proper operation between 3.6 and 3.12. I just had to make sure our installers worked properly and performed a venv install when appropriate (almost required with 3.12 now) and our code ran without warning and without throwing a wobbler because of missing stuff that was there before 3.12 because I don’t want to tell people to spend extra efforts to downgrade and/or mess up their install on Win11.
I’m just now thinking out los here, but would it make sense to use a PowerShell script to silently install miniconda and create a venv with a specific version?
Something like
@echo off
REM Download Miniconda installer (replace URL with the latest version)
powershell -Command "Invoke-WebRequest https://repo.anaconda.com/miniconda/Miniconda3-latest-Windows-x86_64.exe -OutFile miniconda.exe"
REM Install Miniconda silently
start /wait "" miniconda.exe /InstallationType=JustMe /RegisterPython=0 /S /D=%UserProfile%\Miniconda3
REM Create a new environment with the specific Python version
call %UserProfile%\Miniconda3\Scripts\activate.bat
call conda create -y -n py39 python=3.9
REM Optional: Set permissions for multi-user access
icacls %UserProfile%\Miniconda3 /grant:r Users:(OI)(CI)F /T
We wouldn’t do this because we have a policy of modifying as little as humanly possible on the target system, because of the nature of some of our customers: we may not have the right to modify stuff right and left willy-nilly on those customer’s systems, and if we do, we have to justify why with a lot of paperwork. So we don’t: we install our stuff in one very formal, very stable spot of the hard disk (on Windows, c:\Users<username>), install missing modules, stick an icon on the desktop and that’s it.
Historically, we also propose to install Python on the system if it’s missing (used to do that system-wide) and advise our customers to install it system-wide too if they did it themselves so we wouldn’t have to modify our installer (again, if we do, we have to justify the change when the customer is under configuration control, so we try to avoid change). Not anymore obviously.
So yeah, we wouldn’t install Miniconda just for the purpose of not changing our installers. At this point, that’s enough change that I can just rewrite the installers: either way, I’ll have to communicate the change to the customers, so I might as well do it right.
But my point was, if Python gracefully handled backward compatibility - as in, for example, you put in the shebang or near the shebang that you want the interpreter and pip to behave like, say, Python 3.10, and Python4.56 behaves like 3.10 did years later - then this is unnecessary work that I wouldn’t have to do.
I had to rewrite our installers and other things to deal with Python 3.12’s new forcibly-enforced kosher way ot doing things because it won’t do thing any other way and compatibility be damned. That’s my beef with Python: after all this time, it’s still unstable and prone to breaking existing codebases.
It’s a real problem for us and it has been for a long time: code that’s out there and working is liable to quit working when the Python interpreter changes. I have no idea how a 33-year-old language, ultra-widely used language such as Python is allowed to change regularly with little regard to backward compatibility like it was an early beta weekend project. This is truly unprofessional and it completely boggles my mind.
I agree, too little regard for backwards compatibility. They also removed distutils which meant I had to fix a load of code that used it. It was bad code that shouldn’t have used it even when written, but still… seems like they didn’t learn their lesson from Python 2.
It’s not like it would be difficult to avoid these issues either. Everyone else just makes you declare your “target version” and then the runtime keeps things compatible with that version - Android via SDK target version, Rust with its editions, hell even CMake got this right. CMake!!
Yes of course. If I run pyflakes or mypy on our code, it’s a complete shitshow. But that’s not the point.
The point is, for better or worse, however imperfect our code is, it run cleanly and predictably in older Python interpreters. When I have to correct legacy stuff that is known to work well, I compromise hundreds of hours of formal and informal testing.
Imperfect code that has been running flawlessly for a long time and has proven its reliability is better than more perfect code that hasn’t been tested as much.
In fact, in certain industries like the aero industry, it doesn’t matter if you find slightly bad code after the system has been certified: it’s frozen and you leave it the hell alone unless it’s critical. Fortunately we’re not exactly in that situation, but we do have customers who require - and pay for - configuration control, and those Python issues kind of make everything more difficult needlessly for us. Lucky for us, our Python packages are mostly support code, so it’s not too critical. But we do have to be careful and thorough.
I just spent days and days fixing years of old code that had incorrect backslash escapes in string literals because they were upgraded from silent deprecationwarnings to very much not silent syntaxwarnings in 3.12 and I couldn’t find a way to shut them up.
Yes I know incorrect string literals are bad and they should be fixed, but this was legacy, well tested code and we were fine with the silent errors. But no: Python forced me to halt what I was doing and fix shit that didn’t need fixing for days, then re-release new versions of all our packages, then warn our customers, write an entry in the company’s website’s support page, blah blah blah… Stupid things that, ya know, takes a truckload of time.
And I also had to rewrite our old installers that used to install Python modules system-wide because, again, 3.12 pip now refuses to install outside of a venv unless you use --break-system-packages, and even if you do, it bitches and moan. I didn’t want to fix our old installers either, but there ya go, same thing: no way to make it revert to the old way and let old code grow old peacefully.
And I had to rewrite our Unix modules that use crypt because inexplicably, for no rational reason, it’s being dropped. WHY??? Yes it’s insecure, but if you need it for backward compatibility reasons, why remove it? Particularly since it’s still there: all you have to do is replace it with passlib.sha512. It’s no biggie, but again, when you have mountains of old code to go through, it takes time to fix it, retest it and re-release it. What’s the point in making things extra-difficult gratuitously?
And before you say “All you had to do was stick to 3.11 or below”, try installing 3.11 and below for all users in Win11: you can’t. Our users getting new Windows machine essentially have to use Python 3.12, meaning I have to fix all our old code so stuff doesn’t break or look like shit.
Sigh…
I love this language. I really do. But the Python people essentially give zero shit about backward-compatibility and every 6 months to 2 years, some small things break that becomes a massive pain in the rear-end.
And as a result, guess what: as long as whatever version of Python I’m currently using works, I am NOT upgrading it, because I don’t like to have to work overtime for nothing. And that’s not great because Python users should want to get the latest and greatest instead of fearing what new things will get broken.
I get your points!
But just out of curiosity, did you try using miniconda to install older python versions? That works wonders for me, also on windows 11.
The problem is, our code has to run on a customer install regardless of the install. Typically they downloaded Python and installed it themselves on their machine, and more often than not, they have stuff running on their machines and modules installed for their own stuff, and I can’t go about telling them to install some other version and/or force them to use a certain tool to do so: it’s gonna break their own config and it’s going to piss them off.
Our code has to run clean without fuss on Linux and Windows, regardless of what’s installed and how. All we require is any Python version between 3.6 and (now…) 3.12. We tell them they can use another version if they want but then they’re on their own if there are problems. We guarantee proper operation between 3.6 and 3.12. I just had to make sure our installers worked properly and performed a venv install when appropriate (almost required with 3.12 now) and our code ran without warning and without throwing a wobbler because of missing stuff that was there before 3.12 because I don’t want to tell people to spend extra efforts to downgrade and/or mess up their install on Win11.
Ok. Thank you for the explanation!
I’m just now thinking out los here, but would it make sense to use a PowerShell script to silently install miniconda and create a venv with a specific version?
Something like
More on that here https://docs.anaconda.com/anaconda/install/silent-mode/
Again, this is just an idea, but if this works then you won’t have a problem anymore (maybe?).
Well that’s one solution.
We wouldn’t do this because we have a policy of modifying as little as humanly possible on the target system, because of the nature of some of our customers: we may not have the right to modify stuff right and left willy-nilly on those customer’s systems, and if we do, we have to justify why with a lot of paperwork. So we don’t: we install our stuff in one very formal, very stable spot of the hard disk (on Windows, c:\Users<username>), install missing modules, stick an icon on the desktop and that’s it.
Historically, we also propose to install Python on the system if it’s missing (used to do that system-wide) and advise our customers to install it system-wide too if they did it themselves so we wouldn’t have to modify our installer (again, if we do, we have to justify the change when the customer is under configuration control, so we try to avoid change). Not anymore obviously.
So yeah, we wouldn’t install Miniconda just for the purpose of not changing our installers. At this point, that’s enough change that I can just rewrite the installers: either way, I’ll have to communicate the change to the customers, so I might as well do it right.
But my point was, if Python gracefully handled backward compatibility - as in, for example, you put in the shebang or near the shebang that you want the interpreter and pip to behave like, say, Python 3.10, and Python4.56 behaves like 3.10 did years later - then this is unnecessary work that I wouldn’t have to do.
I had to rewrite our installers and other things to deal with Python 3.12’s new forcibly-enforced kosher way ot doing things because it won’t do thing any other way and compatibility be damned. That’s my beef with Python: after all this time, it’s still unstable and prone to breaking existing codebases.
It’s a real problem for us and it has been for a long time: code that’s out there and working is liable to quit working when the Python interpreter changes. I have no idea how a 33-year-old language, ultra-widely used language such as Python is allowed to change regularly with little regard to backward compatibility like it was an early beta weekend project. This is truly unprofessional and it completely boggles my mind.
I agree, too little regard for backwards compatibility. They also removed distutils which meant I had to fix a load of code that used it. It was bad code that shouldn’t have used it even when written, but still… seems like they didn’t learn their lesson from Python 2.
It’s not like it would be difficult to avoid these issues either. Everyone else just makes you declare your “target version” and then the runtime keeps things compatible with that version - Android via SDK target version, Rust with its editions, hell even CMake got this right. CMake!!
Oh yeah, I forgot about the distutils thing. Yeah, I had to fix that too. But at least it wasn’t hours of wading through and retesting old code.
In fairness for the invalid escape sequence thing static linters (Pylint, Pyright, etc.) should be already telling you about it.
Yes of course. If I run pyflakes or mypy on our code, it’s a complete shitshow. But that’s not the point.
The point is, for better or worse, however imperfect our code is, it run cleanly and predictably in older Python interpreters. When I have to correct legacy stuff that is known to work well, I compromise hundreds of hours of formal and informal testing.
Imperfect code that has been running flawlessly for a long time and has proven its reliability is better than more perfect code that hasn’t been tested as much.
In fact, in certain industries like the aero industry, it doesn’t matter if you find slightly bad code after the system has been certified: it’s frozen and you leave it the hell alone unless it’s critical. Fortunately we’re not exactly in that situation, but we do have customers who require - and pay for - configuration control, and those Python issues kind of make everything more difficult needlessly for us. Lucky for us, our Python packages are mostly support code, so it’s not too critical. But we do have to be careful and thorough.