This repository was archived by the owner on Mar 3, 2023. It is now read-only.
Repository navigation
This repository was archived by the owner on Mar 3, 2023. It is now read-only.
Migrate Python dependency From Python 2 to Python 3 #20585
Description
Summary
Python 2 is being deprecated by Debian and Ubuntu, in favor of Python 3, in a move toward not shipping Python 2 at all. (Python 2 is already End-Of-Life'd by the Python maintainers *). In the long term, it will help Atom stay installable/working on these distros to specify a Python 3 dependency (in addition to or instead of Python 2). Even longer-term, Python 2 will no-longer be available, so it will make sense to drop it entirely. If Python 2 gives Atom any trouble in the medium-term, it can simply be dropped.
Motivation
Doing this keeps Atom installable and working on newer distros/operating systems.
Furthermore, making sure the build process doesn't require Python 2 makes it easy to continue building Atom once Python 2 is End-of-Life'd.
Describe alternatives you've considered
- Un-bundle
node-gypand remove any other usage of Python from within this package. - OR: Allow users to install Atom on all platforms without requiring Python on their system.
Additional context
- According to this comment, Python is only needed for node-gyp.
- See Fix .deb install on ubuntu 20.04 #20406 for some prior discussion, see Can't install the .deb package in Ubuntu 20.04 (Focal Fossa) development branch (due to changes in Ubuntu's python packaging) #20356 for lots more discussion. These are examples of the kinds of packaging troubles that might arise relating to the Python 2 EOL and transition to Python 3.
- I am personally more of a Debian/Ubuntu person myself, but this applies to all OSes and platforms.
Reacted by Peter J. Mello, MR-no-names, Varac, Alexander Adam, shioko, Kevin, Douglas Silva and Aptrug
Reacted by shioko
Reacted by shioko
Reacted by shioko
Activity
DeeDeeG commentedon Apr 3, 2020
Info for Fedora (RPM package): I don't think we have to worry about this for Fedora/RPM-based distros.
The current RPM package doesn't even list python in its dependencies:
Furthermore: the default package manager for Fedora,
dnf, is written in Python. Python 3 is installed out-of-the-box on recent Fedora, and can't be uninstalled, becausednfrefuses to uninstall itself (or by extension, its own dependencies). So we can pretty safely assume Python 3 is available on supported Fedora releases.(As of writing this, Fedora 30 and 31 are still active/supported. Fedora 32 will be out April 20th. Judging by past releases, Fedora 30 will probably become EOL about a month later/late May.)
DeeDeeG commentedon Apr 3, 2020
I found a sort of "minimal reproducer" to invoke usage of Python.
atominstalled.apm install tree-view.gypwhen being built/installed.)This will error out if your python setup is inadequate for running
gyp, with an error like this:DeeDeeG commentedon Apr 3, 2020
Full error message (click to expand)
`atom --version` output (click to expand)
`apm --version` output (click to expand)
DeeDeeG commentedon Apr 3, 2020
Short summary: We need newer
node-gyp. (The current dependencies foratominclude a version ofnode-gypthat is too old to support Python 3.)I get the impression that Python really is only needed for
apm, which in turn only needs it fornode-gyp.(If anyone knows this to be wrong, please do let me know/post a correction in this issue! It's a critical assumption I'm making, in order to narrow down the path forward toward get atom migrated fully to Python 3.)
apmbundles its ownnodeandnpmversions. (apmdepends on a pinnednpmversion.)npmin turn depends on a version range ofnode-gyp.There is a sort of "dependency chain" beyond the
atomrepo that must be traversed, making sure the whole chain supports Python 3:atom>apm>npm>node-gypThese are some pretty old bundled versions.
apm@2.4.5(andapm'smasterbranch on GitHub) specify npm@6.2.0npm@6.2.0specifies node-gyp@^3.7.0.npm'slatestbranch (currentlynpm@6.14.4) specifies node-gyp@^5.1.0.npmon the WIP 7.x beta branch: node-gyp@^5.1.0node-gyponly works withpython3"by default" (without an "experimental" CLI flag used) starting with version 6.0: https://github.com/nodejs/node-gyp/blob/master/CHANGELOG.md#v600-2019-10-04 [Edit: Or the latest 5.x series release, 5.1.0. This has Python 3 support backported.]Edit:
npm6.14.2 or newer depends onnode-gyp5.1.0, which has Python 3 support backported. We should depend onnpm6.14.2 or newer inapm.Bottom line: We need newer
node-gypthat supports Python 3.DeeDeeG commentedon Apr 6, 2020
See efforts over at atom/apm#875
I believe that PR is already working, but the tests at that repo need updating if the PR is going to pass tests/CI.
(Regarding my previous comment: This seems easier to solve than I had assumed. The only thing required at the
apmrepo turned out to be bumpingnpm.Background info:
The latest
npmreleases, as it turns out, already depend onnode-gypnew enough to supportpython3. See: npm/cli#927 and npm/cli#771More background on
node-gypPython 3 support: Crucially,node-gyp5.1.0 was released after the most recentnode-gyp6.x release. 5.1.0 also supports Python 3. It's just different fromnode-gyp6.x in that 5.1.0 usespython2preferentially overpython3in some cases. (node-gyp6.x preferspython3in those cases.) The PR I linked to at the beginning of this comment upgrades apm'snpmdependency to a version that depends onnode-gyp5.1.0... So the only thing required atapmturned out to be bumpingnpm. And... updating the CI tests at that repo. The only thing required at this repo, then, would be bumpingapmto a new version once atom/apm#875 or similar lands atapm.)DeeDeeG commentedon Apr 6, 2020
By the way, none of the other platforms' Atom installers appear to require Python in any way, shape or form. They install even if there is no Python on your system.
Only the Debian/Ubuntu (
.deb) packages seem to "require" a Python dependency.Edit to add: Not having Python on the system presumably breaks
apm... Maybe the installers for other platforms bundle a Python interpreter? I know macOS ships with Python (2.7.x still!!!!), so it probably isn't needed there. And Fedora, well... Fedora ships with Python 3... Hmm. (apmwould probably be broken by default on Fedora, then?) But I'm not aware what is being done for Windows.Have to check into all of these.
DeeDeeG commentedon Apr 18, 2020
Can confirm the Windows packaging doesn't include Python.
Zipped apm
apm -v:Error message when trying to
apm install tree-view(no Python):Full error message (click to expand):
Installed apm
apm -v:Installed apm, error message when doing
apm install tree-view:Full error message (click to expand):
Miscellaneous note: I also noticed a package that won't install without Git on the system.
Final note: many packages can be successfully installed without using node-gyp (and thus Python) nor using Git, so there is some lee-way regarding if either of those should be bundled, or if they should be considered install-time dependencies at all. Maybe bundling or requiring Python on the system at install-time for Atom is heavy-handed?
DeeDeeG commentedon Apr 25, 2020
I have had the chance to confirm the macOS app doesn't bundle Python with it, nor does it seem to check whether there is a certain version of Python on the system.
So that leaves the Debian/Ubuntu (
.deb) package as the only one currently specifying a Python dependency.As mentioned in the other thread, IMO this could be made an optional dependency, or dropped, in the
.debpackage.DeeDeeG commentedon Apr 25, 2020
Regarding Fedora: I found that the current
apmwith oldnode-gypis unlikely to work, broadly speaking, on modern Fedora.Python 3 is the default
pythonthere; To useapmon Fedora 31, one has to remove the symlink frompython->python3, and then manually symlinkpython->python2. There is no package that does this for you (unlike on Ubuntu 20.04 or Debian Buster), so it has to be done by hand.IMO, the elegant solution (good for all OSes) is to support both Python 2 and Python 3 in
apm. Once again, see: atom/apm#875Until then there is no packaging we can do to have
apmwork out of the box on Fedora. (As far as I can tell.)DeeDeeG commentedon May 4, 2020
atom/apm#875 --> (now atom/apm#887) is almost done.
The only things required after that, here at this repo (
atom/atom), are to:apmto a version having Upgrade node gyp with fixes apm#887 merged. (Updateapm/package.jsonin this repo.)npmto 6.14.2 or newer inscript/package.jsonin this repo..deband/or.rpmpackages to reflect Python 3 support.Proof-of-concept branch: https://github.com/DeeDeeG/atom/commits/update-apm-test
DeeDeeG commentedon May 13, 2020
Update: There are a few smaller things left to do, actually.Some of these things are beyond my skill level at the moment (particularly, I'm struggling with asynchronous Javascript/race conditions, and I don't know how to use callbacks properly at the moment), but I'll get to them as and when I get the time. (Starting with the ones I know how to fix, like simple documentation updates...) Help wanted/would be much appreciated!Edit: Done and ready for review.Edit 2: Merged: #20856
Copy-pasting from here: #20670 (comment)
Actually, I did find some parts of this repo that could be updated to reflect Python 3. Here are my findings looking at all files that mention "python" (case-insensitive search):
Not related to actual use of the
pythoninterpreter/executable by Atom (so, no changes needed):Click to expand.
language-python, a package that "Adds syntax highlighting and snippets to Python files in Atom.")language-pythonagain.)language-python)language-pythonandtree-sitter-python, which are both related to Python syntax highlighting.)language-pythonagain)language-pythonandtree-sitter-pythonagain. Test-related anyhow, not built/used in the final Atom product for end-users.)pythonandpython-function, respectively.)Actually related to
pythonusage (with suggested changes):pythonpackage on Ubuntu. Maybe this should be updated to suggestpython3orpython2ifpythonisn't available.)pythondependency in the.debpackage for Debian/Ubuntu of Atom. Debating whether to drop this/make it optional, update it, etc.)lintian) that checks for common issues with.debpackages. But it's a very "sensitive" linter and IMO has lots of false positives/noise. No-one has been running this linter apparently, so maybe remove the metadata for it?)python3-aware, or ready for the split ofpythonintopython2andpython3on Linux. Should probably be updated.)For some reason, thisverifyPythoncheck is only being run on Windows??Need to re-write this test if it is going to work against Python 3. Help wanted/appreciated!node-gyp's responsibility to continue to support Python 2 and 3 properly. Continuing to test against just Python 2 could be okay, I suppose.DeeDeeG commentedon May 13, 2020
DeeDeeG commentedon May 27, 2020
Update:
For Ubuntu 20.10As of Ubuntu 19.10, Canonical have moved every form of Python 2 to the "universe" component of the repositories. (Meaning, "community-maintained," as opposed to "Canonical-supported" -- Canonical being the company that runs Ubuntu. Per: https://help.ubuntu.com/community/Repositories/Ubuntu)Tried to install Python 2 in an Ubuntu 20.10 VM (click to expand).
None of these can be installed anymore:
Doesn't mean end-users won't be able to install Python 2, just that Canonical are serious about demoting it from being officially "supported." Now that we support Python 3, I think Python 3 should be the default Python we target, where that makes sense on a technical basis.
DeeDeeG commentedon Jul 10, 2020
#20856 is merged! 🎉
The very last thing here is to update the Docs, "Hacking on Atom Core" to reflect Python 3 support.
See: atom/flight-manual.atom.io#630
DeeDeeG commentedon Nov 14, 2020
Docs were updated (atom/flight-manual.atom.io#630). Atom is Python-3-ready/compatible.
Happened a while ago, just now getting around to closing this.