From a275cb9fc7ec616df8a64a258e0cf55bc927e999 Mon Sep 17 00:00:00 2001 From: Inada Naoki Date: Thu, 12 Dec 2019 15:21:26 +0900 Subject: [PATCH 01/33] Use utf-8 to read README On Windows, default file encoding is not UTF-8 in most environments. --- source/tutorials/packaging-projects.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/source/tutorials/packaging-projects.rst b/source/tutorials/packaging-projects.rst index fb3a53934..a17cc0ef2 100644 --- a/source/tutorials/packaging-projects.rst +++ b/source/tutorials/packaging-projects.rst @@ -62,7 +62,7 @@ Open :file:`setup.py` and enter the following content. Update the package name t import setuptools - with open("README.md", "r") as fh: + with open("README.md", "r", encoding="utf-8") as fh: long_description = fh.read() setuptools.setup( From d8c125bcebcb8c6ec2011345308c39aa3db412c5 Mon Sep 17 00:00:00 2001 From: pastelmind Date: Fri, 13 Mar 2020 11:38:51 +0900 Subject: [PATCH 02/33] Organize index of guides into 3 categories Split the toctree of guides into three broad categories: - Installing Packages - Includes guides for installing and managing dependencies. - Building and Publishing Projects - Includes guides for project configuration, distribution - Miscellaneous - Guides that do not clearly fit into any of the above This commit primarily affects the Guides index page; it does not create nested lists for each category in the sidebar. --- source/guides/index.rst | 19 +++++++++++++++---- 1 file changed, 15 insertions(+), 4 deletions(-) diff --git a/source/guides/index.rst b/source/guides/index.rst index 9b4f89fb6..6c9b710ac 100644 --- a/source/guides/index.rst +++ b/source/guides/index.rst @@ -7,13 +7,18 @@ introduction to packaging, see :doc:`/tutorials/index`. .. toctree:: :maxdepth: 1 + :caption: Installing Packages: - tool-recommendations installing-using-pip-and-virtual-environments installing-stand-alone-command-line-tools installing-using-linux-tools installing-scientific-packages multi-version-installs + +.. toctree:: + :maxdepth: 1 + :caption: Building and Publishing Projects: + distributing-packages-using-setuptools using-manifest-in single-sourcing-package-version @@ -23,10 +28,16 @@ introduction to packaging, see :doc:`/tutorials/index`. supporting-windows-using-appveyor packaging-namespace-packages creating-and-discovering-plugins - analyzing-pypi-package-downloads - index-mirrors-and-caches - hosting-your-own-index migrating-to-pypi-org using-testpypi making-a-pypi-friendly-readme publishing-package-distribution-releases-using-github-actions-ci-cd-workflows + +.. toctree:: + :maxdepth: 1 + :caption: Miscellaneous: + + tool-recommendations + analyzing-pypi-package-downloads + index-mirrors-and-caches + hosting-your-own-index From cb87c0141a214fdd35cde490d454395c7189b3b1 Mon Sep 17 00:00:00 2001 From: Ben Nuttall Date: Fri, 25 Sep 2020 10:51:25 +0100 Subject: [PATCH 03/33] Remove Requires-Python note from piwheels project description --- source/key_projects.rst | 9 +-------- 1 file changed, 1 insertion(+), 8 deletions(-) diff --git a/source/key_projects.rst b/source/key_projects.rst index cd9f93543..8b84e4088 100644 --- a/source/key_projects.rst +++ b/source/key_projects.rst @@ -517,16 +517,9 @@ piwheels piwheels is a website, and software underpinning it, that fetches source code distribution packages from PyPI and compiles them into binary wheels that are optimized for installation onto Raspberry Pi -computers. pip in Raspbian is pre-configured to use piwheels.org as +computers. Raspberry Pi OS pre-configures pip to use piwheels.org as an additional index to PyPI. - .. warning:: - - Note that piwheels `does not yet fully support - `__ :pep:`503` and - thus some users have trouble installing certain wheels; this is in - progress. - .. _poetry: poetry From b7fff2a2fbeab77768eafdd4f9913cd34b3157fd Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=E8=B0=AD=E4=B9=9D=E9=BC=8E?= <109224573@qq.com> Date: Tue, 6 Oct 2020 16:08:17 +0800 Subject: [PATCH 04/33] fix setuptools entry_point link --- source/overview.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/source/overview.rst b/source/overview.rst index c82f46591..e1beeb45b 100644 --- a/source/overview.rst +++ b/source/overview.rst @@ -167,7 +167,7 @@ Python's native packaging is mostly built for distributing reusable code, called libraries, between developers. You can piggyback **tools**, or basic applications for developers, on top of Python's library packaging, using technologies like `setuptools entry_points -`_. +`_. Libraries are building blocks, not complete applications. For distributing applications, there's a whole new world of technologies From ea7641e17698a4c2c914a7de9dcebb111d632248 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=E8=B0=AD=E4=B9=9D=E9=BC=8E?= <109224573@qq.com> Date: Tue, 6 Oct 2020 16:22:25 +0800 Subject: [PATCH 05/33] fix cx_Freeze link --- source/overview.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/source/overview.rst b/source/overview.rst index e1beeb45b..0c9d100f2 100644 --- a/source/overview.rst +++ b/source/overview.rst @@ -315,7 +315,7 @@ and a good amount of effort. A selection of Python freezers: * `pyInstaller `_ - Cross-platform -* `cx_Freeze `_ - Cross-platform +* `cx_Freeze `_ - Cross-platform * `constructor `_ - For command-line installers * `py2exe `_ - Windows only * `py2app `_ - Mac only From 22f77846bb80bd67aafecc954ec47193e3d047e6 Mon Sep 17 00:00:00 2001 From: Matheus Felipe <50463866+matheusfelipeog@users.noreply.github.com> Date: Fri, 23 Oct 2020 14:12:53 -0300 Subject: [PATCH 06/33] Fix sphinx command and docs directory text --- source/tutorials/creating-documentation.rst | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/source/tutorials/creating-documentation.rst b/source/tutorials/creating-documentation.rst index a2c0a7a69..6bf38e2db 100644 --- a/source/tutorials/creating-documentation.rst +++ b/source/tutorials/creating-documentation.rst @@ -24,14 +24,14 @@ For other installation methods, see this `installation guide`_ by Sphinx. Getting Started With Sphinx --------------------------- -Create a ``doc`` directory inside your project to hold your documentation: +Create a ``docs`` directory inside your project to hold your documentation: .. code-block:: bash cd /path/to/project mkdir docs -Run ``spinx-quickstart`` inside the ``docs`` directory: +Run ``sphinx-quickstart`` inside the ``docs`` directory: .. code-block:: bash From 9b1b421da77461a12b3d5d4a68f538fa183e671b Mon Sep 17 00:00:00 2001 From: Tim Swast Date: Tue, 27 Oct 2020 13:28:34 -0500 Subject: [PATCH 07/33] update PyPI package download queries to use file_downloads table --- .../analyzing-pypi-package-downloads.rst | 161 ++++++++++-------- 1 file changed, 91 insertions(+), 70 deletions(-) diff --git a/source/guides/analyzing-pypi-package-downloads.rst b/source/guides/analyzing-pypi-package-downloads.rst index 2ee84c858..386ce2849 100644 --- a/source/guides/analyzing-pypi-package-downloads.rst +++ b/source/guides/analyzing-pypi-package-downloads.rst @@ -68,23 +68,25 @@ the `BigQuery quickstart guide Data schema ----------- -Linehaul writes an entry in a ``the-psf.pypi.downloadsYYYYMMDD`` table for each +Linehaul writes an entry in a ``the-psf.pypi.file_downloads`` table for each download. The table contains information about what file was downloaded and how it was downloaded. Some useful columns from the `table schema -`__ +`__ include: -+------------------------+-----------------+-----------------------+ -| Column | Description | Examples | -+========================+=================+=======================+ -| file.project | Project name | ``pipenv``, ``nose`` | -+------------------------+-----------------+-----------------------+ -| file.version | Package version | ``0.1.6``, ``1.4.2`` | -+------------------------+-----------------+-----------------------+ -| details.installer.name | Installer | pip, `bandersnatch`_ | -+------------------------+-----------------+-----------------------+ -| details.python | Python version | ``2.7.12``, ``3.6.4`` | -+------------------------+-----------------+-----------------------+ ++------------------------+-----------------+-----------------------------+ +| Column | Description | Examples | ++========================+=================+=============================+ +| timestamp | Date and time | ``2020-03-09 00:33:03 UTC`` | ++------------------------+-----------------+-----------------------------+ +| file.project | Project name | ``pipenv``, ``nose`` | ++------------------------+-----------------+-----------------------------+ +| file.version | Package version | ``0.1.6``, ``1.4.2`` | ++------------------------+-----------------+-----------------------------+ +| details.installer.name | Installer | pip, `bandersnatch`_ | ++------------------------+-----------------+-----------------------------+ +| details.python | Python version | ``2.7.12``, ``3.6.4`` | ++------------------------+-----------------+-----------------------------+ Useful queries @@ -92,11 +94,9 @@ Useful queries Run queries in the `BigQuery web UI`_ by clicking the "Compose query" button. -Note that the rows are stored in separate tables for each day, which helps +Note that the rows are stored in a partitioned, which helps limit the cost of queries. These example queries analyze downloads from -recent history by using `wildcard tables -`__ to -select all tables and then filter by ``_TABLE_SUFFIX``. +recent history by filtering on the ``timestamp`` column. Counting package downloads ~~~~~~~~~~~~~~~~~~~~~~~~~~ @@ -108,18 +108,17 @@ The following query counts the total number of downloads for the project #standardSQL SELECT COUNT(*) AS num_downloads - FROM `the-psf.pypi.downloads*` + FROM `the-psf.pypi.file_downloads` WHERE file.project = 'pytest' -- Only query the last 30 days of history - AND _TABLE_SUFFIX - BETWEEN FORMAT_DATE( - '%Y%m%d', DATE_SUB(CURRENT_DATE(), INTERVAL 30 DAY)) - AND FORMAT_DATE('%Y%m%d', CURRENT_DATE()) + AND DATE(timestamp) + BETWEEN DATE_SUB(CURRENT_DATE(), INTERVAL 30 DAY) + AND CURRENT_DATE()) +---------------+ | num_downloads | +===============+ -| 2117807 | +| 20531925 | +---------------+ To only count downloads from pip, filter on the ``details.installer.name`` @@ -129,71 +128,94 @@ column. #standardSQL SELECT COUNT(*) AS num_downloads - FROM `the-psf.pypi.downloads*` + FROM `the-psf.pypi.file_downloads` WHERE file.project = 'pytest' AND details.installer.name = 'pip' -- Only query the last 30 days of history - AND _TABLE_SUFFIX - BETWEEN FORMAT_DATE( - '%Y%m%d', DATE_SUB(CURRENT_DATE(), INTERVAL 30 DAY)) - AND FORMAT_DATE('%Y%m%d', CURRENT_DATE()) + AND DATE(timestamp) + BETWEEN DATE_SUB(CURRENT_DATE(), INTERVAL 30 DAY) + AND CURRENT_DATE()) +---------------+ | num_downloads | +===============+ -| 1829322 | +| 19391645 | +---------------+ Package downloads over time ~~~~~~~~~~~~~~~~~~~~~~~~~~~ -To group by monthly downloads, use the ``_TABLE_SUFFIX`` pseudo-column. Also -use the pseudo-column to limit the tables queried and the corresponding -costs. +To group by monthly downloads, use the ``TIMESTAMP_TRUNC`` function. Also +filtering by this column reduces corresponding costs. (Warning: This query +processes over 500 GB of data.) :: #standardSQL SELECT COUNT(*) AS num_downloads, - SUBSTR(_TABLE_SUFFIX, 1, 6) AS `month` - FROM `the-psf.pypi.downloads*` + DATE_TRUNC(DATE(timestamp), MONTH) AS `month` + FROM `the-psf.pypi.file_downloads` WHERE file.project = 'pytest' -- Only query the last 6 months of history - AND _TABLE_SUFFIX - BETWEEN FORMAT_DATE( - '%Y%m01', DATE_SUB(CURRENT_DATE(), INTERVAL 6 MONTH)) - AND FORMAT_DATE('%Y%m%d', CURRENT_DATE()) + AND DATE(timestamp) + BETWEEN DATE_TRUNC(DATE_SUB(CURRENT_DATE(), INTERVAL 6 MONTH), MONTH) + AND CURRENT_DATE() GROUP BY `month` ORDER BY `month` DESC -+---------------+--------+ -| num_downloads | month | -+===============+========+ -| 1956741 | 201801 | -+---------------+--------+ -| 2344692 | 201712 | -+---------------+--------+ -| 1730398 | 201711 | -+---------------+--------+ -| 2047310 | 201710 | -+---------------+--------+ -| 1744443 | 201709 | -+---------------+--------+ -| 1916952 | 201708 | -+---------------+--------+ - -More queries -~~~~~~~~~~~~ - -- `Data driven decisions using PyPI download statistics - `__ -- `PyPI queries gist `__ -- `Python versions over time - `__ -- `Non-Windows downloads, grouped by platform - `__ ++---------------+------------+ +| num_downloads | month | ++===============+============+ +| 1956741 | 2018-01-01 | ++---------------+------------+ +| 2344692 | 2017-12-01 | ++---------------+------------+ +| 1730398 | 2017-11-01 | ++---------------+------------+ +| 2047310 | 2017-10-01 | ++---------------+------------+ +| 1744443 | 2017-09-01 | ++---------------+------------+ +| 1916952 | 2017-08-01 | ++---------------+------------+ + +Python versions over time +~~~~~~~~~~~~~~~~~~~~~~~~~ + +Extract the Python version from the ``details.python`` column. + +:: + + #standardSQL + SELECT + REGEXP_EXTRACT(details.python, r"[0-9]+\.[0-9]+") AS python_version, + COUNT(*) AS num_downloads, + FROM `the-psf.pypi.file_downloads` + WHERE + -- Only query the last 6 months of history + DATE(timestamp) + BETWEEN DATE_TRUNC(DATE_SUB(CURRENT_DATE(), INTERVAL 6 MONTH), MONTH) + AND CURRENT_DATE() + GROUP BY `python_version` + ORDER BY `num_downloads` DESC + ++--------+---------------+ +| python | num_downloads | ++========+===============+ +| 3.7 | 12990683561 | ++--------+---------------+ +| 3.6 | 9035598511 | ++--------+---------------+ +| 2.7 | 8467785320 | ++--------+---------------+ +| 3.8 | 4581627740 | ++--------+---------------+ +| 3.5 | 2412533601 | ++--------+---------------+ +| null | 1641456718 | ++--------+---------------+ Caveats ======= @@ -229,13 +251,12 @@ the official Python client library for BigQuery. query_job = client.query(""" SELECT COUNT(*) AS num_downloads - FROM `the-psf.pypi.downloads*` + FROM `the-psf.pypi.file_downloads` WHERE file.project = 'pytest' - -- Only query the last 30 days of history - AND _TABLE_SUFFIX - BETWEEN FORMAT_DATE( - '%Y%m%d', DATE_SUB(CURRENT_DATE(), INTERVAL 30 DAY)) - AND FORMAT_DATE('%Y%m%d', CURRENT_DATE())""") + -- Only query the last 30 days of history + AND DATE(timestamp) + BETWEEN DATE_SUB(CURRENT_DATE(), INTERVAL 30 DAY) + AND CURRENT_DATE()""") results = query_job.result() # Waits for job to complete. for row in results: From 3f444395ce00892fcb5f5925d19c139bd536713d Mon Sep 17 00:00:00 2001 From: Tim Swast Date: Tue, 27 Oct 2020 13:32:01 -0500 Subject: [PATCH 08/33] move warning to correct section --- source/guides/analyzing-pypi-package-downloads.rst | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/source/guides/analyzing-pypi-package-downloads.rst b/source/guides/analyzing-pypi-package-downloads.rst index 386ce2849..d89115e85 100644 --- a/source/guides/analyzing-pypi-package-downloads.rst +++ b/source/guides/analyzing-pypi-package-downloads.rst @@ -146,8 +146,7 @@ Package downloads over time ~~~~~~~~~~~~~~~~~~~~~~~~~~~ To group by monthly downloads, use the ``TIMESTAMP_TRUNC`` function. Also -filtering by this column reduces corresponding costs. (Warning: This query -processes over 500 GB of data.) +filtering by this column reduces corresponding costs. :: @@ -184,7 +183,8 @@ processes over 500 GB of data.) Python versions over time ~~~~~~~~~~~~~~~~~~~~~~~~~ -Extract the Python version from the ``details.python`` column. +Extract the Python version from the ``details.python`` column. Warning: This +query processes over 500 GB of data. :: From d3f138b99456fc37bec1d5044a421540ca04f68a Mon Sep 17 00:00:00 2001 From: Paul Moore Date: Sat, 31 Oct 2020 15:07:34 +0000 Subject: [PATCH 09/33] Add sdist format specification and metadata field (PEP 643) --- .../binary-distribution-format.rst | 8 ++++ source/specifications/core-metadata.rst | 33 ++++++++++--- .../specifications/distribution-formats.rst | 25 ---------- source/specifications/index.rst | 11 ++++- .../source-distribution-format.rst | 48 +++++++++++++++++++ 5 files changed, 93 insertions(+), 32 deletions(-) create mode 100644 source/specifications/binary-distribution-format.rst delete mode 100644 source/specifications/distribution-formats.rst create mode 100644 source/specifications/source-distribution-format.rst diff --git a/source/specifications/binary-distribution-format.rst b/source/specifications/binary-distribution-format.rst new file mode 100644 index 000000000..587b4ff3b --- /dev/null +++ b/source/specifications/binary-distribution-format.rst @@ -0,0 +1,8 @@ + +.. _binary-distribution-format: + +========================== +Binary distribution format +========================== + +The binary distribution format (``wheel``) is defined in :pep:`427`. diff --git a/source/specifications/core-metadata.rst b/source/specifications/core-metadata.rst index 3f3e4bcf7..529ad642e 100644 --- a/source/specifications/core-metadata.rst +++ b/source/specifications/core-metadata.rst @@ -4,10 +4,6 @@ Core metadata specifications ============================ -The current core metadata file format, version 2.1, is specified in :pep:`566`. -It defines the following specification as the canonical source for the core -metadata file format. - Fields defined in the following specification should be considered valid, complete and not subject to change. The required fields are: @@ -32,7 +28,8 @@ Metadata-Version .. versionadded:: 1.0 -Version of the file format; legal values are "1.0", "1.1", "1.2" and "2.1". +Version of the file format; legal values are "1.0", "1.1", "1.2", "2.1" +and "2.2". Automated tools consuming metadata SHOULD warn if ``metadata_version`` is greater than the highest version they support, and MUST fail if @@ -46,7 +43,7 @@ all of the needed fields. Example:: - Metadata-Version: 2.1 + Metadata-Version: 2.2 Name @@ -82,6 +79,30 @@ Example:: Version: 1.0a2 +Dynamic (multiple use) +====================== + +.. versionadded:: 2.2 + +Only permitted in source distribution (sdist) file metadata. + +A string containing the name of another core metadata field. Only the following +metadata field names may be used as the value of the ``Dynamic`` field. + +* ``Platform`` +* ``Supported-Platform`` +* ``Requires-Dist`` +* ``Requires-External`` +* ``Provides-Extra`` +* ``Provides-Dist`` +* ``Obsoletes-Dist`` + +Unless specified via ``Dynamic``, *all* metadata in a wheel must be identical to +the metadata of the sdist from which the wheel was built. If a field is named in +``Dynamic`` in the sdist, then there is no constraint on what value that field +may have in wheels built from the sdist. + + Platform (multiple use) ======================= diff --git a/source/specifications/distribution-formats.rst b/source/specifications/distribution-formats.rst deleted file mode 100644 index b56fff4bd..000000000 --- a/source/specifications/distribution-formats.rst +++ /dev/null @@ -1,25 +0,0 @@ - -.. _distribution-formats: - -==================== -Distribution formats -==================== - - -Source distribution format -========================== - -The accepted style of source distribution format based -on ``pyproject.toml``, defined in :pep:`518` and adopted by :pep:`517` -has not been implemented yet. - -There is also the legacy source distribution format, implicitly defined by -the behaviour of ``distutils`` module in the standard library, -when executing ``setup.py sdist``. - -.. _binary-distribution-format: - -Binary distribution format -========================== - -The binary distribution format (``wheel``) is defined in :pep:`427`. diff --git a/source/specifications/index.rst b/source/specifications/index.rst index 8cf0f909e..4b1d656e8 100644 --- a/source/specifications/index.rst +++ b/source/specifications/index.rst @@ -19,12 +19,21 @@ Package Distribution Metadata version-specifiers dependency-specifiers declaring-build-dependencies - distribution-formats platform-compatibility-tags recording-installed-packages entry-points +Package Distribution File Formats +--------------------------------- + +.. toctree:: + :maxdepth: 1 + + source-distribution-format + binary-distribution-format + + Package Index Interfaces ------------------------ diff --git a/source/specifications/source-distribution-format.rst b/source/specifications/source-distribution-format.rst new file mode 100644 index 000000000..c507420bd --- /dev/null +++ b/source/specifications/source-distribution-format.rst @@ -0,0 +1,48 @@ + +.. _source-distribution-format: + +========================== +Source distribution format +========================== + +The current standard format of source distribution format is identified by the +presence of a ``pyproject.toml`` file in the distribution archive. The layout +of such a distribution was originally specified in :pep:`517` and is formally +documented here. + +There is also the legacy source distribution format, implicitly defined by the +behaviour of ``distutils`` module in the standard library, when executing +``setup.py sdist``. This document does not attempt to standardise this format, +except to note that if a legacy sdist contains a ``PKG-INFO`` file holding +metadata version 2.2 or later, that sdist MUST follow the rules for sdists +defined by the metadata specification. + +Source distributions are also known as *sdists* for short. + +Source distribution file name +============================= + +The file name of a sdist is not currently standardised, although the *de facto* +form is ``{name}-{version}.tar.gz``, where ``{name}`` is the canonicalized form +of the project name (see :pep:`503` for the canonicalization rules) and +``{version}`` is the project version. + +The name and version components of the filename MUST match the values stored +in the metadata contained in the file. + +Source distribution file format +=============================== + +A .tar.gz source distribution (sdist) contains a single top-level directory +called ``{name}-{version}`` (e.g. ``foo-1.0``), containing the source files of +the package. The name and version MUST match the metadata stored in the file. +This directory must also contain a ``pyproject.toml`` in the format defined in +:ref:`declaring-build-dependencies`, and a ``PKG-INFO`` file containing +metadata in the format described in the :ref:`core-metadata` specification. The +metadata MUST conform to at least version 2.2 of the metadata specification. + +No other content of a sdist is required or defined. Build systems can store +whatever information they need in the sdist to build the project. + +The tarball should use the modern POSIX.1-2001 pax tar format, which specifies +UTF-8 based file names. From 67c57a00cde4494ee0f46ddf99d39316380d1562 Mon Sep 17 00:00:00 2001 From: Paul Moore Date: Sat, 31 Oct 2020 15:15:58 +0000 Subject: [PATCH 10/33] Clarify that the constraints on wheel metadata only apply for version 2.2 and later --- source/specifications/core-metadata.rst | 3 +++ 1 file changed, 3 insertions(+) diff --git a/source/specifications/core-metadata.rst b/source/specifications/core-metadata.rst index 529ad642e..0664f9118 100644 --- a/source/specifications/core-metadata.rst +++ b/source/specifications/core-metadata.rst @@ -102,6 +102,9 @@ the metadata of the sdist from which the wheel was built. If a field is named in ``Dynamic`` in the sdist, then there is no constraint on what value that field may have in wheels built from the sdist. +If the sdist metadata version is older than version 2.2, then there are no +constraints on the metadata of wheels built from that sdist. + Platform (multiple use) ======================= From 311b35823f5f9c5ee7caf32651f7ceee49e43add Mon Sep 17 00:00:00 2001 From: Paul Moore Date: Sat, 31 Oct 2020 20:29:15 +0000 Subject: [PATCH 11/33] Fix details of sdist name format --- source/specifications/source-distribution-format.rst | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/source/specifications/source-distribution-format.rst b/source/specifications/source-distribution-format.rst index c507420bd..bd07ecd00 100644 --- a/source/specifications/source-distribution-format.rst +++ b/source/specifications/source-distribution-format.rst @@ -24,8 +24,8 @@ Source distribution file name The file name of a sdist is not currently standardised, although the *de facto* form is ``{name}-{version}.tar.gz``, where ``{name}`` is the canonicalized form -of the project name (see :pep:`503` for the canonicalization rules) and -``{version}`` is the project version. +of the project name (see :pep:`503` for the canonicalization rules) with ``-`` +characters replaced with ``_``, and ``{version}`` is the project version. The name and version components of the filename MUST match the values stored in the metadata contained in the file. From 5da5bc64d5a5a508033ba632627a0701507f0301 Mon Sep 17 00:00:00 2001 From: Paul Moore Date: Sun, 1 Nov 2020 20:51:27 +0000 Subject: [PATCH 12/33] Apply suggestions from code review Co-authored-by: Sviatoslav Sydorenko --- source/specifications/binary-distribution-format.rst | 2 +- source/specifications/source-distribution-format.rst | 8 ++++---- 2 files changed, 5 insertions(+), 5 deletions(-) diff --git a/source/specifications/binary-distribution-format.rst b/source/specifications/binary-distribution-format.rst index 587b4ff3b..a9ac1bf46 100644 --- a/source/specifications/binary-distribution-format.rst +++ b/source/specifications/binary-distribution-format.rst @@ -5,4 +5,4 @@ Binary distribution format ========================== -The binary distribution format (``wheel``) is defined in :pep:`427`. +The binary distribution format (:term:`wheel `) is defined in :pep:`427`. diff --git a/source/specifications/source-distribution-format.rst b/source/specifications/source-distribution-format.rst index bd07ecd00..a31730393 100644 --- a/source/specifications/source-distribution-format.rst +++ b/source/specifications/source-distribution-format.rst @@ -6,13 +6,13 @@ Source distribution format ========================== The current standard format of source distribution format is identified by the -presence of a ``pyproject.toml`` file in the distribution archive. The layout +presence of a :file:`pyproject.toml` file in the distribution archive. The layout of such a distribution was originally specified in :pep:`517` and is formally documented here. There is also the legacy source distribution format, implicitly defined by the behaviour of ``distutils`` module in the standard library, when executing -``setup.py sdist``. This document does not attempt to standardise this format, +:command:`setup.py sdist`. This document does not attempt to standardise this format, except to note that if a legacy sdist contains a ``PKG-INFO`` file holding metadata version 2.2 or later, that sdist MUST follow the rules for sdists defined by the metadata specification. @@ -33,10 +33,10 @@ in the metadata contained in the file. Source distribution file format =============================== -A .tar.gz source distribution (sdist) contains a single top-level directory +A ``.tar.gz`` source distribution (sdist) contains a single top-level directory called ``{name}-{version}`` (e.g. ``foo-1.0``), containing the source files of the package. The name and version MUST match the metadata stored in the file. -This directory must also contain a ``pyproject.toml`` in the format defined in +This directory must also contain a :file:`pyproject.toml` in the format defined in :ref:`declaring-build-dependencies`, and a ``PKG-INFO`` file containing metadata in the format described in the :ref:`core-metadata` specification. The metadata MUST conform to at least version 2.2 of the metadata specification. From 505ee22f0983cc01cf38c4d021ddb66a9d4b5693 Mon Sep 17 00:00:00 2001 From: Pradyun Gedam Date: Mon, 9 Nov 2020 09:34:53 +0530 Subject: [PATCH 13/33] Correctly use globaltoc.html for global ToC --- source/conf.py | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/source/conf.py b/source/conf.py index 6406fc87e..29d180e8a 100644 --- a/source/conf.py +++ b/source/conf.py @@ -196,8 +196,8 @@ # Custom sidebar templates, filenames relative to this file. html_sidebars = { - '**': ['localtoc.html', 'relations.html'], - 'index': ['localtoc.html'] + '**': ['globaltoc.html', 'relations.html'], + 'index': ['globaltoc.html'] } # Additional templates that should be rendered to pages, maps page names to From 7d00c1c32bcdc39c3e990f0f56344b4345828563 Mon Sep 17 00:00:00 2001 From: James Gerity <57360646+jgerityneurala@users.noreply.github.com> Date: Wed, 11 Nov 2020 11:31:33 -0500 Subject: [PATCH 14/33] Update link to egg format --- source/glossary.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/source/glossary.rst b/source/glossary.rst index 50278ef45..55041cbd7 100644 --- a/source/glossary.rst +++ b/source/glossary.rst @@ -44,7 +44,7 @@ Glossary A :term:`Built Distribution` format introduced by :ref:`setuptools`, which is being replaced by :term:`Wheel`. For details, see `The Internal Structure of Python Eggs - `_ and + `_ and `Python Eggs `_ Extension Module From 51b19db230f68c399ce3f92fbcc2d4b47c99de2a Mon Sep 17 00:00:00 2001 From: Sergey Fedoseev Date: Thu, 12 Nov 2020 10:51:05 +0500 Subject: [PATCH 15/33] fix BigQuery examples --- source/guides/analyzing-pypi-package-downloads.rst | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/source/guides/analyzing-pypi-package-downloads.rst b/source/guides/analyzing-pypi-package-downloads.rst index d89115e85..c4d53fa4d 100644 --- a/source/guides/analyzing-pypi-package-downloads.rst +++ b/source/guides/analyzing-pypi-package-downloads.rst @@ -113,7 +113,7 @@ The following query counts the total number of downloads for the project -- Only query the last 30 days of history AND DATE(timestamp) BETWEEN DATE_SUB(CURRENT_DATE(), INTERVAL 30 DAY) - AND CURRENT_DATE()) + AND CURRENT_DATE() +---------------+ | num_downloads | @@ -134,7 +134,7 @@ column. -- Only query the last 30 days of history AND DATE(timestamp) BETWEEN DATE_SUB(CURRENT_DATE(), INTERVAL 30 DAY) - AND CURRENT_DATE()) + AND CURRENT_DATE() +---------------+ | num_downloads | From 886bb6297e156e80736db734f511a906851c465a Mon Sep 17 00:00:00 2001 From: Paul Moore Date: Fri, 13 Nov 2020 18:13:14 +0000 Subject: [PATCH 16/33] Various updates for PEP 643 1. Remove the whitelist of fields allowed to be dynamic. 2. Allow Dynamic outside of source distributions. --- source/specifications/core-metadata.rst | 43 ++++++++++--------- .../source-distribution-format.rst | 8 ++-- 2 files changed, 27 insertions(+), 24 deletions(-) diff --git a/source/specifications/core-metadata.rst b/source/specifications/core-metadata.rst index 0664f9118..0e6cdbc3b 100644 --- a/source/specifications/core-metadata.rst +++ b/source/specifications/core-metadata.rst @@ -84,26 +84,29 @@ Dynamic (multiple use) .. versionadded:: 2.2 -Only permitted in source distribution (sdist) file metadata. - -A string containing the name of another core metadata field. Only the following -metadata field names may be used as the value of the ``Dynamic`` field. - -* ``Platform`` -* ``Supported-Platform`` -* ``Requires-Dist`` -* ``Requires-External`` -* ``Provides-Extra`` -* ``Provides-Dist`` -* ``Obsoletes-Dist`` - -Unless specified via ``Dynamic``, *all* metadata in a wheel must be identical to -the metadata of the sdist from which the wheel was built. If a field is named in -``Dynamic`` in the sdist, then there is no constraint on what value that field -may have in wheels built from the sdist. - -If the sdist metadata version is older than version 2.2, then there are no -constraints on the metadata of wheels built from that sdist. +A string containing the name of another core metadata field. The field +names ``Name`` and ``Version`` may not be specified in this field. + +When found in the metadata of a source distribution, the following +rules apply: + +1. If a field is *not* marked as ``Dynamic``, then the value of the field + in any wheel built from the sdist MUST match the value in the sdist. + If the field is not in the sdist, and not marked as ``Dynamic``, then + it MUST NOT be present in the wheel. +2. If a field is marked as ``Dynamic``, it may contain any valid value in + a wheel built from the sdist (including not being present at all). + +If the sdist metadata version is older than version 2.2, then all fields should +be treated as if they were specified with ``Dynamic`` (i.e. there are no special +restrictions on the metadata of wheels built from the sdist). + +In any context other than a source distribution, ``Dynamic`` is for information +only, and indicates that the field value was calculated at wheel build time, +and may not be the same as the value in the sdist or in other wheels for the +project. + +Full details of the semantics of ``Dynamic`` are described in :pep:`643`. Platform (multiple use) diff --git a/source/specifications/source-distribution-format.rst b/source/specifications/source-distribution-format.rst index bd07ecd00..4fd514dd8 100644 --- a/source/specifications/source-distribution-format.rst +++ b/source/specifications/source-distribution-format.rst @@ -13,9 +13,9 @@ documented here. There is also the legacy source distribution format, implicitly defined by the behaviour of ``distutils`` module in the standard library, when executing ``setup.py sdist``. This document does not attempt to standardise this format, -except to note that if a legacy sdist contains a ``PKG-INFO`` file holding -metadata version 2.2 or later, that sdist MUST follow the rules for sdists -defined by the metadata specification. +except to note that if a legacy source distribution contains a ``PKG-INFO`` +file holding metadata version 2.2 or later, then it MUST follow the rules +applicable to source distributions defined in the metadata specification. Source distributions are also known as *sdists* for short. @@ -33,7 +33,7 @@ in the metadata contained in the file. Source distribution file format =============================== -A .tar.gz source distribution (sdist) contains a single top-level directory +A ``.tar.gz`` source distribution (sdist) contains a single top-level directory called ``{name}-{version}`` (e.g. ``foo-1.0``), containing the source files of the package. The name and version MUST match the metadata stored in the file. This directory must also contain a ``pyproject.toml`` in the format defined in From 7b150eddc25550589571f5453559b17734b50744 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Daniel=20D=C6=B0=C6=A1ng?= Date: Wed, 18 Nov 2020 11:11:58 -0300 Subject: [PATCH 17/33] Fix typo and dead link --- source/overview.rst | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/source/overview.rst b/source/overview.rst index c82f46591..1ae53cb7b 100644 --- a/source/overview.rst +++ b/source/overview.rst @@ -308,14 +308,14 @@ program into one of these formats, most of which involve embedding the Python interpreter and any other dependencies into a single executable file. -This approach, called *freezing*, offers wide compatiblity and +This approach, called *freezing*, offers wide compatibility and seamless user experience, though often requires multiple technologies, and a good amount of effort. A selection of Python freezers: * `pyInstaller `_ - Cross-platform -* `cx_Freeze `_ - Cross-platform +* `cx_Freeze `_ - Cross-platform * `constructor `_ - For command-line installers * `py2exe `_ - Windows only * `py2app `_ - Mac only From 801fdccaee3b3451a85ba4733b1ce1cc6bc2400d Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Filipe=20La=C3=ADns?= Date: Fri, 20 Nov 2020 17:29:00 +0000 Subject: [PATCH 18/33] key_projects: add pypa/build MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Signed-off-by: Filipe Laíns --- source/key_projects.rst | 16 ++++++++++++++++ 1 file changed, 16 insertions(+) diff --git a/source/key_projects.rst b/source/key_projects.rst index 8b84e4088..0f6f1178e 100644 --- a/source/key_projects.rst +++ b/source/key_projects.rst @@ -31,6 +31,22 @@ context of automated tests) and to prevent heavily loading PyPI's Content Delivery Network (CDN). +.. _build: + +build +===== + +`Docs `__ | +`Issues `__ | +`GitHub `__ | +`PyPI `__ | +User IRC:`#pypa `__ | +Dev IRC:`#pypa-dev `__ + +``build`` is a PEP-517 compatible Python package builder. It provides a CLI to +build packages, as well as a Python API. + + .. _distlib: distlib From 208d12199147c7243da51cf672d69935e99473c2 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Filipe=20La=C3=ADns?= Date: Fri, 20 Nov 2020 17:31:16 +0000 Subject: [PATCH 19/33] guides: replace pep517 CLI usage with pypa/build MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit See https://github.com/pypa/pep517/issues/91 Signed-off-by: Filipe Laíns --- .../publish-to-test-pypi.yml | 12 ++++++------ ...releases-using-github-actions-ci-cd-workflows.rst | 4 ++-- 2 files changed, 8 insertions(+), 8 deletions(-) diff --git a/source/guides/github-actions-ci-cd-sample/publish-to-test-pypi.yml b/source/guides/github-actions-ci-cd-sample/publish-to-test-pypi.yml index da073b470..7e8fbe4a1 100644 --- a/source/guides/github-actions-ci-cd-sample/publish-to-test-pypi.yml +++ b/source/guides/github-actions-ci-cd-sample/publish-to-test-pypi.yml @@ -13,19 +13,19 @@ jobs: uses: actions/setup-python@v1 with: python-version: 3.7 - - name: Install pep517 + - name: Install pypa/build run: >- python -m pip install - pep517 + build --user - name: Build a binary wheel and a source tarball run: >- python -m - pep517.build - --source - --binary - --out-dir dist/ + build + --sdist + --wheel + --outdir dist/ . # Actually publish to PyPI/TestPyPI - name: Publish distribution 📦 to Test PyPI diff --git a/source/guides/publishing-package-distribution-releases-using-github-actions-ci-cd-workflows.rst b/source/guides/publishing-package-distribution-releases-using-github-actions-ci-cd-workflows.rst index 9c1981afa..6ec4a97f9 100644 --- a/source/guides/publishing-package-distribution-releases-using-github-actions-ci-cd-workflows.rst +++ b/source/guides/publishing-package-distribution-releases-using-github-actions-ci-cd-workflows.rst @@ -87,13 +87,13 @@ Then, add the following under the ``build-n-publish`` section: .. literalinclude:: github-actions-ci-cd-sample/publish-to-test-pypi.yml :language: yaml :start-after: runs-on: - :end-before: Install pep517 + :end-before: Install build This will download your repository into the CI runner and then install and activate Python 3.7. And now we can build dists from source. In this example, we'll -use ``pep517`` package, assuming that your project has a +use ``build`` package, assuming that your project has a ``pyproject.toml`` properly set up (see :pep:`517`/:pep:`518`). From 6e186ddd872e40c348d7f9930b0cc25b6f6b3fa0 Mon Sep 17 00:00:00 2001 From: Pradyun Gedam <3275593+pradyunsg@users.noreply.github.com> Date: Sat, 21 Nov 2020 13:01:09 +0000 Subject: [PATCH 20/33] Update include directive's end-before Co-authored-by: Sviatoslav Sydorenko --- ...stribution-releases-using-github-actions-ci-cd-workflows.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/source/guides/publishing-package-distribution-releases-using-github-actions-ci-cd-workflows.rst b/source/guides/publishing-package-distribution-releases-using-github-actions-ci-cd-workflows.rst index 6ec4a97f9..aeb1572a6 100644 --- a/source/guides/publishing-package-distribution-releases-using-github-actions-ci-cd-workflows.rst +++ b/source/guides/publishing-package-distribution-releases-using-github-actions-ci-cd-workflows.rst @@ -87,7 +87,7 @@ Then, add the following under the ``build-n-publish`` section: .. literalinclude:: github-actions-ci-cd-sample/publish-to-test-pypi.yml :language: yaml :start-after: runs-on: - :end-before: Install build + :end-before: Install pypa/build This will download your repository into the CI runner and then install and activate Python 3.7. From 90b054d0980ce0d414b5495a050ac10485ee98a4 Mon Sep 17 00:00:00 2001 From: Brett Cannon Date: Sun, 22 Nov 2020 17:15:07 -0800 Subject: [PATCH 21/33] Specify PEP 621 --- source/specifications/core-metadata.rst | 32 +- .../declaring-project-metadata.rst | 299 ++++++++++++++++++ source/specifications/entry-points.rst | 6 +- source/specifications/index.rst | 1 + 4 files changed, 335 insertions(+), 3 deletions(-) create mode 100644 source/specifications/declaring-project-metadata.rst diff --git a/source/specifications/core-metadata.rst b/source/specifications/core-metadata.rst index 3f3e4bcf7..320db015d 100644 --- a/source/specifications/core-metadata.rst +++ b/source/specifications/core-metadata.rst @@ -49,6 +49,8 @@ Example:: Metadata-Version: 2.1 +.. _core-metadata-name: + Name ==== @@ -69,6 +71,8 @@ Example:: Name: BeagleVote +.. _core-metadata-version: + Version ======= @@ -113,6 +117,8 @@ Example:: Supported-Platform: i386-win32-2791 +.. _core-metadata-summary: + Summary ======= @@ -130,7 +136,9 @@ Example:: link targets like this one, so that links to the individual sections are not broken. + .. _description-optional: +.. _core-metadata-description: Description =========== @@ -175,7 +183,9 @@ Alternatively, the distribution's description may instead be provided in the message body (i.e., after a completely blank line following the headers, with no indentation or other special formatting necessary). + .. _description-content-type-optional: +.. _core-metadata-description-content-type: Description-Content-Type ======================== @@ -261,7 +271,9 @@ So for the last example above, the ``charset`` defaults to ``UTF-8`` and the ``variant`` defaults to ``GFM`` and thus it is equivalent to the example before it. + .. _keywords-optional: +.. _core-metadata-keywords: Keywords ======== @@ -305,7 +317,9 @@ A string containing the URL from which this version of the distribution can be downloaded. (This means that the URL can't be something like ".../BeagleVote-latest.tgz", but instead must be ".../BeagleVote-0.45.tgz".) + .. _author-optional: +.. _core-metadata-author: Author ====== @@ -320,7 +334,9 @@ Example:: Author: C. Schultz, Universal Features Syndicate, Los Angeles, CA + .. _author-email-optional: +.. _core-metadata-author-email: Author-email ============ @@ -340,7 +356,9 @@ addresses:: Author-email: cschultz@example.com, snoopy@peanuts.com + .. _maintainer-optional: +.. _core-metadata-maintainer: Maintainer ========== @@ -359,7 +377,9 @@ Example:: Maintainer: C. Schultz, Universal Features Syndicate, Los Angeles, CA + .. _maintainer-email-optional: +.. _core-metadata-maintainer-email: Maintainer-email ================ @@ -383,7 +403,9 @@ addresses:: Maintainer-email: cschultz@example.com, snoopy@peanuts.com + .. _license-optional: +.. _core-metadata-license: License ======= @@ -407,6 +429,7 @@ Examples:: .. _metadata-classifier: +.. _core-metadata-classifier: Classifier (multiple use) ========================= @@ -426,6 +449,8 @@ Examples:: Classifier: Environment :: Console (Text Based) +.. _core-metadata-requires-dist: + Requires-Dist (multiple use) ============================ @@ -466,6 +491,8 @@ Examples:: Requires-Dist: pywin32 >1.0; sys_platform == 'win32' +.. _core-metadata-requires-python: + Requires-Python =============== @@ -519,6 +546,8 @@ Examples:: Requires-External: make; sys_platform != "win32" +.. _core-metadata-project-url: + Project-URL (multiple-use) ========================== @@ -533,8 +562,9 @@ Example:: The label is free text limited to 32 characters. -.. _metadata_provides_extra: +.. _metadata_provides_extra: +.. _core-metadata-provides-extra: .. _provides-extra-optional-multiple-use: Provides-Extra (multiple use) diff --git a/source/specifications/declaring-project-metadata.rst b/source/specifications/declaring-project-metadata.rst new file mode 100644 index 000000000..c06ed46e3 --- /dev/null +++ b/source/specifications/declaring-project-metadata.rst @@ -0,0 +1,299 @@ +.. _declaring-project-metadata: + +========================== +Declaring project metadata +========================== + +:pep:`621` specifies how to write a project's +:ref:`core metadata ` in a ``pyproject.toml`` file for +packaging-related tools to consume. It defines the following +specification as the canonical source for the format used. + +The fields defined in this specification MUST be in a table named +``[project]`` in ``pyproject.toml``. No tools may add fields to this +table which are not defined by this specification. For tools wishing +to store their own settings in ``pyproject.toml``, they may use the +``[tool]`` table as defined in the +:ref:`build dependency decleration specification `. +The lack of a ``[project]`` table implicitly means the build back-end +will dynamically provide all fields. + +The only fields required to be statically defined are: + +- ``name`` + +The fields which are required but may be specified *either* statically +or listed as dynamic are: + +- ``version`` + +All other fields are considered optional and my be specified +statically, listed as dynamic, or left unspecified. + + +``name`` +======== + +- TOML_ type: string +- Corresponding :ref:`core metadata ` field: + :ref:`Name ` + +The name of the project. + +Tools SHOULD normalize this name, as specified by :pep:`503`, as soon +as it is read for internal consistency. + + +``version`` +=========== + +- TOML_ type: string +- Corresponding :ref:`core metadata ` field: + :ref:`Version ` + +The version of the project as supported by :pep:`440`. + +Users SHOULD prefer to specify already-normalized versions. + + +``description`` +=============== + +- TOML_ type: string +- Corresponding :ref:`core metadata ` field: + :ref:`Summary ` + +The summary description of the project. + + +``readme`` +========== + +- TOML_ type: string or table +- Corresponding :ref:`core metadata ` field: + :ref:`Description ` and + :ref:`Description-Content-Type ` + +The full description of the project (i.e. the README). + +The field accepts either a string or a table. If it is a string then +it is the relative path to a text file containing the full +description. Tools MUST assume the file's encoding is UTF-8. If the +file path ends in a case-insensitive ``.md`` suffix, then tools MUST +assume the content-type is ``text/markdown``. If the file path ends in +a case-insensitive ``.rst``, then tools MUST assume the content-type +is ``text/x-rst``. If a tool recognizes more extensions than this PEP, +they MAY infer the content-type for the user without specifying this +field as ``dynamic``. For all unrecognized suffixes when a +content-type is not provided, tools MUST raise an error. + +The ``readme`` field may also take a table. The ``file`` key has a +string value representing a relative path to a file containing the +full description. The ``text`` key has a string value which is the +full description. These keys are mutually-exclusive, thus tools MUST +raise an error if the metadata specifies both keys. + +A table specified in the ``readme`` field also has a ``content-type`` +field which takes a string specifying the content-type of the full +description. A tool MUST raise an error if the metadata does not +specify this field in the table. If the metadata does not specify the +``charset`` parameter, then it is assumed to be UTF-8. Tools MAY +support other encodings if they choose to. Tools MAY support +alternative content-types which they can transform to a content-type +as supported by the :ref:`core metadata `. Otherwise +tools MUST raise an error for unsupported content-types. + + +``requires-python`` +=================== + +- TOML_ type: string +- Corresponding :ref:`core metadata ` field: + :ref:`Requires-Python ` + +The Python version requirements of the project. + + +``licene`` +========== + +- TOML_ type: table +- Corresponding :ref:`core metadata ` field: + :ref:`License ` + +The table may have one of two keys. The ``file`` key has a string +value that is a relative file path to the file which contains the +license for the project. Tools MUST assume the file's encoding is +UTF-8. The ``text`` key has a string value which is the license of the +project. These keys are mutually exclusive, so a tool MUST raise an +error if the metadata specifies both keys. + + +``authors``/``maintainers`` +=========================== + +- TOML_ type: Array of inline tables with string keys and values +- Corresponding :ref:`core metadata ` field: + :ref:`Author `, + :ref:`Author-email `, + :ref:`Maintainer `, and + :ref:`Maintainer-email ` + +The people or organizations considered to be the "authors" of the +project. The exact meaning is open to interpretation — it may list the +original or primary authors, current maintainers, or owners of the +package. + +The "maintainers" field is similar to "authors" in that its exact +meaning is open to interpretation. + +These fields accept an array of tables with 2 keys: ``name`` and +``email``. Both values must be strings. The ``name`` value MUST be a +valid email name (i.e. whatever can be put as a name, before an email, +in `RFC 822`_) and not contain commas. The ``email`` value MUST be a +valid email address. Both keys are optional. + +Using the data to fill in :ref:`core metadata ` is as +follows: + +1. If only ``name`` is provided, the value goes in + :ref:`Author ` or + :ref:`Maintainer ` as appropriate. +2. If only ``email`` is provided, the value goes in + :ref:`Author-email ` or + :ref:`Maintainer-email ` + as appropriate. +3. If both ``email`` and ``name`` are provided, the value goes in + :ref:`Author-email ` or + :ref:`Maintainer-email ` + as appropriate, with the format ``{name} <{email}>``. +4. Multiple values should be separated by commas. + + +``keywords`` +============ + +- TOML_ type: array of strings +- Corresponding :ref:`core metadata ` field: + :ref:`Keywords ` + +The keywords for the project. + + +``classifiers`` +=============== + +- TOML_ type: array of strings +- Corresponding :ref:`core metadata ` field: + :ref:`Classifier ` + +Trove classifiers which apply to the project. + + +``urls`` +======== + +- TOML_ type: table with keys and values of strings +- Corresponding :ref:`core metadata ` field: + :ref:`Project-URL ` + +A table of URLs where the key is the URL label and the value is the +URL itself. + + +Entry points +============ + +- TOML_ type: table (``[project.scripts]``, ``[project.gui-scripts]``, + and ``[project.entry-points]``) +- :ref:`Entry points specification ` + +There are three tables related to entry points. The +``[project.scripts]`` table corresponds to the ``console_scripts`` +group in the :ref:`entry points specification `. The key +of the table is the name of the entry point and the value is the +object reference. + +The ``[project.gui-scripts]`` table corresponds to the ``gui_scripts`` +group in the :ref:`entry points specification `. Its +format is the same as ``[project.scripts]``. + +The ``[project.entry-points]`` table is a collection of tables. Each +sub-table's name is an entry point group. The key and value semantics +are the same as ``[project.scripts]``. Users MUST NOT create +nested sub-tables but instead keep the entry point groups to only one +level deep. + +Build back-ends MUST raise an error if the metadata defines a +``[project.entry-points.console_scripts]`` or +``[project.entry-points.gui_scripts]`` table, as they would +be ambiguous in the face of ``[project.scripts]`` and +``[project.gui-scripts]``, respectively. + + +``dependencies``/``optional-dependencies`` +========================================== + +- TOML_ type: Array of :pep:`508` strings (``dependencies``), and a + table with values of arrays of :pep:`508` strings + (``optional-dependencies``) +- Corresponding :ref:`core metadata ` field: + :ref:`Requires-Dist ` and + :ref:`Provides-Extra ` + +The (optional) dependencies of the project. + +For ``dependencies``, it is a key whose value is an array of strings. +Each string represents a dependency of the project and MUST be +formatted as a valid :pep:`508` string. Each string maps directly to +a :ref:`Requires-Dist ` entry. + +For ``optional-dependencies``, it is a table where each key specifies +an extra and whose value is an array of strings. The strings of the +arrays must be valid :pep:`508` strings. The keys MUST be valid values +for :ref:`Provides-Extra `. Each value +in the array thus becomes a corresponding +:ref:`Requires-Dist ` entry for the +matching :ref:`Provides-Extra ` +metadata. + + +``dynamic`` +=========== + +- TOML_ type: array of string +- A corresponding :ref:`core metadata ` field does not + exist + +Specifies which fields listed by this PEP were intentionally +unspecified so another tool can/will provide such metadata +dynamically. This clearly delineates which metadata is purposefully +unspecified and expected to stay unspecified compared to being +provided via tooling later on. + +- A build back-end MUST honour statically-specified metadata (which + means the metadata did not list the field in ``dynamic``). +- A build back-end MUST raise an error if the metadata specifies + ``name`` in ``dynamic``. +- If the :ref:`core metadata ` specification lists a + field as "Required", then the metadata MUST specify the field + statically or list it in ``dynamic`` (build back-ends MUST raise an + error otherwise, i.e. it should not be possible for a required field + to not be listed somehow in the ``[project]`` table). +- If the :ref:`core metadata ` specification lists a + field as "Optional", the metadata MAY list it in ``dynamic`` if the + expectation is a build back-end will provide the data for the field + later. +- Build back-ends MUST raise an error if the metadata specifies a + field statically as well as being listed in ``dynamic``. +- If the metadata does not list a field in ``dynamic``, then a build + back-end CANNOT fill in the requisite metadata on behalf of the user + (i.e. ``dynamic`` is the only way to allow a tool to fill in + metadata and the user must opt into the filling in). +- Build back-ends MUST raise an error if the metadata specifies a + field in dynamic but the build back-end was unable to provide the + data for it. + + +.. _RFC 822: https://tools.ietf.org/html/rfc822 +.. _TOML: https://toml.io/ diff --git a/source/specifications/entry-points.rst b/source/specifications/entry-points.rst index 005fd8eab..ae10bd61e 100644 --- a/source/specifications/entry-points.rst +++ b/source/specifications/entry-points.rst @@ -1,3 +1,5 @@ +.. _entry-points: + ========================== Entry points specification ========================== @@ -108,12 +110,12 @@ For tools writing the file, it is recommended only to insert a space between the object reference and the left square bracket. For example:: - + [console_scripts] foo = foomod:main # One which depends on extras: foobar = foomod:main_bar [bar,baz] - + # pytest plugins refer to a module, so there is no ':obj' [pytest11] nbval = nbval.plugin diff --git a/source/specifications/index.rst b/source/specifications/index.rst index 8cf0f909e..7e6f9b769 100644 --- a/source/specifications/index.rst +++ b/source/specifications/index.rst @@ -19,6 +19,7 @@ Package Distribution Metadata version-specifiers dependency-specifiers declaring-build-dependencies + declaring-project-metadata distribution-formats platform-compatibility-tags recording-installed-packages From 1f46c25bd564ab50e7d27761ca1ecd4caf96f89a Mon Sep 17 00:00:00 2001 From: Brett Cannon Date: Mon, 23 Nov 2020 09:59:25 -0800 Subject: [PATCH 22/33] Apply suggestions from code review Co-authored-by: Paul Moore --- source/specifications/declaring-project-metadata.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/source/specifications/declaring-project-metadata.rst b/source/specifications/declaring-project-metadata.rst index c06ed46e3..dbf276fad 100644 --- a/source/specifications/declaring-project-metadata.rst +++ b/source/specifications/declaring-project-metadata.rst @@ -114,7 +114,7 @@ tools MUST raise an error for unsupported content-types. The Python version requirements of the project. -``licene`` +``license`` ========== - TOML_ type: table From c19e3e751e5695f6724d3ee2c68e1b0664c2cf5e Mon Sep 17 00:00:00 2001 From: Brett Cannon Date: Tue, 24 Nov 2020 18:30:54 -0800 Subject: [PATCH 23/33] Address review comments --- .../declaring-project-metadata.rst | 50 +++++++++++-------- 1 file changed, 29 insertions(+), 21 deletions(-) diff --git a/source/specifications/declaring-project-metadata.rst b/source/specifications/declaring-project-metadata.rst index dbf276fad..7ce953110 100644 --- a/source/specifications/declaring-project-metadata.rst +++ b/source/specifications/declaring-project-metadata.rst @@ -9,6 +9,12 @@ Declaring project metadata packaging-related tools to consume. It defines the following specification as the canonical source for the format used. +There are two kinds of metadata: *static* and *dynamic*. Static +metadata is specified in the ``pyproject.toml`` file directly and +cannot be specified or changed by a tool. Dynamic metadata is listed +via the ``dynamic`` field (defined later in this specification) and +represents metadata that a tool will later provide. + The fields defined in this specification MUST be in a table named ``[project]`` in ``pyproject.toml``. No tools may add fields to this table which are not defined by this specification. For tools wishing @@ -77,21 +83,22 @@ The summary description of the project. The full description of the project (i.e. the README). The field accepts either a string or a table. If it is a string then -it is the relative path to a text file containing the full -description. Tools MUST assume the file's encoding is UTF-8. If the -file path ends in a case-insensitive ``.md`` suffix, then tools MUST -assume the content-type is ``text/markdown``. If the file path ends in -a case-insensitive ``.rst``, then tools MUST assume the content-type -is ``text/x-rst``. If a tool recognizes more extensions than this PEP, -they MAY infer the content-type for the user without specifying this -field as ``dynamic``. For all unrecognized suffixes when a -content-type is not provided, tools MUST raise an error. +it is a path relative to ``pyproject.toml`` to a text file containing +the full description. Tools MUST assume the file's encoding is UTF-8. +If the file path ends in a case-insensitive ``.md`` suffix, then tools +MUST assume the content-type is ``text/markdown``. If the file path +ends in a case-insensitive ``.rst``, then tools MUST assume the +content-type is ``text/x-rst``. If a tool recognizes more extensions +than this PEP, they MAY infer the content-type for the user without +specifying this field as ``dynamic``. For all unrecognized suffixes +when a content-type is not provided, tools MUST raise an error. The ``readme`` field may also take a table. The ``file`` key has a -string value representing a relative path to a file containing the -full description. The ``text`` key has a string value which is the -full description. These keys are mutually-exclusive, thus tools MUST -raise an error if the metadata specifies both keys. +string value representing a path relative to ``pyproject.toml`` to a +file containing the full description. The ``text`` key has a string +value which is the full description. These keys are +mutually-exclusive, thus tools MUST raise an error if the metadata +specifies both keys. A table specified in the ``readme`` field also has a ``content-type`` field which takes a string specifying the content-type of the full @@ -115,18 +122,18 @@ The Python version requirements of the project. ``license`` -========== +=========== - TOML_ type: table - Corresponding :ref:`core metadata ` field: :ref:`License ` The table may have one of two keys. The ``file`` key has a string -value that is a relative file path to the file which contains the -license for the project. Tools MUST assume the file's encoding is -UTF-8. The ``text`` key has a string value which is the license of the -project. These keys are mutually exclusive, so a tool MUST raise an -error if the metadata specifies both keys. +value that is a file path relative to ``pyproject.toml`` to the file +which contains the license for the project. Tools MUST assume the +file's encoding is UTF-8. The ``text`` key has a string value which is +the license of the project. These keys are mutually exclusive, so a +tool MUST raise an error if the metadata specifies both keys. ``authors``/``maintainers`` @@ -291,8 +298,9 @@ provided via tooling later on. (i.e. ``dynamic`` is the only way to allow a tool to fill in metadata and the user must opt into the filling in). - Build back-ends MUST raise an error if the metadata specifies a - field in dynamic but the build back-end was unable to provide the - data for it. + field in ``dynamic`` but the build back-end was unable to determine + the data for it (omitting the data, if determined to be the accurate + value, is acceptable). .. _RFC 822: https://tools.ietf.org/html/rfc822 From 7cf802e50798881cfae8215118f84a15f88dedd5 Mon Sep 17 00:00:00 2001 From: Alan Chan Date: Sun, 29 Nov 2020 04:40:29 +0800 Subject: [PATCH 24/33] Fix duplicate phrase about poetry (#799) Co-authored-by: Pradyun Gedam <3275593+pradyunsg@users.noreply.github.com> --- source/key_projects.rst | 9 ++++----- 1 file changed, 4 insertions(+), 5 deletions(-) diff --git a/source/key_projects.rst b/source/key_projects.rst index 0f6f1178e..cf8fb05ed 100644 --- a/source/key_projects.rst +++ b/source/key_projects.rst @@ -547,11 +547,10 @@ poetry poetry is a command-line tool to handle dependency installation and isolation as well as building and packaging of Python packages. It -uses ``pyproject.toml`` and provides its own dependency resolver, and, -instead of depending on the resolver functionality within :ref:`pip`, -provides its own dependency resolver. It attempts to speed users' -experience of installation and dependency resolution by locally -caching metadata about dependencies. +uses ``pyproject.toml`` and, instead of depending on the resolver +functionality within :ref:`pip`, provides its own dependency resolver. +It attempts to speed users' experience of installation and dependency +resolution by locally caching metadata about dependencies. .. _pypiserver: From 8b5cd5370a9c821960b2d8467b516226349bcb4b Mon Sep 17 00:00:00 2001 From: wim glenn Date: Fri, 4 Dec 2020 14:03:35 -0600 Subject: [PATCH 25/33] fix orphaned link --- source/guides/distributing-packages-using-setuptools.rst | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-) diff --git a/source/guides/distributing-packages-using-setuptools.rst b/source/guides/distributing-packages-using-setuptools.rst index 5b46467aa..ff7a360a3 100644 --- a/source/guides/distributing-packages-using-setuptools.rst +++ b/source/guides/distributing-packages-using-setuptools.rst @@ -508,9 +508,8 @@ entry_points Use this keyword to specify any plugins that your project provides for any named entry points that may be defined by your project or others that you depend on. -For more information, see the section on `Dynamic Discovery of Services and -Plugins -`_ +For more information, see the section on `Advertising Behavior +`_ from the :ref:`setuptools` docs. The most commonly used entry point is "console_scripts" (see below). From 4514b4361c1b5892a2f9630eb757769a3317da95 Mon Sep 17 00:00:00 2001 From: Stein Magnus Jodal Date: Sun, 6 Dec 2020 17:11:03 +0100 Subject: [PATCH 26/33] Fix references with wrong link text The "easy_install" references links to a section with "setuptools" as the title. This title is then used as the link text, making the section incomprehensible for people not familiar with the relationship between pip, easy_install, and setuptools. --- source/discussions/pip-vs-easy-install.rst | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/source/discussions/pip-vs-easy-install.rst b/source/discussions/pip-vs-easy-install.rst index 84c13cd31..011a40244 100644 --- a/source/discussions/pip-vs-easy-install.rst +++ b/source/discussions/pip-vs-easy-install.rst @@ -6,12 +6,12 @@ pip vs easy_install =================== -`easy_install` was released in 2004, as part of :ref:`setuptools`. It was +:ref:`easy_install ` was released in 2004, as part of :ref:`setuptools`. It was notable at the time for installing :term:`packages ` from :term:`PyPI ` using requirement specifiers, and automatically installing dependencies. -:ref:`pip` came later in 2008, as alternative to `easy_install`, although still +:ref:`pip` came later in 2008, as alternative to :ref:`easy_install `, although still largely built on top of :ref:`setuptools` components. It was notable at the time for *not* installing packages as :term:`Eggs ` or from :term:`Eggs ` (but rather simply as 'flat' packages from :term:`sdists Date: Sat, 12 Dec 2020 01:25:03 +0900 Subject: [PATCH 27/33] Move 2 "Misc" items under "Installing Packages" --- source/guides/index.rst | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/source/guides/index.rst b/source/guides/index.rst index 6c9b710ac..bd977ff04 100644 --- a/source/guides/index.rst +++ b/source/guides/index.rst @@ -14,6 +14,8 @@ introduction to packaging, see :doc:`/tutorials/index`. installing-using-linux-tools installing-scientific-packages multi-version-installs + index-mirrors-and-caches + hosting-your-own-index .. toctree:: :maxdepth: 1 @@ -39,5 +41,3 @@ introduction to packaging, see :doc:`/tutorials/index`. tool-recommendations analyzing-pypi-package-downloads - index-mirrors-and-caches - hosting-your-own-index From dcfa4b72d99959b7eaa668a82a1da2be17705824 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Ville=20Skytt=C3=A4?= Date: Mon, 14 Dec 2020 01:05:38 +0200 Subject: [PATCH 28/33] Refer to PyPI API tokens as such in GitHub secret names, spell in uppercase GitHub seems to default to uppercase nowadays, and in PyPI terms this is about API tokens instead of passwords as such. Refs https://github.com/pypa/gh-action-pypi-publish/pull/52#pullrequestreview-550969422 --- .../github-actions-ci-cd-sample/publish-to-test-pypi.yml | 4 ++-- ...ribution-releases-using-github-actions-ci-cd-workflows.rst | 4 ++-- 2 files changed, 4 insertions(+), 4 deletions(-) diff --git a/source/guides/github-actions-ci-cd-sample/publish-to-test-pypi.yml b/source/guides/github-actions-ci-cd-sample/publish-to-test-pypi.yml index 7e8fbe4a1..361203b49 100644 --- a/source/guides/github-actions-ci-cd-sample/publish-to-test-pypi.yml +++ b/source/guides/github-actions-ci-cd-sample/publish-to-test-pypi.yml @@ -31,10 +31,10 @@ jobs: - name: Publish distribution 📦 to Test PyPI uses: pypa/gh-action-pypi-publish@master with: - password: ${{ secrets.test_pypi_password }} + password: ${{ secrets.TEST_PYPI_API_TOKEN }} repository_url: https://test.pypi.org/legacy/ - name: Publish distribution 📦 to PyPI if: startsWith(github.ref, 'refs/tags') uses: pypa/gh-action-pypi-publish@master with: - password: ${{ secrets.pypi_password }} + password: ${{ secrets.PYPI_API_TOKEN }} diff --git a/source/guides/publishing-package-distribution-releases-using-github-actions-ci-cd-workflows.rst b/source/guides/publishing-package-distribution-releases-using-github-actions-ci-cd-workflows.rst index aeb1572a6..3c4676334 100644 --- a/source/guides/publishing-package-distribution-releases-using-github-actions-ci-cd-workflows.rst +++ b/source/guides/publishing-package-distribution-releases-using-github-actions-ci-cd-workflows.rst @@ -37,11 +37,11 @@ Let's begin! 🚀 2. In a separate browser tab or window, go to the ``Settings`` tab of your target repository and then click on `Secrets`_ in the left sidebar. -3. Create a new secret called ``pypi_password`` and copy-paste +3. Create a new secret called ``PYPI_API_TOKEN`` and copy-paste the token from the fist step. 4. Now, go to https://test.pypi.org/manage/account/#api-tokens and repeat the steps. Save that TestPyPI token on GitHub - as ``test_pypi_password``. + as ``TEST_PYPI_API_TOKEN``. .. attention:: From e08be9c316dc090a3a9036c92a8bba416ecd083e Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Ville=20Skytt=C3=A4?= Date: Mon, 14 Dec 2020 00:59:36 +0200 Subject: [PATCH 29/33] Spelling fixes --- source/guides/installing-using-linux-tools.rst | 2 +- source/guides/installing-using-pip-and-virtual-environments.rst | 2 +- source/guides/multi-version-installs.rst | 2 +- source/guides/packaging-binary-extensions.rst | 2 +- ...stribution-releases-using-github-actions-ci-cd-workflows.rst | 2 +- source/overview.rst | 2 +- source/specifications/declaring-project-metadata.rst | 2 +- source/specifications/recording-installed-packages.rst | 2 +- 8 files changed, 8 insertions(+), 8 deletions(-) diff --git a/source/guides/installing-using-linux-tools.rst b/source/guides/installing-using-linux-tools.rst index 9e14d9e68..2082cc72f 100644 --- a/source/guides/installing-using-linux-tools.rst +++ b/source/guides/installing-using-linux-tools.rst @@ -98,7 +98,7 @@ To install pip, wheel, and setuptools, in a parallel, non-system environment (using yum) then there are two options: -1. Use the "Sofware Collections" feature to enable a parallel collection that +1. Use the "Software Collections" feature to enable a parallel collection that includes pip, setuptools, and wheel. * For Redhat, see here: diff --git a/source/guides/installing-using-pip-and-virtual-environments.rst b/source/guides/installing-using-pip-and-virtual-environments.rst index f575cd140..a83c6719f 100644 --- a/source/guides/installing-using-pip-and-virtual-environments.rst +++ b/source/guides/installing-using-pip-and-virtual-environments.rst @@ -7,7 +7,7 @@ for Python 2. These are the lowest-level tools for managing Python packages and are recommended if higher-level tools do not suit your needs. .. note:: This doc uses the term **package** to refer to a - :term:`Distribution Package` which is different from a :term:`Import + :term:`Distribution Package` which is different from an :term:`Import Package` that which is used to import modules in your Python source code. diff --git a/source/guides/multi-version-installs.rst b/source/guides/multi-version-installs.rst index 29f7a6195..15ca1f382 100644 --- a/source/guides/multi-version-installs.rst +++ b/source/guides/multi-version-installs.rst @@ -32,7 +32,7 @@ This can be worked around by setting all dependencies in ``__main__.__requires__`` before importing ``pkg_resources`` for the first time, but that approach does mean that standard command line invocations of the affected tools can't be used - it's necessary to write a custom -wrapper script or use ``python -c ''`` to invoke the application's +wrapper script or use ``python -c ''`` to invoke the application's main entry point directly. Refer to the `pkg_resources documentation diff --git a/source/guides/packaging-binary-extensions.rst b/source/guides/packaging-binary-extensions.rst index 777c39e94..2a5a5ea87 100644 --- a/source/guides/packaging-binary-extensions.rst +++ b/source/guides/packaging-binary-extensions.rst @@ -112,7 +112,7 @@ profiling has identified the code where the speed increase is worth additional maintenance effort), a number of other alternatives should also be considered: -* look for existing optimised alternatives. The CPython standard libary +* look for existing optimised alternatives. The CPython standard library includes a number of optimised data structures and algorithms (especially in the builtins and the ``collections`` and ``itertools`` modules). The Python Package Index also offers additional alternatives. Sometimes, the diff --git a/source/guides/publishing-package-distribution-releases-using-github-actions-ci-cd-workflows.rst b/source/guides/publishing-package-distribution-releases-using-github-actions-ci-cd-workflows.rst index 3c4676334..4dd1969b6 100644 --- a/source/guides/publishing-package-distribution-releases-using-github-actions-ci-cd-workflows.rst +++ b/source/guides/publishing-package-distribution-releases-using-github-actions-ci-cd-workflows.rst @@ -38,7 +38,7 @@ Let's begin! 🚀 tab of your target repository and then click on `Secrets`_ in the left sidebar. 3. Create a new secret called ``PYPI_API_TOKEN`` and copy-paste - the token from the fist step. + the token from the first step. 4. Now, go to https://test.pypi.org/manage/account/#api-tokens and repeat the steps. Save that TestPyPI token on GitHub as ``TEST_PYPI_API_TOKEN``. diff --git a/source/overview.rst b/source/overview.rst index 797c61804..eb0f75abd 100644 --- a/source/overview.rst +++ b/source/overview.rst @@ -416,7 +416,7 @@ virtualenv `Virtualenvs `_ have -been an indispensible tool for multiple generations of Python +been an indispensable tool for multiple generations of Python developer, but are slowly fading from view, as they are being wrapped by higher-level tools. With packaging in particular, virtualenvs are used as a primitive in `the dh-virtualenv tool diff --git a/source/specifications/declaring-project-metadata.rst b/source/specifications/declaring-project-metadata.rst index 7ce953110..2a54f1640 100644 --- a/source/specifications/declaring-project-metadata.rst +++ b/source/specifications/declaring-project-metadata.rst @@ -20,7 +20,7 @@ The fields defined in this specification MUST be in a table named table which are not defined by this specification. For tools wishing to store their own settings in ``pyproject.toml``, they may use the ``[tool]`` table as defined in the -:ref:`build dependency decleration specification `. +:ref:`build dependency declaration specification `. The lack of a ``[project]`` table implicitly means the build back-end will dynamically provide all fields. diff --git a/source/specifications/recording-installed-packages.rst b/source/specifications/recording-installed-packages.rst index 815a44c98..e2fb2d730 100644 --- a/source/specifications/recording-installed-packages.rst +++ b/source/specifications/recording-installed-packages.rst @@ -21,7 +21,7 @@ History and change workflow =========================== The metadata described here was first specified in :pep:`376`, and later -ammended in :pep:`627`. +amended in :pep:`627`. It was formerly known as *Database of Installed Python Distributions*. Further amendments (except trivial language or typography fixes) must be made through the PEP process (see :pep:`1`). From 767a430d5d78bebf0515719caf781e728e61dbfe Mon Sep 17 00:00:00 2001 From: Pradyun Gedam <3275593+pradyunsg@users.noreply.github.com> Date: Sat, 2 Jan 2021 11:39:16 +0000 Subject: [PATCH 30/33] Tweak opening "important note" in installing-packages tutorial This flips the order of "synonym" vs "meaning", to convey the meaning before using a loaded term that'll link to the glossary. This should help make it less intimidating to first-time readers as well as generally make it easier to follow the sentence. --- source/tutorials/installing-packages.rst | 17 +++++++++-------- 1 file changed, 9 insertions(+), 8 deletions(-) diff --git a/source/tutorials/installing-packages.rst b/source/tutorials/installing-packages.rst index c87a98f27..44503b32f 100644 --- a/source/tutorials/installing-packages.rst +++ b/source/tutorials/installing-packages.rst @@ -7,14 +7,15 @@ Installing Packages This section covers the basics of how to install Python :term:`packages `. -It's important to note that the term "package" in this context is being used as -a synonym for a :term:`distribution ` (i.e. a bundle of -software to be installed), not to refer to the kind of :term:`package ` that you import in your Python source code (i.e. a container of -modules). It is common in the Python community to refer to a :term:`distribution -` using the term "package". Using the term "distribution" -is often not preferred, because it can easily be confused with a Linux -distribution, or another larger software distribution like Python itself. +It's important to note that the term "package" in this context is being used to +describe a bundle of software to be installed (i.e. as a synonym for a +:term:`distribution `). It does not to refer to the kind +of :term:`package ` that you import in your Python source code +(i.e. a container of modules). It is common in the Python community to refer to +a :term:`distribution ` using the term "package". Using +the term "distribution" is often not preferred, because it can easily be +confused with a Linux distribution, or another larger software distribution +like Python itself. .. contents:: Contents From 0575db255fb458cae1bcd0870c7ba4e649e69d72 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?St=C3=A9phane=20Bidoul?= Date: Sun, 3 Jan 2021 16:01:17 +0100 Subject: [PATCH 31/33] PEP 610 Add direct_url.json specification (#690) The `resolved_revision` feature from the PEP is omitted for now. It can be added later when we have implementation experience. --- source/specifications/direct-url.rst | 295 ++++++++++++++++++ source/specifications/index.rst | 1 + .../recording-installed-packages.rst | 11 + 3 files changed, 307 insertions(+) create mode 100644 source/specifications/direct-url.rst diff --git a/source/specifications/direct-url.rst b/source/specifications/direct-url.rst new file mode 100644 index 000000000..7df786cae --- /dev/null +++ b/source/specifications/direct-url.rst @@ -0,0 +1,295 @@ + +.. _direct-url: + +========================================================== +Recording the Direct URL Origin of installed distributions +========================================================== + +This document specifies a :file:`direct_url.json` file in the +:file:`*.dist-info` directory of an installed distribution, to record the +Direct URL Origin of the distribution. The layout of this file was originally +specified in :pep:`610` and is formally documented here. + +.. contents:: Contents + :local: + +Specification +============= + +The :file:`direct_url.json` file MUST be created in the :file:`*.dist-info` +directory by installers when installing a distribution from a requirement +specifying a direct URL reference (including a VCS URL). + +This file MUST NOT be created when installing a distribution from an other +type of requirement (i.e. name plus version specifier). + +This JSON file MUST be a dictionary, compliant with `RFC 8259 +`_ and UTF-8 encoded. + +If present, it MUST contain at least two fields. The first one is ``url``, with +type ``string``. Depending on what ``url`` refers to, the second field MUST be +one of ``vcs_info`` (if ``url`` is a VCS reference), ``archive_info`` (if +``url`` is a source archives or a wheel), or ``dir_info`` (if ``url`` is a +local directory). These info fields have a (possibly empty) subdictionary as +value, with the possible keys defined below. + +``url`` MUST be stripped of any sensitive authentication information, +for security reasons. + +The user:password section of the URL MAY however +be composed of environment variables, matching the following regular +expression:: + + \$\{[A-Za-z0-9-_]+\}(:\$\{[A-Za-z0-9-_]+\})? + +Additionally, the user:password section of the URL MAY be a +well-known, non security sensitive string. A typical example is ``git`` +in the case of an URL such as ``ssh://git@gitlab.com/user/repo``. + +When ``url`` refers to a VCS repository, the ``vcs_info`` key MUST be present +as a dictionary with the following keys: + +- A ``vcs`` key (type ``string``) MUST be present, containing the name of the VCS + (i.e. one of ``git``, ``hg``, ``bzr``, ``svn``). Other VCS's SHOULD be registered by + writing a PEP to amend this specification. + The ``url`` value MUST be compatible with the corresponding VCS, + so an installer can hand it off without transformation to a + checkout/download command of the VCS. +- A ``requested_revision`` key (type ``string``) MAY be present naming a + branch/tag/ref/commit/revision/etc (in a format compatible with the VCS) + to install. +- A ``commit_id`` key (type ``string``) MUST be present, containing the + exact commit/revision number that was installed. + If the VCS supports commit-hash + based revision identifiers, such commit-hash MUST be used as + ``commit_id`` in order to reference the immutable + version of the source code that was installed. + +When ``url`` refers to a source archive or a wheel, the ``archive_info`` key +MUST be present as a dictionary with the following key: + +- A ``hash`` key (type ``string``) SHOULD be present, with value + ``=``. + It is RECOMMENDED that only hashes which are unconditionally provided by + the latest version of the standard library's ``hashlib`` module be used for + source archive hashes. At time of writing, that list consists of 'md5', + 'sha1', 'sha224', 'sha256', 'sha384', and 'sha512'. + +When ``url`` refers to a local directory, the ``dir_info`` key MUST be +present as a dictionary with the following key: + +- ``editable`` (type: ``boolean``): ``true`` if the distribution was installed + in editable mode, ``false`` otherwise. If absent, default to ``false``. + +When ``url`` refers to a local directory, it MUST have the ``file`` sheme and +be compliant with `RFC 8089 `_. In +particular, the path component must be absolute. Symbolic links SHOULD be +preserved when making relative paths absolute. + +.. note:: + + When the requested URL has the file:// scheme and points to a local directory that happens to contain a + VCS checkout, installers MUST NOT attempt to infer any VCS information and + therefore MUST NOT output any VCS related information (such as ``vcs_info``) + in :file:`direct_url.json`. + +A top-level ``subdirectory`` field MAY be present containing a directory path, +relative to the root of the VCS repository, source archive or local directory, +to specify where ``pyproject.toml`` or ``setup.py`` is located. + +.. note:: + + As a general rule, installers should as much as possible preserve the + information that was provided in the requested URL when generating + :file:`direct_url.json`. For example user:password environment variables + should be preserved and ``requested_revision`` should reflect the revision that was + provided in the requested URL as faithfully as possible. This information is + however *enriched* with more precise data, such as ``commit_id``. + +Registered VCS +============== + +This section lists the registered VCS's; expanded, VCS-specific information +on how to use the ``vcs``, ``requested_revision``, and other fields of +``vcs_info``; and in +some cases additional VCS-specific fields. +Tools MAY support other VCS's although it is RECOMMENDED to register +them by writing a PEP to amend this specification. The ``vcs`` field SHOULD be the command name +(lowercased). Additional fields that would be necessary to +support such VCS SHOULD be prefixed with the VCS command name. + +Git +--- + +Home page + + https://git-scm.com/ + +vcs command + + git + +``vcs`` field + + git + +``requested_revision`` field + + A tag name, branch name, Git ref, commit hash, shortened commit hash, + or other commit-ish. + +``commit_id`` field + + A commit hash (40 hexadecimal characters sha1). + +.. note:: + + Installers can use the ``git show-ref`` and ``git symbolic-ref`` commands + to determine if the ``requested_revision`` corresponds to a Git ref. + In turn, a ref beginning with ``refs/tags/`` corresponds to a tag, and + a ref beginning with ``refs/remotes/origin/`` after cloning corresponds + to a branch. + +Mercurial +--------- + +Home page + + https://www.mercurial-scm.org/ + +vcs command + + hg + +``vcs`` field + + hg + +``requested_revision`` field + + A tag name, branch name, changeset ID, shortened changeset ID. + +``commit_id`` field + + A changeset ID (40 hexadecimal characters). + +Bazaar +------ + +Home page + + https://bazaar.canonical.com/ + +vcs command + + bzr + +``vcs`` field + + bzr + +``requested_revision`` field + + A tag name, branch name, revision id. + +``commit_id`` field + + A revision id. + +Subversion +---------- + +Home page + + https://subversion.apache.org/ + +vcs command + + svn + +``vcs`` field + + svn + +``requested_revision`` field + + ``requested_revision`` must be compatible with ``svn checkout`` ``--revision`` option. + In Subversion, branch or tag is part of ``url``. + +``commit_id`` field + + Since Subversion does not support globally unique identifiers, + this field is the Subversion revision number in the corresponding + repository. + +Examples +======== + +Example direct_url.json +----------------------- + +Source archive: + +.. code:: + + { + "url": "https://github.com/pypa/pip/archive/1.3.1.zip", + "archive_info": { + "hash": "sha256=2dc6b5a470a1bde68946f263f1af1515a2574a150a30d6ce02c6ff742fcc0db8" + } + } + +Git URL with tag and commit-hash: + +.. code:: + + { + "url": "https://github.com/pypa/pip.git", + "vcs_info": { + "vcs": "git", + "requested_revision": "1.3.1", + "commit_id": "7921be1537eac1e97bc40179a57f0349c2aee67d" + } + } + +Local directory: + +.. code:: + + { + "url": "file:///home/user/project", + "dir_info": {} + } + +Local directory installed in editable mode: + +.. code:: + + { + "url": "file:///home/user/project", + "dir_info": { + "editable": true + } + } + + +Example pip commands and their effect on direct_url.json +-------------------------------------------------------- + +Commands that generate a ``direct_url.json``: + +* pip install https://example.com/app-1.0.tgz +* pip install https://example.com/app-1.0.whl +* pip install "git+https://example.com/repo/app.git#egg=app&subdirectory=setup" +* pip install ./app +* pip install file:///home/user/app +* pip install --editable "git+https://example.com/repo/app.git#egg=app&subdirectory=setup" + (in which case, ``url`` will be the local directory where the git repository has been + cloned to, and ``dir_info`` will be present with ``"editable": true`` and no + ``vcs_info`` will be set) +* pip install -e ./app + +Commands that *do not* generate a ``direct_url.json`` + +* pip install app +* pip install app --no-index --find-links https://example.com/ diff --git a/source/specifications/index.rst b/source/specifications/index.rst index f14f3949d..6a282b243 100644 --- a/source/specifications/index.rst +++ b/source/specifications/index.rst @@ -23,6 +23,7 @@ Package Distribution Metadata platform-compatibility-tags recording-installed-packages entry-points + direct-url Package Distribution File Formats diff --git a/source/specifications/recording-installed-packages.rst b/source/specifications/recording-installed-packages.rst index e2fb2d730..2cac472cf 100644 --- a/source/specifications/recording-installed-packages.rst +++ b/source/specifications/recording-installed-packages.rst @@ -173,3 +173,14 @@ This value should be used for informational purposes only. For example, if a tool is asked to uninstall a project but finds no ``RECORD`` file, it may suggest that the tool named in ``INSTALLER`` may be able to do the uninstallation. + +The direct_url.json file +======================== + +This file MUST be created by installers when installing a distribution from a +requirement specifying a direct URL reference (including a VCS URL). + +This file MUST NOT be created when installing a distribution from an other type +of requirement (i.e. name plus version specifier). + +Its detailed specification is at :ref:`direct-url`. From 63ec2a761649f89b1204aa7c5a728852cc17bd25 Mon Sep 17 00:00:00 2001 From: Hugo van Kemenade Date: Sun, 3 Jan 2021 23:29:57 +0200 Subject: [PATCH 32/33] Fix typos --- source/overview.rst | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/source/overview.rst b/source/overview.rst index eb0f75abd..4e1779670 100644 --- a/source/overview.rst +++ b/source/overview.rst @@ -450,7 +450,7 @@ ride. This impression is mostly a byproduct of Python's versatility. Once you understand the natural boundaries between each packaging solution, you begin to realize that the varied landscape is a small price Python programmers pay for using one of the most -balanced, flexible language available. +balanced, flexible languages available. .. Editing notes: @@ -503,7 +503,7 @@ balanced, flexible language available. - Avoid words that trivialize using JupyterLab such as “simply” or “just.” Tasks that developers find simple or - easy may not be for users." + easy may not be for users. Among other useful points. Read more here: https://jupyterlab.readthedocs.io/en/latest/developer/documentation.html From 2210aca817971d843fb20e87ccad70f6a296a84f Mon Sep 17 00:00:00 2001 From: chris Date: Fri, 15 Jan 2021 01:54:07 +0530 Subject: [PATCH 33/33] Grammatically updated the docs --- source/tutorials/packaging-projects.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/source/tutorials/packaging-projects.rst b/source/tutorials/packaging-projects.rst index 7ab9021fa..34dd54613 100644 --- a/source/tutorials/packaging-projects.rst +++ b/source/tutorials/packaging-projects.rst @@ -103,7 +103,7 @@ minimal set: package. - ``description`` is a short, one-sentence summary of the package. - ``long_description`` is a detailed description of the package. This is - shown on the package detail package on the Python Package Index. In + shown on the package detail page on the Python Package Index. In this case, the long description is loaded from :file:`README.md` which is a common pattern. - ``long_description_content_type`` tells the index what type of markup is