PEP 599 – The manylinux2014 Platform Tag
- Author:
- Dustin Ingram <di at python.org>
- Sponsor:
- Paul Moore <p.f.moore at gmail.com>
- BDFL-Delegate:
- Paul Moore <p.f.moore at gmail.com>
- Discussions-To:
- Discourse thread
- Status:
- Superseded
- Type:
- Informational
- Topic:
- Packaging
- Created:
- 29-Apr-2019
- Post-History:
- 29-Apr-2019
- Superseded-By:
- 600
- Resolution:
- Discourse message
Abstract
This PEP proposes the creation of a manylinux2014
platform tag to
succeed the manylinux2010
tag introduced by PEP 513. It also
proposes that PyPI and pip
both be updated to support uploading,
downloading, and installing manylinux2014
distributions on
compatible platforms.
Rationale
CentOS 6 is now the oldest supported CentOS release, and will receive
maintenance updates through November 30th, 2020, [1] at which point
it will reach end-of-life, and no further updates such as security
patches will be made available. All wheels built under the
manylinux2010
images will remain at obsolete versions after that
point.
Therefore, we propose the continuation of the existing manylinux
standard, and that a new PEP 425-style platform tag called
manylinux2014
be derived from CentOS 7 and that the manylinux
toolchain, PyPI, and pip
be updated to support it.
Similar to how PEP 571 and PEP 513 drew allowed shared
libraries and their symbol versions from CentOS 5.11 and CentOS 6,
respectively, a manylinux2014
platform tag will draw its libraries
and symbol versions from CentOS 7, which will reach end-of-life on
June 30th, 2024. [1]
The manylinuxYYYY
pattern has a number of advantages that motivate
continuing with the current status quo:
- Well-defined Docker images with clearly specified compatible libraries;
- No need to survey for compatibility issues across multiple releases;
- A single build image and
auditwheel
profile per architecture.
There are also some disadvantages:
- Requires drafting a new PEP for every new standard;
- Requires adding the new platform tag to installers (e.g.,
pip
); - Installers are unable to install a platform tag which predates a given release.
There are also challenges which would exist for any proposal,
including the time and effort it takes to define, prepare and release
the Docker images and corresponding auditwheel
profiles. These
challenges were experienced in the long rollout period for
manylinux2010
, which took approximately 1 year from PEP acceptance
to compatible build environment published. [3]
However, if this PEP can be an indicator, the process is now well-defined and easily repeatable, which should increase the timeline for rollout of a newer, updated platform tag.
The manylinux2014
policy
The following criteria determine a linux
wheel’s eligibility for
the manylinux2014
tag:
- The wheel may only contain binary executables and shared objects
compiled for one of the following architectures supported by CentOS
7, or a CentOS 7 compatible base image (such as ubi7): [4]
x86_64 i686 aarch64 armv7l ppc64 ppc64le s390x
This list adds support for ARMv7 (armv7l), ARMv8 (aarch64) and PowerPC (ppc64, ppc64le) architectures supported by the CentOS Alternative Architecture Special Interest Group, as well as the IBM Z (s390x) architecture. [5]
- The wheel’s binary executables or shared objects may not link
against externally-provided libraries except those in the following
list:
libgcc_s.so.1 libstdc .so.6 libm.so.6 libdl.so.2 librt.so.1 libc.so.6 libnsl.so.1 libutil.so.1 libpthread.so.0 libresolv.so.2 libX11.so.6 libXext.so.6 libXrender.so.1 libICE.so.6 libSM.so.6 libGL.so.1 libgobject-2.0.so.0 libgthread-2.0.so.0 libglib-2.0.so.0
This list is identical to the externally-provided libraries originally allowed for
manylinux2010
, with one exception:libcrypt.so.1
was removed due to being deprecated in Fedora 30.libpythonX.Y
remains ineligible for inclusion for the same reasons outlined in PEP 513.On Debian-based systems, these libraries are provided by the packages:
Package Libraries libc6 libdl.so.2, libresolv.so.2, librt.so.1, libc.so.6, libpthread.so.0, libm.so.6, libutil.so.1, libnsl.so.1 libgcc1 libgcc_s.so.1 libgl1 libGL.so.1 libglib2.0-0 libgobject-2.0.so.0, libgthread-2.0.so.0, libglib-2.0.so.0 libice6 libICE.so.6 libsm6 libSM.so.6 libstdc 6 libstdc .so.6 libx11-6 libX11.so.6 libxext6 libXext.so.6 libxrender1 libXrender.so.1 On RPM-based systems, they are provided by these packages:
Package Libraries glib2 libglib-2.0.so.0, libgthread-2.0.so.0, libgobject-2.0.so.0 glibc libresolv.so.2, libutil.so.1, libnsl.so.1, librt.so.1, libpthread.so.0, libdl.so.2, libm.so.6, libc.so.6 libICE libICE.so.6 libX11 libX11.so.6 libXext: libXext.so.6 libXrender libXrender.so.1 libgcc: libgcc_s.so.1 libstdc libstdc .so.6 mesa libGL.so.1 - If the wheel contains binary executables or shared objects linked
against any allowed libraries that also export versioned symbols,
they may only depend on the following maximum versions:
GLIBC_2.17 CXXABI_1.3.7, CXXABI_TM_1 is also allowed GLIBCXX_3.4.19 GCC_4.8.0
As an example,
manylinux2014
wheels may include binary artifacts that requireglibc
symbols at versionGLIBC_2.12
, because this an earlier version than the maximum ofGLIBC_2.17
. - If a wheel is built for any version of CPython 2 or CPython
versions 3.0 up to and including 3.2, it must include a CPython
ABI tag indicating its Unicode ABI. A
manylinux2014
wheel built against Python 2, then, must include either thecpy27mu
tag indicating it was built against an interpreter with the UCS-4 ABI or thecpy27m
tag indicating an interpreter with the UCS-2 ABI. (PEP 3149 [7]) - A wheel must not require the
PyFPE_jbuf
symbol. This is achieved by building it against a Python compiled without the--with-fpectl
configure
flag.
Compilation of Compliant Wheels
Like manylinux1
, the auditwheel
tool adds manylinux2014
platform tags to linux
wheels built by pip wheel
or
bdist_wheel
in a manylinux2014
Docker container.
Docker Images
A manylinux2014
Docker image based on CentOS 7 x86_64 should be
provided for building binary linux
wheels that can reliably be
converted to manylinux2014
wheels. This image will come with a
full compiler suite installed (gcc
, g
, and gfortran
4.8.5) as well as the latest releases of Python and pip
.
Auditwheel
The auditwheel
tool will also be updated to produce
manylinux2014
wheels. [8] Its behavior and purpose will be
otherwise unchanged from PEP 513.
Platform Detection for Installers
Platforms may define a manylinux2014_compatible
boolean attribute
on the _manylinux
module described in PEP 513. A platform is
considered incompatible with manylinux2014
if the attribute is
False
.
If the _manylinux
module is not found, or it does not have the
attribute manylinux2014_compatible
, tools may fall back to
checking for glibc. If the platform has glibc 2.17 or newer, it is
assumed to be compatible unless the _manylinux
module says
otherwise.
Specifically, the algorithm we propose is:
def is_manylinux2014_compatible():
# Only Linux, and only supported architectures
from distutils.util import get_platform
if get_platform() not in [
"linux-x86_64",
"linux-i686",
"linux-aarch64",
"linux-armv7l",
"linux-ppc64",
"linux-ppc64le",
"linux-s390x",
]:
return False
# Check for presence of _manylinux module
try:
import _manylinux
return bool(_manylinux.manylinux2014_compatible)
except (ImportError, AttributeError):
# Fall through to heuristic check below
pass
# Check glibc version. CentOS 7 uses glibc 2.17.
# PEP 513 contains an implementation of this function.
return have_compatible_glibc(2, 17)
Backwards compatibility with manylinux2010
wheels
As explained in PEP 513, the specified symbol versions for
manylinux1
allowed libraries constitute an upper bound. The
same is true for the symbol versions defined for manylinux2014
in
this PEP. As a result, manylinux1
and manylinux2010
wheels
are considered manylinux2014
wheels. A pip
that recognizes
the manylinux2014
platform tag will thus install manylinux2010
wheels for manylinux2014
platforms – even when explicitly set –
when no manylinux2014
wheels are available.
PyPI Support
PyPI should permit wheels containing the manylinux2014
platform
tag to be uploaded in the same way that it permits manylinux2010
.
If technically feasible, PyPI should attempt to verify the
compatibility of manylinux2014
wheels, but that capability is not
a requirement for adoption of this PEP.
Package authors should not upload non-compliant manylinux2014
wheels to PyPI, and should be aware that PyPI may begin blocking
non-compliant wheels from being uploaded.
References
Acceptance
Copyright
This document is placed in the public domain or under the CC0-1.0-Universal license, whichever is more permissive.
Source: https://github.com/python/peps/blob/main/peps/pep-0599.rst
Last modified: 2023-09-09 17:39:29 GMT