Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update to the latest c-code template #73

Merged
merged 3 commits into from
Sep 13, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
64 changes: 53 additions & 11 deletions .github/workflows/tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -61,8 61,8 @@ env:
PIP_NO_PYTHON_VERSION_WARNING: 1
PIP_NO_WARN_SCRIPT_LOCATION: 1

CFLAGS: -Ofast -pipe
CXXFLAGS: -Ofast -pipe
CFLAGS: -O3 -pipe
CXXFLAGS: -O3 -pipe
# Uploading built wheels for releases.
# TWINE_PASSWORD is encrypted and stored directly in the
# github repo settings.
Expand Down Expand Up @@ -91,6 91,7 @@ jobs:
# Sigh. Note that the matrix must be kept in sync
# with `test`, and `docs` must use a subset.
runs-on: ${{ matrix.os }}
if: github.event_name != 'pull_request' || github.event.pull_request.head.repo.full_name != github.event.pull_request.base.repo.full_name
strategy:
fail-fast: false
matrix:
Expand All @@ -104,7 105,7 @@ jobs:
- "3.8"
- "3.9"
- "3.10"
- "3.11.0-beta.3"
- "3.11.0-rc.1"
os: [ubuntu-20.04, macos-latest]
exclude:
- os: macos-latest
Expand Down Expand Up @@ -142,18 143,19 @@ jobs:

- name: Install Build Dependencies (PyPy2)
if: >
startsWith(matrix.python-version, 'pypy-2.7')
startsWith(matrix.python-version, 'pypy-2.7')
run: |
pip install -U pip
pip install -U setuptools wheel twine "cffi != 1.15.1"
- name: Install Build Dependencies (other Python versions)
if: >
!startsWith(matrix.python-version, 'pypy-2.7')
!startsWith(matrix.python-version, 'pypy-2.7')
run: |
pip install -U pip
pip install -U setuptools wheel twine cffi

- name: Build zodbpickle
- name: Build zodbpickle (3.11.0-rc.1)
if: ${{ startsWith(matrix.python-version, '3.11.0-rc.1') }}
run: |
# Next, build the wheel *in place*. This helps ccache, and also lets us cache the configure
# output (pip install uses a random temporary directory, making this difficult).
Expand All @@ -162,6 164,33 @@ jobs:
# Also install it, so that we get dependencies in the (pip) cache.
pip install -U 'faulthandler; python_version == "2.7" and platform_python_implementation == "CPython"'
pip install --pre .[test]
- name: Build zodbpickle (Python 3.10 on MacOS)
if: >
startsWith(runner.os, 'Mac')
&& startsWith(matrix.python-version, '3.10')
env:
_PYTHON_HOST_PLATFORM: macosx-11-x86_64
run: |
# Next, build the wheel *in place*. This helps ccache, and also lets us cache the configure
# output (pip install uses a random temporary directory, making this difficult).
python setup.py build_ext -i
python setup.py bdist_wheel
# Also install it, so that we get dependencies in the (pip) cache.
pip install -U 'faulthandler; python_version == "2.7" and platform_python_implementation == "CPython"'
pip install .[test]

- name: Build zodbpickle (all other versions)
if: >
!startsWith(runner.os, 'Mac')
|| !startsWith(matrix.python-version, '3.10')
run: |
# Next, build the wheel *in place*. This helps ccache, and also lets us cache the configure
# output (pip install uses a random temporary directory, making this difficult).
python setup.py build_ext -i
python setup.py bdist_wheel
# Also install it, so that we get dependencies in the (pip) cache.
pip install -U 'faulthandler; python_version == "2.7" and platform_python_implementation == "CPython"'
pip install .[test]

- name: Check zodbpickle build
run: |
Expand All @@ -181,7 210,7 @@ jobs:
&& startsWith(github.ref, 'refs/tags')
&& startsWith(runner.os, 'Mac')
&& !startsWith(matrix.python-version, 'pypy')
&& !startsWith(matrix.python-version, '3.11.0-beta.3')
&& !startsWith(matrix.python-version, '3.11.0-rc.1')
env:
TWINE_PASSWORD: ${{ secrets.TWINE_PASSWORD }}
run: |
Expand All @@ -203,7 232,7 @@ jobs:
- "3.8"
- "3.9"
- "3.10"
- "3.11.0-beta.3"
- "3.11.0-rc.1"
os: [ubuntu-20.04, macos-latest]
exclude:
- os: macos-latest
Expand Down Expand Up @@ -244,7 273,8 @@ jobs:
with:
name: zodbpickle-${{ runner.os }}-${{ matrix.python-version }}.whl
path: dist/
- name: Install zodbpickle
- name: Install zodbpickle 3.11.0-rc.1
if: ${{ startsWith(matrix.python-version, '3.11.0-rc.1') }}
run: |
pip install -U wheel setuptools
# coverage has a wheel on PyPI for a future python version which is
Expand All @@ -256,13 286,25 @@ jobs:
# might also save some build time?
unzip -n dist/zodbpickle-*whl -d src
pip install --pre -U -e .[test]
- name: Install zodbpickle
if: ${{ !startsWith(matrix.python-version, '3.11.0-rc.1') }}
run: |
pip install -U wheel setuptools
pip install -U coverage
pip install -U 'faulthandler; python_version == "2.7" and platform_python_implementation == "CPython"'
# Unzip into src/ so that testrunner can find the .so files
# when we ask it to load tests from that directory. This
# might also save some build time?
unzip -n dist/zodbpickle-*whl -d src
pip install -U -e .[test]
- name: Run tests with C extensions
if: ${{ !startsWith(matrix.python-version, 'pypy') }}
run: |
python -m coverage run -p -m zope.testrunner --test-path=src --auto-color --auto-progress -m zodbpickle.tests.test_pickle$
python -m coverage run -p -m zope.testrunner --test-path=src --auto-color --auto-progress
- name: Run tests without C extensions
run:
PURE_PYTHON=1 python -m coverage run -p -m zope.testrunner --test-path=src --auto-color --auto-progress -m zodbpickle.tests.test_pickle$
# coverage makes PyPy run about 3x slower!
PURE_PYTHON=1 python -m coverage run -p -m zope.testrunner --test-path=src --auto-color --auto-progress
- name: Report Coverage
run: |
coverage combine
Expand Down
28 changes: 24 additions & 4 deletions .manylinux-install.sh
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 26,20 @@ ls -ld /cache/pip
# We need some libraries because we build wheels from scratch:
yum -y install libffi-devel

tox_env_map() {
case $1 in
*"cp27"*) echo 'py27';;
*"cp35"*) echo 'py35';;
*"cp311"*) echo 'py311';;
*"cp36"*) echo 'py36';;
*"cp37"*) echo 'py37';;
*"cp38"*) echo 'py38';;
*"cp39"*) echo 'py39';;
*"cp310"*) echo 'py310';;
*) echo 'py';;
esac
}

# Compile wheels
for PYBIN in /opt/python/*/bin; do
if \
Expand All @@ -37,12 51,18 @@ for PYBIN in /opt/python/*/bin; do
[[ "${PYBIN}" == *"cp38"* ]] || \
[[ "${PYBIN}" == *"cp39"* ]] || \
[[ "${PYBIN}" == *"cp310"* ]] ; then
"${PYBIN}/pip" install --pre -e /io/
"${PYBIN}/pip" wheel /io/ --pre -w wheelhouse/
if [[ "${PYBIN}" == *"cp311"* ]] ; then
"${PYBIN}/pip" install --pre -e /io/
"${PYBIN}/pip" wheel /io/ --pre -w wheelhouse/
else
"${PYBIN}/pip" install -e /io/
"${PYBIN}/pip" wheel /io/ -w wheelhouse/
fi
if [ `uname -m` == 'aarch64' ]; then
cd /io/
"${PYBIN}/pip" install tox
"${PYBIN}/tox" -e py
${PYBIN}/pip install tox
TOXENV=$(tox_env_map "${PYBIN}")
${PYBIN}/tox -e ${TOXENV}
cd ..
fi
rm -rf /io/build /io/*.egg-info
Expand Down
19 changes: 1 addition & 18 deletions .meta.toml
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 2,7 @@
# https://github.com/zopefoundation/meta/tree/master/config/c-code
[meta]
template = "c-code"
commit-id = "cde9741a5a0d9d04cee25cb617ee1590afebb17d"
commit-id = "8f6ec60e8e65cc890fbf4635134b1789dd386f12"

[python]
with-appveyor = true
Expand All @@ -14,15 14,6 @@ with-future-python = true

[tox]
use-flake8 = false
testenv-commands = [
"# there's test_pickle_2 for Python 2 and test_pickle_3 for Python 3",
"# different test files because metaclass syntax differs on Pythons",
"# test_pickle picks and returns the appropriate one",
"# without this we'd get an import error (actually syntax error) for one",
"# of them",
"zope-testrunner --test-path=src -m zodbpickle.tests.test_pickle$ {posargs:-vc}",
]
coverage-command = "coverage run -m zope.testrunner --test-path=src -m zodbpickle.tests.test_pickle$ {posargs:-vc}"

[coverage]
fail-under = 63
Expand Down Expand Up @@ -59,11 50,3 @@ global-env-vars = [
" TWINE_PASSWORD:",
" secure: aoZC/ rvJKg8B5GMGIxd1YzSugYzr1vrR0NzX NBCGjc64YaTc/K4kIRj4eqD0p11Nhl2aGUgHF6qMUcHIAAUybaKU/3Kyp BwxU5eGX/NZv9S6AsBGVyiHD4oourkczzeB/HGrICEm9TeTcR3lc70SrpxSpYwnDyLv8CUpk0D/iUgQyFT/BERDsBTJEPI3HV/ngmMQrj9g2H7PkfF0yR5ss8AclYcIdWOKTvTapcGLerlK mFm5muturTCT2njYMP2cv NV3 Ih6OxSpQlWjw==",
]
test-steps = [
"- zope-testrunner --test-path=src -m zodbpickle.tests.test_pickle$",
]

[github-actions]
test-commands = [
"python -m coverage run -p -m zope.testrunner --test-path=src --auto-color --auto-progress -m zodbpickle.tests.test_pickle$",
]
3 changes: 3 additions & 0 deletions CHANGES.rst
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 7,9 @@

- Add support for Python 3.11 (as of 3.11.0b3).

- Disable unsafe math optimizations in C code. See `pull request 73
<https://github.com/zopefoundation/zodbpickle/pull/73>`_.


2.3 (2022-04-22)
================
Expand Down
2 changes: 1 addition & 1 deletion appveyor.yml
Original file line number Diff line number Diff line change
Expand Up @@ -50,7 50,7 @@ build_script:
- python -W ignore setup.py -q bdist_wheel

test_script:
- zope-testrunner --test-path=src -m zodbpickle.tests.test_pickle$
- zope-testrunner --test-path=src
artifacts:
- path: 'dist\*.whl'
name: wheel
Expand Down
6 changes: 0 additions & 6 deletions buildout.cfg
Original file line number Diff line number Diff line change
Expand Up @@ -8,12 8,6 @@ parts =
recipe = zc.recipe.testrunner
eggs =
zodbpickle [test]
# there's test_pickle_2 for Python 2 and test_pickle_3 for Python 3
# different test files because metaclass syntax differs on Pythons
# test_pickle picks and returns the appropriate one
# without this we'd get an import error (actually syntax error) for one
# of them
defaults = ['-m', 'zodbpickle.tests.test_pickle$']

[scripts]
recipe = zc.recipe.egg
Expand Down
32 changes: 32 additions & 0 deletions src/zodbpickle/tests/test_compile_flags.py
Original file line number Diff line number Diff line change
@@ -0,0 1,32 @@
##############################################################################
#
# Copyright (c) 2022 Zope Foundation and Contributors.
# All Rights Reserved.
#
# This software is subject to the provisions of the Zope Public License,
# Version 2.1 (ZPL). A copy of the ZPL should accompany this distribution.
# THIS SOFTWARE IS PROVIDED "AS IS" AND ANY AND ALL EXPRESS OR IMPLIED
# WARRANTIES ARE DISCLAIMED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
# WARRANTIES OF TITLE, MERCHANTABILITY, AGAINST INFRINGEMENT, AND FITNESS
# FOR A PARTICULAR PURPOSE
#
##############################################################################
import struct
import unittest

try:
import zodbpickle._pickle # noqa: try to load a C module for side effects
except ImportError: # pragma: no cover
pass


class TestFloatingPoint(unittest.TestCase):

def test_no_fast_math_optimization(self):
# Building with -Ofast enables -ffast-math, which sets certain FPU
# flags that can cause breakage elsewhere. A library such as BTrees
# has no business changing global FPU flags for the entire process.
zero_bits = struct.unpack("!Q", struct.pack("!d", 0.0))[0]
next_up = zero_bits 1
smallest_subnormal = struct.unpack("!d", struct.pack("!Q", next_up))[0]
self.assertNotEqual(smallest_subnormal, 0.0)
10 changes: 5 additions & 5 deletions src/zodbpickle/tests/test_pickle.py
Original file line number Diff line number Diff line change
Expand Up @@ -43,10 43,10 @@ def test_dumps(self):
def test_suite():
import sys
if sys.version_info[0] >= 3:
from .test_pickle_3 import test_suite
from .pickle_3_tests import test_suite
else:
from .test_pickle_2 import test_suite
return unittest.TestSuite((
from .pickle_2_tests import test_suite
return unittest.TestSuite([
test_suite(),
unittest.makeSuite(TestImportability),
))
unittest.defaultTestLoader.loadTestsFromName(__name__),
])
11 changes: 3 additions & 8 deletions tox.ini
Original file line number Diff line number Diff line change
Expand Up @@ -18,18 18,13 @@ envlist =

[testenv]
usedevelop = true
pip_pre = true
pip_pre = py311: true
deps =
setenv =
pure: PURE_PYTHON=1
!pure-!pypy-!pypy3: PURE_PYTHON=0
commands =
# there's test_pickle_2 for Python 2 and test_pickle_3 for Python 3
# different test files because metaclass syntax differs on Pythons
# test_pickle picks and returns the appropriate one
# without this we'd get an import error (actually syntax error) for one
# of them
zope-testrunner --test-path=src -m zodbpickle.tests.test_pickle$ {posargs:-vc}
zope-testrunner --test-path=src {posargs:-vc}
extras =
test

Expand All @@ -43,7 38,7 @@ setenv =
PURE_PYTHON=1
commands =
mkdir -p {toxinidir}/parts/htmlcov
coverage run -m zope.testrunner --test-path=src -m zodbpickle.tests.test_pickle$ {posargs:-vc}
coverage run -m zope.testrunner --test-path=src {posargs:-vc}
coverage html -i
coverage report -i -m --fail-under=63

Expand Down