Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

BUG: mesonpy embeds random path in .pyx files #19101

Closed
bmwiedemann opened this issue Aug 21, 2023 · 3 comments · Fixed by #19123
Closed

BUG: mesonpy embeds random path in .pyx files #19101

bmwiedemann opened this issue Aug 21, 2023 · 3 comments · Fixed by #19123
Labels
Build issues Issues with building from source, including different choices of architecture, compilers and OS defect A clear bug or issue that prevents SciPy from being installed or used as expected Meson Items related to the introduction of Meson as the new build system for SciPy
Milestone
1.11.3

Comments

@bmwiedemann
Copy link
Contributor

Describe your issue.

While working on reproducible builds for openSUSE, I found that
our python-scipy 1.11.1 package varies between builds. One of the reasons is that a random mesonpy path gets embedded in cython_special.pyx, _ufuncs_cxx.pyx and _ufuncs.pyx

Reproducing Code Example

build it twice

Error message

+++ /usr/lib64/python3.9/site-packages/scipy/special/cython_special.pyx     2023-07-27 00:00:00.000000000 +0000
@@ -1088,50 +1088,50 @@
 from ._agm cimport agm as _func_agm
 ctypedef double _proto_agm_t(double, double) noexcept nogil
 cdef _proto_agm_t *_proto_agm_t_var = &_func_agm
-cdef extern from r"/home/abuild/rpmbuild/BUILD/scipy-1.11.1/.mesonpy-nb_0k4ap/build/scipy/special/_ufuncs_defs.h":
+cdef extern from r"/home/abuild/rpmbuild/BUILD/scipy-1.11.1/.mesonpy-il1v1fcx/build/scipy/special/_ufuncs_defs.h":
     cdef npy_int _func_airy_wrap "airy_wrap"(npy_double, npy_double *, npy_double *, npy_double *, npy_double *)nogil
-cdef extern from r"/home/abuild/rpmbuild/BUILD/scipy-1.11.1/.mesonpy-nb_0k4ap/build/scipy/special/_ufuncs_defs.h":
+cdef extern from r"/home/abuild/rpmbuild/BUILD/scipy-1.11.1/.mesonpy-il1v1fcx/build/scipy/special/_ufuncs_defs.h":

SciPy/NumPy/Python version and system information

1.11.1 1.25.2 3.11.4
@bmwiedemann bmwiedemann added the defect A clear bug or issue that prevents SciPy from being installed or used as expected label Aug 21, 2023
@j-bowhay j-bowhay added the Meson Items related to the introduction of Meson as the new build system for SciPy label Aug 21, 2023
@dschmitz89 dschmitz89 added the Build issues Issues with building from source, including different choices of architecture, compilers and OS label Aug 22, 2023
@rgommers
Copy link
Member

Thank you for the report @bmwiedemann. I agree that that can be considered a bug.

The issue here stems from this line in scipy/special/_generate_pyx.py:

defs.append(f"cdef extern from r"{proto_h_filename}":")

where proto_h_filename is an absolute path. Given that meson-python creates a tmpdir, absolute paths in installed files are no good.

However, the more desirable fix here is to not install this header at all, since we don"t need it in the installed package. For why we currently install it, see:

scipy/scipy/special/meson.build

Lines 287 to 291 in f8036bc

# TODO: this installs all generated files, while we want to install only:
# _ufuncs.pyi, cython_special.pxd
# There"s no good way around this currently, because py3.install_sources
# doesn"t accept generated targets. See TODO near the end of
# scipy/linalg/meson.build for more details.

@rgommers
Copy link
Member

In this case, we should be able to change _generate_pyx.py to use a relative path though, since the referenced header is located right next to the .pyx file. So that"s the easier fix here.

@rgommers
Copy link
Member

There"s a bunch of other code gen scripts with cdef extern from, but the ones I checked all used relative paths.

@bmwiedemann bmwiedemann mentioned this issue Aug 24, 2023
Merged
andyfaff pushed a commit that referenced this issue Aug 24, 2023
@bmwiedemann
BLD: Avoid absolute pathnames in .pyx files
29657e2 
to allow for reproducible builds
even when (unneeded) .pyx files are installed and shipped in OS packages

Fixes: #19101

This patch was done while working on reproducible builds for openSUSE.
dschult pushed a commit to dschult/scipy that referenced this issue Aug 25, 2023
@bmwiedemann @dschult
BLD: Avoid absolute pathnames in .pyx files
704e426 
to allow for reproducible builds
even when (unneeded) .pyx files are installed and shipped in OS packages

Fixes: scipy#19101

This patch was done while working on reproducible builds for openSUSE.
@rgommers rgommers added this to the 1.12.0 milestone Aug 25, 2023
@tylerjereddy tylerjereddy modified the milestones: 1.12.0, 1.11.3 Sep 21, 2023
tylerjereddy pushed a commit to tylerjereddy/scipy that referenced this issue Sep 21, 2023
@bmwiedemann @tylerjereddy
BLD: Avoid absolute pathnames in .pyx files
a7023a4 
to allow for reproducible builds
even when (unneeded) .pyx files are installed and shipped in OS packages

Fixes: scipy#19101

This patch was done while working on reproducible builds for openSUSE.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
Build issues Issues with building from source, including different choices of architecture, compilers and OS defect A clear bug or issue that prevents SciPy from being installed or used as expected Meson Items related to the introduction of Meson as the new build system for SciPy
Projects
None yet
Milestone
1.11.3
Development

Successfully merging a pull request may close this issue.

BLD: Avoid absolute pathnames in .pyx files bmwiedemann/scipy
5 participants
@rgommers @bmwiedemann @tylerjereddy @dschmitz89 @j-bowhay