Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Reproducibility issues when using th PEP517 backend with C extensions #703

Closed
christian-heusel opened this issue Nov 5, 2024 · 3 comments

Comments

@christian-heusel
Copy link

Hello everyone 👋🏻

I maintain the ueberzug package in Arch Linux that recently switched to using meson-python. Now that I have migrated the package to the new build backend on the new release the repo package has become unreproducible. This is most likely due to a path embedding in the package / debug package.

Since this seems to be considered a bug in meson I thought I'd just open an issue here:

Meson aims to support reproducible builds out of the box with zero additional work (assuming the rest of the build environment is set up for reproducibility). If you ever find a case where this is not happening, it is a bug. Please file an issue with as much information as possible and we'll get it fixed.

https://mesonbuild.com/Reproducible-builds.html

The diff on the package itself contains not much information:

--- /tmp/rebuilderddAxlb8/inputs/ueberzug-18.3.0-1-x86_64.pkg.tar.zst
+++ /tmp/rebuilderddAxlb8/out/ueberzug-18.3.0-1-x86_64.pkg.tar.zst
├── ueberzug-18.3.0-1-x86_64.pkg.tar
│ ├── file list
│ │ @@ -1,9 +1,9 @@
│ │  -rw-r--r--   0 root         (0) root         (0)     6709 2024-11-04 09:38:44.000000 .BUILDINFO
│ │ --rw-r--r--   0 root         (0) root         (0)     3346 2024-11-04 09:38:44.000000 .MTREE
│ │ +-rw-r--r--   0 root         (0) root         (0)     3344 2024-11-04 09:38:44.000000 .MTREE
│ │  -rw-r--r--   0 root         (0) root         (0)      657 2024-11-04 09:38:44.000000 .PKGINFO
│ │  drwxr-xr-x   0 root         (0) root         (0)        0 2024-11-04 09:38:44.000000 usr/
│ │  drwxr-xr-x   0 root         (0) root         (0)        0 2024-11-04 09:38:44.000000 usr/bin/
│ │  -rwxr-xr-x   0 root         (0) root         (0)      215 2024-11-04 09:38:44.000000 usr/bin/ueberzug
│ │  drwxr-xr-x   0 root         (0) root         (0)        0 2024-11-04 09:38:44.000000 usr/lib/
│ │  drwxr-xr-x   0 root         (0) root         (0)        0 2024-11-04 09:38:44.000000 usr/lib/python3.12/
│ │  drwxr-xr-x   0 root         (0) root         (0)        0 2024-11-04 09:38:44.000000 usr/lib/python3.12/site-packages/
│ ├── .MTREE
│ │ ├── .MTREE-content
│ │ │ @@ -7,15 +7,15 @@
│ │ │  ./usr/bin time=1730713124.0 type=dir
│ │ │  ./usr/bin/ueberzug time=1730713124.0 size=215 sha256digest=b522624b354c337d1df752b23d358cccd7711adb83fef3e85cdccf3403d6b210
│ │ │  ./usr/lib time=1730713124.0 type=dir
│ │ │  ./usr/lib/python3.12 time=1730713124.0 type=dir
│ │ │  ./usr/lib/python3.12/site-packages time=1730713124.0 type=dir
│ │ │  /set mode=644
│ │ │  ./usr/lib/python3.12/site-packages/ueberzug time=1730713124.0 mode=755 type=dir
│ │ │ -./usr/lib/python3.12/site-packages/ueberzug/X.cpython-312-x86_64-linux-gnu.so time=1730713124.0 mode=755 size=29056 sha256digest=7a898e301c3b8758eb54cd10f8c20543fef7a20b49ca2a5b6a733f84efc0f4c2
│ │ │ +./usr/lib/python3.12/site-packages/ueberzug/X.cpython-312-x86_64-linux-gnu.so time=1730713124.0 mode=755 size=29056 sha256digest=5110efe2c8729a583a2ebfbbf152a76afe691897dbce4cf787c376896bc41526
│ │ │  ./usr/lib/python3.12/site-packages/ueberzug/__init__.py time=1730713124.0 size=339 sha256digest=95e90a97c28dd1c465d97ddb8b466dececfa78c99a010bc120654e8bfce6f065
│ │ │  ./usr/lib/python3.12/site-packages/ueberzug/__main__.py time=1730713124.0 mode=755 size=1823 sha256digest=42c89d2b99fbd4c94f410ef13bd61c2b7d311fb41679b21aaf55ce2dc68a7116
│ │ │  ./usr/lib/python3.12/site-packages/ueberzug/action.py time=1730713124.0 size=11430 sha256digest=34bad4d4f30de325406b76d4150f668834432de9ef17e235a0aa3d3eb1c10d22
│ │ │  ./usr/lib/python3.12/site-packages/ueberzug/batch.py time=1730713124.0 size=8269 sha256digest=1a07cd958339477b2cc76a75ea618fc50e2335edf699df422d59db3006361443
│ │ │  ./usr/lib/python3.12/site-packages/ueberzug/conversion.py time=1730713124.0 size=1888 sha256digest=943aa4b945a4ce556250276c1946c1c7879ce65cad570ca87ccb543e6b451fbe
│ │ │  ./usr/lib/python3.12/site-packages/ueberzug/files.py time=1730713124.0 size=1226 sha256digest=817b4c0b7a7d9c2b713c3f1757a09c3bf06b7a984861ae9c2c15f62e11cca3bc
│ │ │  ./usr/lib/python3.12/site-packages/ueberzug/geometry.py time=1730713124.0 size=522 sha256digest=46fa80346163bb28571aef981c7cd812a64ff5e46d3a570adfe8e5bc0c1ed22f
│ │ │ @@ -86,10 +86,10 @@
│ │ │  ./usr/lib/python3.12/site-packages/ueberzug/lib/v0/__init__.py time=1730713124.0 size=13530 sha256digest=3328fca102caf101b9402ac2392910be247e7606ec5fef499bafb4afedc6f181
│ │ │  ./usr/lib/python3.12/site-packages/ueberzug/lib/v0/__pycache__ time=1730713124.0 mode=755 type=dir
│ │ │  ./usr/lib/python3.12/site-packages/ueberzug/lib/v0/__pycache__/__init__.cpython-312.opt-1.pyc time=1730713124.0 size=21498 sha256digest=808aec8b9f8693cc53eef6bfd635b6a7bcdea105ceee43de519a52a5af4c5059
│ │ │  ./usr/lib/python3.12/site-packages/ueberzug/lib/v0/__pycache__/__init__.cpython-312.pyc time=1730713124.0 size=21526 sha256digest=c1595fe23bdde7ed7335724c4069c94ccd08f3bb4be79e79f68e730dd26ea296
│ │ │  ./usr/lib/python3.12/site-packages/ueberzug-18.3.0.dist-info time=1730713124.0 mode=755 type=dir
│ │ │  ./usr/lib/python3.12/site-packages/ueberzug-18.3.0.dist-info/LICENSE time=1730713124.0 size=35147 sha256digest=8ceb4b9ee5adedde47b31e975c1d90c73ad27b6b165a1dcd80c7c545eb65b903
│ │ │  ./usr/lib/python3.12/site-packages/ueberzug-18.3.0.dist-info/METADATA time=1730713124.0 size=41966 sha256digest=608a1cf2e0d316e102f5aba7f1eca10bc093147072f8e616e77080a936f2a335
│ │ │ -./usr/lib/python3.12/site-packages/ueberzug-18.3.0.dist-info/RECORD time=1730713124.0 size=2409 sha256digest=22e82ea7bafe22b26b942f4dc408d047fe8a1b979f5c027f0127bba195b95140
│ │ │ +./usr/lib/python3.12/site-packages/ueberzug-18.3.0.dist-info/RECORD time=1730713124.0 size=2409 sha256digest=3d1cafaaf355c84bae41b0dcc56b4e16f77ee02465efe95502e923a650690262
│ │ │  ./usr/lib/python3.12/site-packages/ueberzug-18.3.0.dist-info/WHEEL time=1730713124.0 size=88 sha256digest=634159ee5c5f58a5b2d30fe69e35e1149cf5f2be37f307e1642a9742d342a6f5
│ │ │  ./usr/lib/python3.12/site-packages/ueberzug-18.3.0.dist-info/entry_points.txt time=1730713124.0 size=53 sha256digest=38fb85108bb7f654f5f65de387720ea401501a2d6d241570c9d43d7ff6790217
│ ├── usr/lib/python3.12/site-packages/ueberzug-18.3.0.dist-info/RECORD
│ │ @@ -1,13 +1,13 @@
│ │  ../../../bin/ueberzug,sha256=tSJiSzVMM30d91KyPTWMzNdxGtuD_vPoXNzPNAPWshA,215
│ │  ueberzug-18.3.0.dist-info/METADATA,sha256=YIoc8uDTFuEC9aun8eyhC8CTFHBy-OYW53CAqTbyozU,41966
│ │  ueberzug-18.3.0.dist-info/WHEEL,sha256=Y0FZ7lxfWKWy0w_mnjXhFJz18r438wfhZCqXQtNCpvU,88
│ │  ueberzug-18.3.0.dist-info/entry_points.txt,sha256=OPuFEIu39lT19l3jh3IOpAFQGi1tJBVwydQ9f_Z5Ahc,53
│ │  ueberzug-18.3.0.dist-info/LICENSE,sha256=jOtLnuWt7d5Hsx6XXB2QxzrSe2sWWh3NgMfFRetluQM,35147
│ │ -ueberzug/X.cpython-312-x86_64-linux-gnu.so,sha256=k7X1ee65C_vhwZfKQB4MAfhM4aftyaHtkcEmoVKR_9o,113784
│ │ +ueberzug/X.cpython-312-x86_64-linux-gnu.so,sha256=XyiOVWJtrFh_4wb_eL02EOETLoROQcE_pC4lpll3EHw,113784
│ │  ueberzug/__init__.py,sha256=lekKl8KN0cRl2X3bi0Zt7Oz6eMmaAQvBIGVOi_zm8GU,339
│ │  ueberzug/__main__.py,sha256=QsidK5n71MlPQQ7xO9YcK30xH7QWebIar1XOLcaKcRY,1823
│ │  ueberzug/action.py,sha256=NLrU1PMN4yVAa3bUFQ9miDRDLenvF-I1oKo9PrHBDSI,11430
│ │  ueberzug/batch.py,sha256=GgfNlYM5R3ssx2p16mGPxQ4jNe32md9CLVnbMAY2FEM,8269
│ │  ueberzug/conversion.py,sha256=lDqkuUWkzlViUCdsGUbBx4ec5lytVwyofMtUPmtFH74,1888
│ │  ueberzug/files.py,sha256=gXtMC3p9nCtxPD8XV6CcO_BrephIYa6cLBX2LhHMo7w,1226
│ │  ueberzug/geometry.py,sha256=RvqANGFjuyhXGu-YHHzYEqZP9eRtOlcK3-jlvAwe0i8,522
│ ├── usr/lib/python3.12/site-packages/ueberzug/X.cpython-312-x86_64-linux-gnu.so
│ │ ├── readelf --wide --notes {}
│ │ │ @@ -1,8 +1,8 @@
│ │ │  
│ │ │  Displaying notes found in: .note.gnu.property
│ │ │    Owner                Data size 	Description
│ │ │    GNU                  0x00000030	NT_GNU_PROPERTY_TYPE_0	      Properties: x86 feature: IBT, SHSTK, x86 feature used: x86, XMM, x86 ISA used: x86-64-baseline
│ │ │  
│ │ │  Displaying notes found in: .note.gnu.build-id
│ │ │    Owner                Data size 	Description
│ │ │ -  GNU                  0x00000014	NT_GNU_BUILD_ID (unique build ID bitstring)	    Build ID: d0c2bc0ad8235c8ff0e4b4911edb0733baf37994
│ │ │ +  GNU                  0x00000014	NT_GNU_BUILD_ID (unique build ID bitstring)	    Build ID: 8cddf4baae52f965ebb93a93981c0c682cbf1041
│ │ ├── readelf --wide --decompress --hex-dump=.gnu_debuglink {}
│ │ │ @@ -1,6 +1,6 @@
│ │ │  
│ │ │  Hex dump of section '.gnu_debuglink':
│ │ │    0x00000000 582e6370 7974686f 6e2d3331 322d7838 X.cpython-312-x8
│ │ │    0x00000010 365f3634 2d6c696e 75782d67 6e752e73 6_64-linux-gnu.s
│ │ │ -  0x00000020 6f2e6465 62756700 afaf760e          o.debug...v.
│ │ │ +  0x00000020 6f2e6465 62756700 928a3b0f          o.debug...;.

But the diffs on the debug packages reveals that there is most likely an issue with the changing path:

│ │ ├── readelf --wide --notes {}
│ │ │ @@ -1,8 +1,8 @@
│ │ │  
│ │ │  Displaying notes found in: .note.gnu.property
│ │ │    Owner                Data size 	Description
│ │ │    GNU                  0x00000030	NT_GNU_PROPERTY_TYPE_0	      Properties: x86 feature: IBT, SHSTK, x86 feature used: x86, XMM, x86 ISA used: x86-64-baseline
│ │ │  
│ │ │  Displaying notes found in: .note.gnu.build-id
│ │ │    Owner                Data size 	Description
│ │ │ -  GNU                  0x00000014	NT_GNU_BUILD_ID (unique build ID bitstring)	    Build ID: 4f2aff58a906e4ad8ae23382df717094116fe451
│ │ │ +  GNU                  0x00000014	NT_GNU_BUILD_ID (unique build ID bitstring)	    Build ID: 4483c37f5316e95346fdff5540a0e7331f1b812e
│ │ ├── readelf --wide --debug-dump=rawline {}
│ │ │ @@ -25,15 +25,15 @@
│ │ │    Opcode 9 has 1 arg
│ │ │    Opcode 10 has 0 args
│ │ │    Opcode 11 has 0 args
│ │ │    Opcode 12 has 1 arg
│ │ │  
│ │ │   The Directory Table (offset 0x22, lines 8, columns 1):
│ │ │    Entry	Name
│ │ │ -  0	(line_strp)	(offset: 0x12): /usr/src/debug/ueberzug/ueberzug-18.3.0/.mesonpy-xl7i9lgh
│ │ │ +  0	(line_strp)	(offset: 0x12): /usr/src/debug/ueberzug/ueberzug-18.3.0/.mesonpy-v7qtp5jb
│ │ │    1	(line_strp)	(offset: 0x4c): ../ueberzug/X
│ │ │    2	(line_strp)	(offset: 0x5a): /usr/include/bits
│ │ │    3	(line_strp)	(offset: 0x6c): /usr/include/python3.12
│ │ │    4	(line_strp)	(offset: 0x84): /usr/lib/gcc/x86_64-pc-linux-gnu/14.2.1/include
│ │ │    5	(line_strp)	(offset: 0xb4): /usr/include/sys
│ │ │    6	(line_strp)	(offset: 0xc5): /usr/include/python3.12/cpython
│ │ │    7	(line_strp)	(offset: 0xe5): /usr/include/X11
│ │ │ @@ -302,15 +302,15 @@
│ │ │    Opcode 9 has 1 arg
│ │ │    Opcode 10 has 0 args
│ │ │    Opcode 11 has 0 args
│ │ │    Opcode 12 has 1 arg
│ │ │  
│ │ │   The Directory Table (offset 0x22d, lines 11, columns 1):
│ │ │    Entry	Name
│ │ │ -  0	(line_strp)	(offset: 0x12): /usr/src/debug/ueberzug/ueberzug-18.3.0/.mesonpy-xl7i9lgh
│ │ │ +  0	(line_strp)	(offset: 0x12): /usr/src/debug/ueberzug/ueberzug-18.3.0/.mesonpy-v7qtp5jb
│ │ │    1	(line_strp)	(offset: 0x4c): ../ueberzug/X
│ │ │    2	(line_strp)	(offset: 0x6c): /usr/include/python3.12
│ │ │    3	(line_strp)	(offset: 0x5a): /usr/include/bits
│ │ │    4	(line_strp)	(offset: 0x84): /usr/lib/gcc/x86_64-pc-linux-gnu/14.2.1/include
│ │ │    5	(line_strp)	(offset: 0xb4): /usr/include/sys
│ │ │    6	(line_strp)	(offset: 0x1ba): /usr/include
│ │ │    7	(line_strp)	(offset: 0xc5): /usr/include/python3.12/cpython
│ │ │ @@ -1433,15 +1433,15 @@
│ │ │    Opcode 9 has 1 arg
│ │ │    Opcode 10 has 0 args
│ │ │    Opcode 11 has 0 args
│ │ │    Opcode 12 has 1 arg
│ │ │  
│ │ │   The Directory Table (offset 0xa29, lines 10, columns 1):
│ │ │    Entry	Name
│ │ │ -  0	(line_strp)	(offset: 0x12): /usr/src/debug/ueberzug/ueberzug-18.3.0/.mesonpy-xl7i9lgh
│ │ │ +  0	(line_strp)	(offset: 0x12): /usr/src/debug/ueberzug/ueberzug-18.3.0/.mesonpy-v7qtp5jb
│ │ │    1	(line_strp)	(offset: 0x4c): ../ueberzug/X
│ │ │    2	(line_strp)	(offset: 0x6c): /usr/include/python3.12
│ │ │    3	(line_strp)	(offset: 0x5a): /usr/include/bits
│ │ │    4	(line_strp)	(offset: 0x84): /usr/lib/gcc/x86_64-pc-linux-gnu/14.2.1/include
│ │ │    5	(line_strp)	(offset: 0xb4): /usr/include/sys
│ │ │    6	(line_strp)	(offset: 0xc5): /usr/include/python3.12/cpython
│ │ │    7	(line_strp)	(offset: 0xe5): /usr/include/X11
│ │ │ @@ -2639,15 +2639,15 @@
│ │ │    Opcode 9 has 1 arg
│ │ │    Opcode 10 has 0 args
│ │ │    Opcode 11 has 0 args
│ │ │    Opcode 12 has 1 arg
│ │ │  
│ │ │   The Directory Table (offset 0x1251, lines 10, columns 1):
│ │ │    Entry	Name
│ │ │ -  0	(line_strp)	(offset: 0x12): /usr/src/debug/ueberzug/ueberzug-18.3.0/.mesonpy-xl7i9lgh
│ │ │ +  0	(line_strp)	(offset: 0x12): /usr/src/debug/ueberzug/ueberzug-18.3.0/.mesonpy-v7qtp5jb
│ │ │    1	(line_strp)	(offset: 0x4c): ../ueberzug/X
│ │ │    2	(line_strp)	(offset: 0x6c): /usr/include/python3.12
│ │ │    3	(line_strp)	(offset: 0x5a): /usr/include/bits
│ │ │    4	(line_strp)	(offset: 0x84): /usr/lib/gcc/x86_64-pc-linux-gnu/14.2.1/include
│ │ │    5	(line_strp)	(offset: 0xb4): /usr/include/sys
│ │ │    6	(line_strp)	(offset: 0x1ba): /usr/include
│ │ │    7	(line_strp)	(offset: 0xc5): /usr/include/python3.12/cpython

The package is built as follows (see the full PKGBUILD):

build() {
    cd $pkgname-$pkgver
    python -m build --wheel --no-isolation
}

package() {
    cd $pkgname-$pkgver
    python -m installer --destdir="$pkgdir" dist/*.whl
}

If there is any more information needed feel free to ask, I hope I provided all the details! 🤗

@dnicolodi
Copy link
Member

It is due to the build path being recorded into debug symbols. See #671. You can make the path deterministic setting a well defined build directory: python -m build --wheel --no-isolation -Cbuild-dir=build.

@eli-schwartz
Copy link
Member

(@dnicolodi This ticket should be closed as a duplicate.)

Indeed, passing the correct arguments you should be passing anyway (that is, all the ones you'd normally pass to meson setup is the correct answer here.

@christian-heusel
Copy link
Author

Alright, thanks for the input everyone! 😊 Closing as a duplicate of Issue #671 👍🏻

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants