888401">

Debian Bug report logs - #888401
diffoscope: report differences in filesystem extended attributes

version graph

Package: diffoscope; Maintainer for diffoscope is Reproducible builds folks <reproducible-builds@lists.alioth.debian.org>; Source for diffoscope is src:diffoscope (PTS, buildd, popcon).

Reported by: Paul Wise <pabs@debian.org>

Date: Thu, 25 Jan 2018 03:51:02 UTC

Severity: wishlist

Found in version diffoscope/90

Fixed in version diffoscope/91

Done: Mattia Rizzolo <mattia@debian.org>

Bug is archived. No further changes may be made.

View this report as an mbox folder, status mbox, maintainer mbox


Report forwarded to debian-bugs-dist@lists.debian.org, Reproducible builds folks <reproducible-builds@lists.alioth.debian.org>:
Bug#888401; Package diffoscope. (Thu, 25 Jan 2018 03:51:04 GMT) (full text, mbox, link).


Acknowledgement sent to Paul Wise <pabs@debian.org>:
New Bug report received and forwarded. Copy sent to Reproducible builds folks <reproducible-builds@lists.alioth.debian.org>. (Thu, 25 Jan 2018 03:51:04 GMT) (full text, mbox, link).


Message #5 received at submit@bugs.debian.org (full text, mbox, > -- > bye, > pabs > > https://wiki.debian.org/PaulWise &In-Reply-To=<1516852134.27073.13.camel@debian.org>">reply):

From: Paul Wise <pabs@debian.org>
To: Debian Bug Tracking System <submit@bugs.debian.org>
Subject: diffoscope: report differences in filesystem extended attributes
Date: Thu, 25 Jan 2018 11:48:54 +0800
[Message part 1 (text/plain, inline)]
Package: diffoscope
Version: 90
Severity: wishlist

It would be nice if diffoscope could report differences in filesystem
xattrs. This would help when comparing directories with files that have
been downloaded with wget (or other downloaders that set xattrs).

In Debian we have two Python 3 modules for xattr support:

python3-xattr python3-pyxattr

~ $ mkdir 1 2
~ $ cd 1
~/1 $ wget -q https://www.debian.org/
~/1 $ cd ..
~ $ cp 1/index.html 2/
~ $ diffoscope --no-progress 1 2
--- 1
+++ 2
+-- stat {}
| @@ -1,8 +1,8 @@
|  
|    Size: 4096      	Blocks: 8          IO Block: 4096   directory
|  Links: 2
|  Access: (0750/drwxr-x---)  Uid: ( 1000/    pabs)   Gid: ( 1000/    pabs)
|  
| -Modify: 2018-01-25 03:27:33.279766812 +0000
| +Modify: 2018-01-25 03:27:45.971611107 +0000
|  
|   Birth: -
|   --- 1/index.html
+-- +++ 2/index.html
| +-- stat {}
| | @@ -1,8 +1,8 @@
| |  
| |    Size: 15060     	Blocks: 32         IO Block: 4096   regular file
| |  Links: 1
| |  Access: (0640/-rw-r-----)  Uid: ( 1000/    pabs)   Gid: ( 1000/    pabs)
| |  
| | -Modify: 2018-01-24 23:45:55.000000000 +0000
| | +Modify: 2018-01-25 03:27:45.971611107 +0000
| |  
| |   Birth: -
~ $ getfattr -dR 1 2
# file: 1/index.html
user.xdg.origin.url="https://www.debian.org/"

-- 
bye,
pabs

https://wiki.debian.org/PaulWise
[signature.asc (application/pgp-signature, inline)]

Information forwarded to debian-bugs-dist@lists.debian.org, Reproducible builds folks <reproducible-builds@lists.alioth.debian.org>:
Bug#888401; Package diffoscope. (Thu, 25 Jan 2018 04:27:02 GMT) (full text, mbox, link).


Acknowledgement sent to Chris Lamb <lamby@debian.org>:
Extra info received and forwarded to list. Copy sent to Reproducible builds folks <reproducible-builds@lists.alioth.debian.org>. (Thu, 25 Jan 2018 04:27:02 GMT) (full text, mbox, link).


Message #10 received at 888401@bugs.debian.org (full text, mbox, reply):

From: Chris Lamb <lamby@debian.org>
To: 888401@bugs.debian.org
Cc: Paul Wise <pabs@debian.org>
Subject: Re: diffoscope: report differences in filesystem extended attributes
Date: Thu, 25 Jan 2018 09:53:29 +0530
tags 888401 + pending
tags 888402 - pending
thanks

Hi Paul,

Oops, I got #888401 and #888402 confused as they were submitted so soon
together and with similar names. Anyway, I've implemented the extended
metadata stuff here:

  https://anonscm.debian.org/git/reproducible/diffoscope.git/commit/?id=8f8bc714271132afce2edbb32150f73504fc67d0

For example:

  --- 1
  +++ 2
  ├── stat {}
  │ @@ -1,8 +1,8 @@
  │  
  │    Size: 4096      	Blocks: 8          IO Block: 4096   directory
  │  Links: 2
  │  Access: (0755/drwxr-xr-x)  Uid: ( 1000/   lamby)   Gid: ( 1000/   lamby)
  │  
  │ -Modify: 2018-01-25 03:55:24.736417351 +0000
  │ +Modify: 2018-01-25 03:55:32.616473909 +0000
  │  
  │   Birth: -
  │   --- 1/index.html
  ├── +++ 2/index.html
  │ ├── stat {}
  │ │ @@ -1,8 +1,8 @@
  │ │  
  │ │    Size: 15060     	Blocks: 32         IO Block: 4096   regular file
  │ │  Links: 1
  │ │  Access: (0644/-rw-r--r--)  Uid: ( 1000/   lamby)   Gid: ( 1000/   lamby)
  │ │  
  │ │ -Modify: 2018-01-24 23:45:55.000000000 +0000
  │ │ +Modify: 2018-01-25 03:55:32.616473909 +0000
  │ │  
  │ │   Birth: -
  │ ├── extended file attributes
  │ │ @@ -1 +0,0 @@
  │ │ -user.xdg.origin.url: https://www.debian.org/

Enjoy!


Best wishes,

-- 
      ,''`.
     : :'  :     Chris Lamb
     `. `'`      lamby@debian.org / chris-lamb.co.uk
       `-



Added tag(s) pending. Request was from Chris Lamb <lamby@debian.org> to control@bugs.debian.org. (Thu, 25 Jan 2018 04:27:04 GMT) (full text, mbox, link).


Information forwarded to debian-bugs-dist@lists.debian.org, Reproducible builds folks <reproducible-builds@lists.alioth.debian.org>:
Bug#888401; Package diffoscope. (Thu, 25 Jan 2018 07:33:03 GMT) (full text, mbox, link).


Acknowledgement sent to Niels Thykier <niels@thykier.net>:
Extra info received and forwarded to list. Copy sent to Reproducible builds folks <reproducible-builds@lists.alioth.debian.org>. (Thu, 25 Jan 2018 07:33:03 GMT) (full text, mbox, link).


Message #17 received at 888401@bugs.debian.org (full text, mbox, does xattr.get_all return values in a sorted/deterministic order? > > Thanks, > ~Niels > > > &References=<1516854209.2250895.1247393992.2B13E52A@webmail.messagingengine.com> <1516854209.2250895.1247393992.2B13E52A@webmail.messagingengine.com> ">reply):

From: Niels Thykier <niels@thykier.net>
To: 888401@bugs.debian.org, Chris Lamb <lamby@debian.org>
Subject: Re: diffoscope: report differences in filesystem extended attributes
Date: Thu, 25 Jan 2018 07:28:00 +0000
On Thu, 25 Jan 2018 09:53:29 +0530 Chris Lamb <lamby@debian.org> wrote:
> tags 888401 + pending
> tags 888402 - pending
> thanks
> 
> Hi Paul,
> 
> Oops, I got #888401 and #888402 confused as they were submitted so soon
> together and with similar names. Anyway, I've implemented the extended
> metadata stuff here:
> 
>   https://anonscm.debian.org/git/reproducible/diffoscope.git/commit/?id=8f8bc714271132afce2edbb32150f73504fc67d0
> 
> For example:
> 
> [...]
> 
> Enjoy!
> 
> 
> Best wishes,
> 
> -- 
>       ,''`.
>      : :'  :     Chris Lamb
>      `. `'`      lamby@debian.org / chris-lamb.co.uk
>        `-
> 
> 

Hi,

I am curious; the code seems like it implicitly rely on xattr.get_all
returning keys in sorted/deterministic order to get a "non-noisy" diff,
does xattr.get_all return values in a sorted/deterministic order?

Thanks,
~Niels




Information forwarded to debian-bugs-dist@lists.debian.org, Reproducible builds folks <reproducible-builds@lists.alioth.debian.org>:
Bug#888401; Package diffoscope. (Thu, 25 Jan 2018 08:39:05 GMT) (full text, mbox, link).


Acknowledgement sent to Chris Lamb <lamby@debian.org>:
Extra info received and forwarded to list. Copy sent to Reproducible builds folks <reproducible-builds@lists.alioth.debian.org>. (Thu, 25 Jan 2018 08:39:05 GMT) (full text, mbox, link).


Message #22 received at 888401@bugs.debian.org (full text, mbox, > does xattr.get_all return values in a sorted/deterministic order? > > When initially implementing this, I thought I saw a sorting call in > the pyxattr code. However, following the chain of: > > https://sources.debian.org/src/python-pyxattr/0.6.0-2/xattr.c/#L597 > | > https://sources.debian.org/src/python-pyxattr/0.6.0-2/xattr.c/#L276 > | > https://sources.debian.org/src/python-pyxattr/0.6.0-2/xattr.c/#L216 > | > https://sources.debian.org/src/attr/1:2.4.47-2/man/man2/listxattr.2/#L99-L101 > > .. it's unordered. Fixed here: > > https://anonscm.debian.org/git/reproducible/diffoscope.git/commit/?id=5fcd71891aad9b7ed5195f490e7e42c3a12fa3d0 > > Thanks for the diligent review as usual! > > > Best wishes, > > -- > ,''`. > : :' : Chris Lamb > `. `'` lamby@debian.org / chris-lamb.co.uk > `- > > &References=<1516854209.2250895.1247393992.2B13E52A@webmail.messagingengine.com> <1516854209.2250895.1247393992.2B13E52A@webmail.messagingengine.com> <1516869376.1530042.1247540528.184A39EC@webmail.messagingengine.com>">reply):

From: Chris Lamb <lamby@debian.org>
To: Niels Thykier <niels@thykier.net>, 888401@bugs.debian.org
Subject: Re: diffoscope: report differences in filesystem extended attributes
Date: Thu, 25 Jan 2018 14:06:16 +0530
Hey Niels,

> I am curious; the code seems like it implicitly rely on xattr.get_all
> returning keys in sorted/deterministic order to get a "non-noisy" diff,
> does xattr.get_all return values in a sorted/deterministic order?

When initially implementing this, I thought I saw a sorting call in
the pyxattr code. However, following the chain of:

  https://sources.debian.org/src/python-pyxattr/0.6.0-2/xattr.c/#L597
  |
  https://sources.debian.org/src/python-pyxattr/0.6.0-2/xattr.c/#L276
  |
  https://sources.debian.org/src/python-pyxattr/0.6.0-2/xattr.c/#L216
  |
  https://sources.debian.org/src/attr/1:2.4.47-2/man/man2/listxattr.2/#L99-L101

.. it's unordered. Fixed here:

  https://anonscm.debian.org/git/reproducible/diffoscope.git/commit/?id=5fcd71891aad9b7ed5195f490e7e42c3a12fa3d0

Thanks for the diligent review as usual!


Best wishes,

-- 
      ,''`.
     : :'  :     Chris Lamb
     `. `'`      lamby@debian.org / chris-lamb.co.uk
       `-



Reply sent to Mattia Rizzolo <mattia@debian.org>:
You have taken responsibility. (Thu, 01 Mar 2018 13:09:25 GMT) (full text, mbox, link).


Notification sent to Paul Wise <pabs@debian.org>:
Bug acknowledged by developer. (Thu, 01 Mar 2018 13:09:25 GMT) (full text, mbox, link).


Message #27 received at 888401-close@bugs.debian.org (full text, mbox, reply):

From: Mattia Rizzolo <mattia@debian.org>
To: 888401-close@bugs.debian.org
Subject: Bug#888401: fixed in diffoscope 91
Date: Thu, 01 Mar 2018 13:05:09 +0000
Source: diffoscope
Source-Version: 91

We believe that the bug you reported is fixed in the latest version of
diffoscope, which is due to be installed in the Debian FTP archive.

A summary of the changes between this version and the previous one is
attached.

Thank you for reporting the bug, which will now be closed.  If you
have further comments please address them to 888401@bugs.debian.org,
and the maintainer will reopen the bug report if appropriate.

Debian distribution maintenance software
pp.
Mattia Rizzolo <mattia@debian.org> (supplier of updated diffoscope package)

(This message was generated automatically at their request; if you
believe that there is a problem with it please contact the archive
administrators by mailing ftpmaster@ftp-master.debian.org)


-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA512

Format: 1.8
Date: Thu, 01 Mar 2018 13:40:48 +0100
Source: diffoscope
Binary: diffoscope
Architecture: source
Version: 91
Distribution: unstable
Urgency: medium
Maintainer: Reproducible builds folks <reproducible-builds@lists.alioth.debian.org>
Changed-By: Mattia Rizzolo <mattia@debian.org>
Description:
 diffoscope - in-depth comparison of files, archives, and directories
Closes: 849386 863879 863880 877726 886736 886963 887180 888112 888401 888402 890528
Changes:
 diffoscope (91) unstable; urgency=medium
 .
   [ Mattia Rizzolo ]
   * d/clean: Remove .cache/.
   * diff: Wrap long regular expression for improved readability.
   * comparators.json: Fix UnicodeDecodeError with a non-UTF8 locale.
   * d/copyright: Update for the new year.
   * d/control: Bump Standards-Version to 4.1.3, no changes needed.
   * tests/android:
     + Skip tests involving abootimg on BE archs (see #725729).
 .
   [ Daniel Shahaf ]
   * diffoscope.diff: Document feeders.  Closes: #863880
   * Optimize the common case of feeders.  Closes: #863879
 .
   [ Juliana Oliveira ]
   * {tempfiles, utils.archive}: catch possible FileNotFoundError.
   * comparators.java: add support for procyon-decompiler.  Closes: #849386
   * {command, feeders, diff}: replaces subprocess.Popen() by .run().
     This is done in preparation for work on multiprocessing (#842837).
     + {command, zip, feeders}: replaces .wait by the new .returncode property.
     + comparators.utils.command: replaces .stderr_content by .stderr.
   * tests:
     + test_progress: fix test case for some broken versions of
       python-progressbar.  Closes: #877726
     + utils.tools: add support for modules on skip_unless* annotations.
 .
   [ Chris Lamb ]
   * comparators:
     + utils/compare:
       - Show extended filesystem metadata even when directly comparing two
         files, not just when we specify two directories.  Closes: #888402
     + macho:
       - If the If the LLVM disassembler does not work, try the
         internal one.  Closes: #886736
       - Always strip the filename, not just when by itself.
     + json:
       - Do some cheap fuzzy parsing to detect JSON files not named .json.
       - Also match unicode JSON files.
       - Optionally compare JSONs with the jsondiff module.  Closes: #888112
     + directory:
       - Report differences in extended file attributes when comparing files.
         Closes: #888401
     + xsb:
       - Add support for comparing XMLBeans binary schemas.
     + barkeley_db:
       - Add support for comparing Berkeley DB files.  Closes: #890528
   * Misc code cleaup.
   * tests:
     + comparators.test_elf: Return '0' if we can't parse the readelf
       version number.  Closes: #886963
   * debian:
     + Explicitly build-depend and recommend e2fsprogs.  Closes: #887180
 .
   [ Ximin Luo ]
   * Partially revert the changes done for #888402 to maintain the current.
     behaviour of --exclude-directory-metadata.
   * Refactor how the configuration is loaded:
     + Move the defaults into a Config.reset() method.
     + reset() the configuration at the end of main(), to help the testsuite.
Checksums-Sha1:
 cf78d47c07e0cea4b0d667c02a0beff82b2cb28b 3429 diffoscope_91.dsc
 f10a233ff4469d6405eac3bc6c03da6d33962efb 8935512 diffoscope_91.tar.xz
 1413267eea08eb05b0664a4ba13b46abc50a9df7 20151 diffoscope_91_amd64.buildinfo
Checksums-Sha256:
 28e12196a5e461349d1cfdfa96bb3914337f6bd6327d13637de80a5bedd08169 3429 diffoscope_91.dsc
 12034827db747b831b0c80a340f607c8ac4b85936610d38985138631b8a4f0ee 8935512 diffoscope_91.tar.xz
 a456c23241bed0e2dc6230a03e3f7897ad9a715f54da1e662b4f7864c0804d5f 20151 diffoscope_91_amd64.buildinfo
Files:
 4a1c8bd026d379c91fec4db88eee3679 3429 devel optional diffoscope_91.dsc
 a72769fc6dc5f80c6a31b8730234444a 8935512 devel optional diffoscope_91.tar.xz
 728e5ffa6fff19b50c3baf3dfe0a2ee6 20151 devel optional diffoscope_91_amd64.buildinfo

-----BEGIN PGP SIGNATURE-----

iQIzBAEBCgAdFiEEi3hoeGwz5cZMTQpICBa54Yx2K60FAlqX9qYACgkQCBa54Yx2
K62jGg/+J2JLNEmq/nVoNa182wyzTzerZVy03V5Pc4WBt1pyZ4DPbvDrAhZvWZmP
qBXr54UxkbbNfXa+fYAph4xr7D5xL4olDEOFRcDf5UJfGwdjaravc6CeE1z915eW
IJkFIOd17ySKlxM5RAVQ1xtQqXiCsBBHa4W2XryleNivJOSOKotEbkEul8MrregE
HREtWF76l7t5Mas2bFxD0xjYAfIYgB8vDLCFnsoOHsqUKksf2PYr60ge2CmCLKUO
Lw/gTcXD+T8k6UBoC2HXmU/2Wgsd9KClAOHUWwdJK78akvvjSC+zBfcewW1vtM1P
lVui0SjSmd7K3suNZgNnrCboKpq7J8ULR8g1GikElxV6orkDsmhX5SfqgsiqWsqw
yZbm1VHRBtOawnQUD8U2bLqVZsTOsE5JMF7ehHjGehEgoCgg3oNTbY8cIAWyd8y/
erk43kNYKhYdHbEfpvGRJekSvJOS8BY4h94ThJ1B4dElUhMzESJ5Fa2S/Ik3fyDS
WKWY8rS3WSons4WL+yjvd7Y2hfuf/IWt6XaG2rge3ODJlwD3BxFrLg7NzkNhH7NJ
a1yITYx2JbMahQZQribpkdPiFcx38y0LcAT25br8AY+asa0fr7qPOfshQfytFiIs
bDfMRuWM5I6KsCwvE4Gtjrawrux03EdbB0piGQzWKpfAmO3qqdE=
=L47V
-----END PGP SIGNATURE-----




Bug archived. Request was from Debbugs Internal Request <owner@bugs.debian.org> to internal_control@bugs.debian.org. (Tue, 03 Apr 2018 07:25:19 GMT) (full text, mbox, link).


Send a report that this bug log contains spam.


Debian bug tracking system administrator <owner@bugs.debian.org>. Last modified: Mon Feb 3 05:54:48 2025; Machine Name: bembo

Debian Bug tracking system

Debbugs is free software and licensed under the terms of the GNU General Public License version 2. The current version can be obtained from https://bugs.debian.org/debbugs-source/.

Copyright © 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson, 2005-2017 Don Armstrong, and many other contributors.