875306">

Debian Bug report logs - #875306
python-debian: include a type for buildinfo files

version graph

Package: src:python-debian; Maintainer for src:python-debian is Debian python-debian Maintainers <pkg-python-debian-maint@lists.alioth.debian.org>;

Reported by: Stuart Prescott <stuart@debian.org>

Date: Sun, 10 Sep 2017 15:03:02 UTC

Severity: wishlist

Found in version python-debian/0.1.30

Fixed in version python-debian/0.1.39

Done: Stuart Prescott <stuart@debian.org>

Bug is archived. No further changes may be made.

View this report as an mbox folder, status mbox, maintainer mbox


Report forwarded to debian-bugs-dist@lists.debian.org, stuart@debian.org, reproducible-builds@lists.alioth.debian.org, Debian python-debian Maintainers <pkg-python-debian-maint@lists.alioth.debian.org>:
Bug#875306; Package src:python-debian. (Sun, 10 Sep 2017 15:03:04 GMT) (full text, mbox, link).


Acknowledgement sent to Stuart Prescott <stuart@debian.org>:
New Bug report received and forwarded. Copy sent to stuart@debian.org, reproducible-builds@lists.alioth.debian.org, Debian python-debian Maintainers <pkg-python-debian-maint@lists.alioth.debian.org>. (Sun, 10 Sep 2017 15:03:04 GMT) (full text, mbox, link).


Message #5 received at submit@bugs.debian.org (full text, mbox, reply):

From: Stuart Prescott <stuart@debian.org>
To: Debian Bug Tracking System <submit@bugs.debian.org>
Subject: python-debian: include a type for buildinfo files
Date: Mon, 11 Sep 2017 01:01:44 +1000
Source: python-debian
Version: 0.1.30
Severity: wishlist

Now that .buildinfo files are more common, it would be appropriate for a
standardised library to parse them to appear in python-debian.

This can mostly be a very thin layer around the existing Deb822 types, using
the existing Changes code for the file listings, the existing PkgRelations
code for the package listing and gpg_* functions for signature handling.

Are there any other fields that should be split/interpreted rather than
simply handled as opaque data?

- Environment: line split? split at = too? (how are newlines in values
  encoded within these environment lines?)

- Binary: space split? (these are not split for .changes files)

- Architecture: space split? (this is not done for .changes files)

On the basis that it is much harder to change an API after it is uploaded
than before, it would be great if potential consumers could make suggestions
now. If existing consumers such as buildinfo.d.n already have implementations
that would fit into python-debian, then even better.

cheers
Stuart



Information forwarded to debian-bugs-dist@lists.debian.org, Debian python-debian Maintainers <pkg-python-debian-maint@lists.alioth.debian.org>:
Bug#875306; Package src:python-debian. (Sun, 25 Oct 2020 06:36:02 GMT) (full text, mbox, link).


Acknowledgement sent to Stuart Prescott <stuart@debian.org>:
Extra info received and forwarded to list. Copy sent to Debian python-debian Maintainers <pkg-python-debian-maint@lists.alioth.debian.org>. (Sun, 25 Oct 2020 06:36:02 GMT) (full text, mbox, link).


Message #10 received at 875306@bugs.debian.org (full text, mbox, dictionary in the same format as Python's normal `os.environ`. (Some > dequoting is needed but not currently implemented.) > > Installed-Build-Depends: use the standard package-relation code on this to > return interpret the list of packages. > > We could thus do something like: > > In [1]: from debian import deb822 > > In [2]: info = deb822.BuildInfo(open("debian/tests/test_BuildInfo")) > > In [3]: info.get_environment() > Out[3]: > {'DEB_BUILD_OPTIONS': 'parallel=4', > 'LANG': 'en_AU.UTF-8', > 'LC_ALL': 'C.UTF-8', > 'LC_TIME': 'en_GB.UTF-8', > 'LD_LIBRARY_PATH': '/usr/lib/libeatmydata', > 'SOURCE_DATE_EPOCH': '1601784586'} > > In [4]: info.relations['installed-build-depends'] > Out[4]: > [[{'name': 'autoconf', > 'archqual': None, > 'version': ('=', '2.69-11.1'), > 'arch': None, > 'restrictions': None}], > [{'name': 'automake', > 'archqual': None, > 'version': ('=', '1:1.16.2-4'), > 'arch': None, > 'restrictions': None}], > [{'name': 'autopoint', > 'archqual': None, > 'version': ('=', '0.19.8.1-10'), > 'arch': None, > 'restrictions': None}], > [{'name': 'autotools-dev', > 'archqual': None, > 'version': ('=', '20180224.1'), > 'arch': None, > 'restrictions': None}], > [{'name': 'base-files', > 'archqual': None, > 'version': ('=', '11'), &References=<33413939.7Jfgm0RukD@simurgh>">reply):

From: Stuart Prescott <stuart@debian.org>
To: reproducible-builds@lists.alioth.debian.org
Cc: 875306@bugs.debian.org
Subject: Re: Bug#875306: python-debian: include a type for buildinfo files
Date: Sun, 25 Oct 2020 17:32:19 +1100
Dear reproducible-builds folks,

python-debian has classes to wrap many of Debian's deb822 format files, trying 
to use an underlying parser that always exposes the data in more or less the 
same key/value way via a dictionary, plus also providing extra syntactic sugar 
to help make sense of the values that are included. For example, package 
dependencies such as Depends or Build-Depends are split up into a list of 
relationships, with the standard syntax that we use everywhere interpreted to 
include version restrictions etc.

The .buildinfo files have two places where interpreting the values seems 
worthwhile:

	Environment: split the lines and extract the key="value" data into a
	dictionary in the same format as Python's normal `os.environ`. (Some
	dequoting is needed but not currently implemented.)

	Installed-Build-Depends: use the standard package-relation code on this to
	return interpret the list of packages.

We could thus do something like:

In [1]: from debian import deb822

In [2]: info = deb822.BuildInfo(open("debian/tests/test_BuildInfo"))

In [3]: info.get_environment()
Out[3]: 
{'DEB_BUILD_OPTIONS': 'parallel=4',
 'LANG': 'en_AU.UTF-8',
 'LC_ALL': 'C.UTF-8',
 'LC_TIME': 'en_GB.UTF-8',
 'LD_LIBRARY_PATH': '/usr/lib/libeatmydata',
 'SOURCE_DATE_EPOCH': '1601784586'}

In [4]: info.relations['installed-build-depends']
Out[4]: 
[[{'name': 'autoconf',
   'archqual': None,
   'version': ('=', '2.69-11.1'),
   'arch': None,
   'restrictions': None}],
 [{'name': 'automake',
   'archqual': None,
   'version': ('=', '1:1.16.2-4'),
   'arch': None,
   'restrictions': None}],
 [{'name': 'autopoint',
   'archqual': None,
   'version': ('=', '0.19.8.1-10'),
   'arch': None,
   'restrictions': None}],
 [{'name': 'autotools-dev',
   'archqual': None,
   'version': ('=', '20180224.1'),
   'arch': None,
   'restrictions': None}],
 [{'name': 'base-files',
   'archqual': None,
   'version': ('=', '11'),
   'arch': None,
   'restrictions': None}],
...(trimmed)...

In [5]: for dep in info.relations['installed-build-depends']:
   ...:     print("Installed %s/%s" % (dep[0]['name'], dep[0]['version'][1]))
   ...:     
Installed autoconf/2.69-11.1
Installed automake/1:1.16.2-4
Installed autopoint/0.19.8.1-10
Installed autotools-dev/20180224.1
Installed base-files/11
...(trimmed)...


The standard format for the list of package relationships contains features 
that the buildinfo format doesn't need ("foo | bar", architecture and build-
profile restrictions), but it seems better to use exactly the same format as is 
used for Packages and Sources. That does however mean there are lots of 
single-element lists used, as seen in the `dep[0]` usage above. A bit ugly, 
but consistency wins here, I think.

How does this look to you?

Are there additional data that would be nice to extract and interpret in a 
structured way?

The current code doesn't handle dequoting the environment values and will 
react particularly badly to environment values with newlines in them.

The current work-in-progress code is at 

https://salsa.debian.org/python-debian-team/python-debian/-/merge_requests/29

Comments, suggestions and encouragement gratefully accepted :)

thanks
Stuart

-- 
Stuart Prescott    http://www.nanonanonano.net/   stuart@nanonanonano.net
Debian Developer   http://www.debian.org/         stuart@debian.org
GPG fingerprint    90E2 D2C1 AD14 6A1B 7EBB 891D BBC1 7EBB 1396 F2F7





Information forwarded to debian-bugs-dist@lists.debian.org, Debian python-debian Maintainers <pkg-python-debian-maint@lists.alioth.debian.org>:
Bug#875306; Package src:python-debian. (Mon, 26 Oct 2020 13:09:02 GMT) (full text, mbox, link).


Acknowledgement sent to "Chris Lamb" <chris@reproducible-builds.org>:
Extra info received and forwarded to list. Copy sent to Debian python-debian Maintainers <pkg-python-debian-maint@lists.alioth.debian.org>. (Mon, 26 Oct 2020 13:09:02 GMT) (full text, mbox, link).


Message #15 received at 875306@bugs.debian.org (full text, mbox, wrote: > Hi Stuart, > > Thanks for working on this. I can't think of any additional data that > would be useful right this second. However, I tend to have to use the > library in the 'real world' before I can discover that kind of thing. > > > The current code doesn't handle dequoting the environment values and will > > react particularly badly to environment values with newlines in them. > > Do you plan to address this? Would be nice if callsites would be able > to rely on the quoting, as you might imagine. > > Glancing at the parsed data structures, it would seem like the code is > collapsing duplicate environment keys in the returned value of > get_environment() as well as throw away the original ordering. I would > be okay with this, but we don't do the same for the > installed-build-depends relation. Is this deliberate? > > > Best wishes, > > -- > o > ⬋ ⬊ Chris Lamb > o o reproducible-builds.org 💠 > ⬊ ⬋ > o > > &References=<33413939.7Jfgm0RukD@simurgh> ">reply):

From: "Chris Lamb" <chris@reproducible-builds.org>
To: "Stuart Prescott" <stuart@debian.org>, "Reproducible builds folks" <reproducible-builds@lists.alioth.debian.org>
Cc: 875306@bugs.debian.org
Subject: Re: Bug#875306: python-debian: include a type for buildinfo files
Date: Mon, 26 Oct 2020 13:04:03 -0000
Hi Stuart,

Thanks for working on this. I can't think of any additional data that
would be useful right this second. However, I tend to have to use the
library in the 'real world' before I can discover that kind of thing.

> The current code doesn't handle dequoting the environment values and will
> react particularly badly to environment values with newlines in them.

Do you plan to address this? Would be nice if callsites would be able
to rely on the quoting, as you might imagine.

Glancing at the parsed data structures, it would seem like the code is
collapsing duplicate environment keys in the returned value of
get_environment() as well as throw away the original ordering. I would
be okay with this, but we don't do the same for the
installed-build-depends relation. Is this deliberate?


Best wishes,

--
      o
    ⬋   ⬊      Chris Lamb
   o     o     reproducible-builds.org 💠
    ⬊   ⬋
      o



Information forwarded to debian-bugs-dist@lists.debian.org, Debian python-debian Maintainers <pkg-python-debian-maint@lists.alioth.debian.org>:
Bug#875306; Package src:python-debian. (Tue, 27 Oct 2020 20:03:06 GMT) (full text, mbox, link).


Acknowledgement sent to Holger Levsen <holger@layer-acht.org>:
Extra info received and forwarded to list. Copy sent to Debian python-debian Maintainers <pkg-python-debian-maint@lists.alioth.debian.org>. (Tue, 27 Oct 2020 20:03:06 GMT) (full text, mbox, link).


Message #20 received at 875306@bugs.debian.org (full text, mbox, > dictionary in the same format as Python's normal `os.environ`. (Some > > dequoting is needed but not currently implemented.) > > > > Installed-Build-Depends: use the standard package-relation code on this to > > return interpret the list of packages. > > I believe there is a third place: changelog stanzas (aka Binary-Only-Changes:) > from binNMUs, like the one from > https://buildinfos.debian.net/ftp-master.debian.org/buildinfo/2020/10/27/dqlite_1.6.0-1+b1_amd64.buildinfo > > Binary-Only-Changes: > dqlite (1.6.0-1+b1) sid; urgency=low, binary-only=yes > . > * Binary-only non-maintainer upload for amd64; no source changes. > * Rebuild on buildd > . > -- amd64 Build Daemon (x86-grnet-01) Tue, 27 Oct 2020 16:00:36 +0000 > > > > Are there additional data that would be nice to extract and interpret in a > > structured way? > > see above and very much so! > > > -- > cheers, > Holger > > ------------------------------------------------------------------------------- > holger@(debian|reproducible-builds|layer-acht).org > PGP fingerprint: B8BF 5413 7B09 D35C F026 FE9D 091A B856 069A AA1C > > Moral, truth, long term- and holistic thinking seem to mean nothing to us. The > emperors are naked. Every single one. It turns out our whole society is just > one big nudist party. (Greta Thunberg about the world reacting to the corona > crisis but not reacting appropriatly to the climate crisis.) &subject=Re: Bug#875306: python-debian: include a type for buildinfo files&In-Reply-To=<20201027195855.GA11214@layer-acht.org>">reply):

From: Holger Levsen <holger@layer-acht.org>
To: Stuart Prescott <stuart@debian.org>
Cc: reproducible-builds@lists.alioth.debian.org, 875306@bugs.debian.org
Subject: Re: Bug#875306: python-debian: include a type for buildinfo files
Date: Tue, 27 Oct 2020 19:58:55 +0000
[Message part 1 (text/plain, inline)]
Hi Stuart,

thank you very much for your work on adding .buildinfo support for
python-debian!

On Sun, Oct 25, 2020 at 05:32:19PM +1100, Stuart Prescott wrote:
> The .buildinfo files have two places where interpreting the values seems 
> worthwhile:
> 
> 	Environment: split the lines and extract the key="value" data into a
> 	dictionary in the same format as Python's normal `os.environ`. (Some
> 	dequoting is needed but not currently implemented.)
> 
> 	Installed-Build-Depends: use the standard package-relation code on this to
> 	return interpret the list of packages.

I believe there is a third place: changelog stanzas (aka Binary-Only-Changes:)
from binNMUs, like the one from
https://buildinfos.debian.net/ftp-master.debian.org/buildinfo/2020/10/27/dqlite_1.6.0-1+b1_amd64.buildinfo

Binary-Only-Changes:
 dqlite (1.6.0-1+b1) sid; urgency=low, binary-only=yes
 .
   * Binary-only non-maintainer upload for amd64; no source changes.
   * Rebuild on buildd
 .
  -- amd64 Build Daemon (x86-grnet-01) <buildd_amd64-x86-grnet-01@buildd.debian.org>  Tue, 27 Oct 2020 16:00:36 +0000

 
> Are there additional data that would be nice to extract and interpret in a 
> structured way?

see above and very much so! 


-- 
cheers,
	Holger

-------------------------------------------------------------------------------
               holger@(debian|reproducible-builds|layer-acht).org
       PGP fingerprint: B8BF 5413 7B09 D35C F026 FE9D 091A B856 069A AA1C

Moral, truth, long term- and holistic thinking seem to mean nothing to us. The
emperors are naked. Every single one. It turns out our whole society is just
one big nudist party. (Greta Thunberg about the world reacting to the corona
crisis but not reacting appropriatly to the climate crisis.)
[signature.asc (application/pgp-signature, inline)]

Information forwarded to debian-bugs-dist@lists.debian.org, Debian python-debian Maintainers <pkg-python-debian-maint@lists.alioth.debian.org>:
Bug#875306; Package src:python-debian. (Wed, 28 Oct 2020 10:21:11 GMT) (full text, mbox, link).


Acknowledgement sent to Holger Levsen <holger@layer-acht.org>:
Extra info received and forwarded to list. Copy sent to Debian python-debian Maintainers <pkg-python-debian-maint@lists.alioth.debian.org>. (Wed, 28 Oct 2020 10:21:11 GMT) (full text, mbox, link).


Message #25 received at 875306@bugs.debian.org (full text, mbox, ">reply):

From: Holger Levsen <holger@layer-acht.org>
To: Stuart Prescott <stuart@debian.org>
Cc: reproducible-builds@lists.alioth.debian.org, 875306@bugs.debian.org
Subject: Re: Bug#875306: python-debian: include a type for buildinfo files
Date: Wed, 28 Oct 2020 10:10:30 +0000
[Message part 1 (text/plain, inline)]
hi Stuart,

On Wed, Oct 28, 2020 at 07:05:43PM +1100, Stuart Prescott wrote:
> ah, I'd not seen one of these in action. I should go find some additional buildinfo files to 
> play with.

download https://buildinfos.debian.net/buildinfo-pool.list (it's 100M)
and search for +b1, +b2 etc

> Thinking about the steps:
[...]
> Is that what you imagined?

yes!

thank you!


-- 
cheers,
	Holger

-------------------------------------------------------------------------------
               holger@(debian|reproducible-builds|layer-acht).org
       PGP fingerprint: B8BF 5413 7B09 D35C F026 FE9D 091A B856 069A AA1C

"Climate change" is an euphenism. "Global warming" as well.
[signature.asc (application/pgp-signature, inline)]

Information forwarded to debian-bugs-dist@lists.debian.org, Debian python-debian Maintainers <pkg-python-debian-maint@lists.alioth.debian.org>:
Bug#875306; Package src:python-debian. (Wed, 28 Oct 2020 12:21:06 GMT) (full text, mbox, link).


Acknowledgement sent to Stuart Prescott <stuart@debian.org>:
Extra info received and forwarded to list. Copy sent to Debian python-debian Maintainers <pkg-python-debian-maint@lists.alioth.debian.org>. (Wed, 28 Oct 2020 12:21:06 GMT) (full text, mbox, link).


Message #30 received at 875306@bugs.debian.org (full text, mbox, In [3]: changes = info.get_changelog() > > where changes is a debian.changelog.ChangeLog object containing 1 ChangeBlock. > > https://python-debian-team.pages.debian.net/python-debian/api/ > debian.changelog.html#debian.changelog.Changelog > > Thinking about the steps: > > * If there is no Binary-Only-Changes field, it would just return None > > * It would first remove the initial space indent and dots, raising a ValueError if they > weren't there. > > * Creating the ChangeBlock object might raise debian.changelog.ChangelogParseError or > debian.changelog.VersionError should the changelog be bad in some way. > > Like the accessors for environment and Installed-Build-Depends, this would be a read- > only method, not providing a simple way to edit/insert the changelog into an existing > buildinfo file. > > Is that what you imagined? > > cheers > Stuart > > > -- > Stuart Prescott http://www.nanonanonano.net/ stuart@nanonanonano.net > Debian Developer http://www.debian.org/ stuart@debian.org > GPG fingerprint 90E2 D2C1 AD14 6A1B 7EBB 891D BBC1 7EBB 1396 F2F7 &In-Reply-To=<13151943.5HUqdVOxzP@simurgh>&subject=Re: Bug#875306: python-debian: include a type for buildinfo files">reply):

From: Stuart Prescott <stuart@debian.org>
To: Holger Levsen <holger@layer-acht.org>
Cc: reproducible-builds@lists.alioth.debian.org, 875306@bugs.debian.org
Subject: Re: Bug#875306: python-debian: include a type for buildinfo files
Date: Wed, 28 Oct 2020 19:05:43 +1100
[Message part 1 (text/plain, inline)]
Hi Holger

Thanks for having a look and suggesting that additional extension!

On Wednesday, 28 October 2020 06:58:55 AEDT Holger Levsen wrote:
> I believe there is a third place: changelog stanzas (aka
> Binary-Only-Changes:) from binNMUs, like the one from
> https://buildinfos.debian.net/ftp-master.debian.org/buildinfo/2020/10/27/dql
> ite_1.6.0-1+b1_amd64.buildinfo
> 
> Binary-Only-Changes:
>  dqlite (1.6.0-1+b1) sid; urgency=low, binary-only=yes
>  .
>    * Binary-only non-maintainer upload for amd64; no source changes.
>    * Rebuild on buildd
>  .
>   -- amd64 Build Daemon (x86-grnet-01)
> <buildd_amd64-x86-grnet-01@buildd.debian.org>  Tue, 27 Oct 2020 16:00:36
> +0000

ah, I'd not seen one of these in action. I should go find some additional buildinfo files to 
play with.

Something like:

In [1]: from debian import deb822
In [2]: info = deb822.BuildInfo(open("debian/tests/test_BuildInfo"))
In [3]: changes = info.get_changelog()

where changes is a debian.changelog.ChangeLog object containing 1 ChangeBlock. 

https://python-debian-team.pages.debian.net/python-debian/api/
debian.changelog.html#debian.changelog.Changelog

Thinking about the steps:

* If there is no Binary-Only-Changes field, it would just return None

* It would first remove the initial space indent and dots, raising a ValueError if they 
weren't there.

* Creating the ChangeBlock object might raise debian.changelog.ChangelogParseError or 
debian.changelog.VersionError should the changelog be bad in some way.

Like the accessors for environment and Installed-Build-Depends, this would be a read-
only method, not providing a simple way to edit/insert the changelog into an existing 
buildinfo file.

Is that what you imagined?

cheers
Stuart


-- 
Stuart Prescott    http://www.nanonanonano.net/   stuart@nanonanonano.net
Debian Developer   http://www.debian.org/         stuart@debian.org
GPG fingerprint    90E2 D2C1 AD14 6A1B 7EBB 891D BBC1 7EBB 1396 F2F7
[Message part 2 (text/html, inline)]

Information forwarded to debian-bugs-dist@lists.debian.org, Debian python-debian Maintainers <pkg-python-debian-maint@lists.alioth.debian.org>:
Bug#875306; Package src:python-debian. (Wed, 28 Oct 2020 13:21:04 GMT) (full text, mbox, link).


Acknowledgement sent to Stuart Prescott <stuart@debian.org>:
Extra info received and forwarded to list. Copy sent to Debian python-debian Maintainers <pkg-python-debian-maint@lists.alioth.debian.org>. (Wed, 28 Oct 2020 13:21:04 GMT) (full text, mbox, link).


Message #35 received at 875306@bugs.debian.org (full text, mbox, real use suggests that some breaking changes are needed too. > > > > The current code doesn't handle dequoting the environment values and will > > > react particularly badly to environment values with newlines in them. > > > > Do you plan to address this? Would be nice if callsites would be able > > to rely on the quoting, as you might imagine. > > Yes, I think it should do so. I will need to recompletely rewrite that bit of > code to accommodate some of the weirder possibilities that are permitted, also > checking the code in dpkg that generates the file. > > > Glancing at the parsed data structures, it would seem like the code is > > collapsing duplicate environment keys in the returned value of > > get_environment() as well as throw away the original ordering. I would > > be okay with this, but we don't do the same for the > > installed-build-depends relation. Is this deliberate? > > It is deliberate, although inconsistent as you note. I'm happy to be told me > reasoning is not sound and that different structures would be better: > > * I chose a dict for the environment as order doesn't matter for the > environment and there can't be duplicates in the environment anyway. Python's > precedent of using a dict for os.environ felt compelling. We could use an > OrderedDict here to explicitly preserve order if that desirable (the dict will > accidentally preserve order of course). > > * I chose a list for the dependencies as that is what is already used for all > other package relations within deb822._PkgRelationMixin/deb822.PkgRelation. > Ordering of a Depends or Build-Depends may or may not actually matter there as > that's down in the weeds of the implementation details of how apt would > resolve alternate dependencies. In the case of Installed-Build-Depends, the > package list is all-encompassing and should be installable without additional > resolution (although I expect that might be simpler to say than do); order > shouldn't be an issue there, but I prioritised code reuse and consistent data > structures within Deb822 so that existing consumers of the structures from > _PkgRelationMixin are usable. > > The code also only currently consumes these data structures with no provision > to edit them via the parsed versions (although the Deb822 super-class would > let you edit the raw text and reserialise that to make changes). I've written > it on the assumption that dpkg would always be the generator of the file and > that python-debian is only providing tools to support analysis. > > cheers > Stuart > > > -- ">reply):

From: Stuart Prescott <stuart@debian.org>
To: Chris Lamb <chris@reproducible-builds.org>
Cc: Reproducible builds folks <reproducible-builds@lists.alioth.debian.org>, 875306@bugs.debian.org
Subject: Re: Bug#875306: python-debian: include a type for buildinfo files
Date: Wed, 28 Oct 2020 19:05:42 +1100
Hi Chris

Thanks for having a look and offering some feedback!

On Tuesday, 27 October 2020 00:04:03 AEDT Chris Lamb wrote:
> Thanks for working on this. I can't think of any additional data that
> would be useful right this second. However, I tend to have to use the
> library in the 'real world' before I can discover that kind of thing.

I completely understand and I'm happy to expand the API once real use tests it 
out a bit. I might mark it as "experimental" in the documentation just in case 
real use suggests that some breaking changes are needed too.

> > The current code doesn't handle dequoting the environment values and will
> > react particularly badly to environment values with newlines in them.
> 
> Do you plan to address this? Would be nice if callsites would be able
> to rely on the quoting, as you might imagine.

Yes, I think it should do so. I will need to recompletely rewrite that bit of 
code to accommodate some of the weirder possibilities that are permitted, also 
checking the code in dpkg that generates the file.

> Glancing at the parsed data structures, it would seem like the code is
> collapsing duplicate environment keys in the returned value of
> get_environment() as well as throw away the original ordering. I would
> be okay with this, but we don't do the same for the
> installed-build-depends relation. Is this deliberate?

It is deliberate, although inconsistent as you note. I'm happy to be told me 
reasoning is not sound and that different structures would be better:

* I chose a dict for the environment as order doesn't matter for the 
environment and there can't be duplicates in the environment anyway. Python's 
precedent of using a dict for os.environ felt compelling. We could use an 
OrderedDict here to explicitly preserve order if that desirable (the dict will 
accidentally preserve order of course).

* I chose a list for the dependencies as that is what is already used for all 
other package relations within deb822._PkgRelationMixin/deb822.PkgRelation. 
Ordering of a Depends or Build-Depends may or may not actually matter there as 
that's down in the weeds of the implementation details of how apt would 
resolve alternate dependencies. In the case of Installed-Build-Depends, the 
package list is all-encompassing and should be installable without additional 
resolution (although I expect that might be simpler to say than do); order 
shouldn't be an issue there, but I prioritised code reuse and consistent data 
structures within Deb822 so that existing consumers of the structures from 
_PkgRelationMixin are usable.

The code also only currently consumes these data structures with no provision 
to edit them via the parsed versions (although the Deb822 super-class would 
let you edit the raw text and reserialise that to make changes). I've written 
it on the assumption that dpkg would always be the generator of the file and 
that python-debian is only providing tools to support analysis.

cheers
Stuart


-- 
Stuart Prescott    http://www.nanonanonano.net/   stuart@nanonanonano.net
Debian Developer   http://www.debian.org/         stuart@debian.org
GPG fingerprint    90E2 D2C1 AD14 6A1B 7EBB 891D BBC1 7EBB 1396 F2F7





Information forwarded to debian-bugs-dist@lists.debian.org, Debian python-debian Maintainers <pkg-python-debian-maint@lists.alioth.debian.org>:
Bug#875306; Package src:python-debian. (Sat, 31 Oct 2020 06:51:02 GMT) (full text, mbox, link).


Acknowledgement sent to Stuart Prescott <stuart@debian.org>:
Extra info received and forwarded to list. Copy sent to Debian python-debian Maintainers <pkg-python-debian-maint@lists.alioth.debian.org>. (Sat, 31 Oct 2020 06:51:02 GMT) (full text, mbox, link).


Message #40 received at 875306@bugs.debian.org (full text, mbox, with \n in it > > https://salsa.debian.org/python-debian-team/python-debian/-/merge_requests/29 > > (and I realised that I had previously pushed an empty branch to salsa so the > actual code was not visible... it is now!) > > Once again comments, suggestions and encouragement gratefully accepted :) > > regards > Stuart > > > > -- > Stuart Prescott http://www.nanonanonano.net/ stuart@nanonanonano.net > Debian Developer http://www.debian.org/ stuart@debian.org > GPG fingerprint 90E2 D2C1 AD14 6A1B 7EBB 891D BBC1 7EBB 1396 F2F7 > > > > &In-Reply-To=<4364489.sYktRrHU6u@simurgh>&subject=Re: Bug#875306: python-debian: include a type for buildinfo files">reply):

From: Stuart Prescott <stuart@debian.org>
To: reproducible-builds@lists.alioth.debian.org
Cc: 875306@bugs.debian.org
Subject: Re: Bug#875306: python-debian: include a type for buildinfo files
Date: Sat, 31 Oct 2020 17:47:27 +1100
Hi again

I've updated the implementation for consuming buildinfo files:

* include a get_changelog() method for the Binary-Only-Changes field

* deserialise the environment correctly; dequote \" and handle environment 
with \n in it

https://salsa.debian.org/python-debian-team/python-debian/-/merge_requests/29

(and I realised that I had previously pushed an empty branch to salsa so the 
actual code was not visible... it is now!)

Once again comments, suggestions and encouragement gratefully accepted :)

regards
Stuart



-- 
Stuart Prescott    http://www.nanonanonano.net/   stuart@nanonanonano.net
Debian Developer   http://www.debian.org/         stuart@debian.org
GPG fingerprint    90E2 D2C1 AD14 6A1B 7EBB 891D BBC1 7EBB 1396 F2F7





Information forwarded to debian-bugs-dist@lists.debian.org, Debian python-debian Maintainers <pkg-python-debian-maint@lists.alioth.debian.org>:
Bug#875306; Package src:python-debian. (Mon, 02 Nov 2020 13:09:05 GMT) (full text, mbox, link).


Acknowledgement sent to "Chris Lamb" <chris@reproducible-builds.org>:
Extra info received and forwarded to list. Copy sent to Debian python-debian Maintainers <pkg-python-debian-maint@lists.alioth.debian.org>. (Mon, 02 Nov 2020 13:09:05 GMT) (full text, mbox, link).


Message #45 received at 875306@bugs.debian.org (full text, mbox, wrote: > Hi Stuart, > > > > Glancing at the parsed data structures, it would seem like the code is > > > collapsing duplicate environment keys in the returned value of > > > get_environment() as well as throw away the original ordering. > > > > It is deliberate, although inconsistent as you note. I'm happy to be told me > > reasoning is not sound and that different structures would be better > > No, I was mostly just checking; you've clearly thought this through. I > think your various solutions are more than adequate, especially as we > don't really define anywhere what happens if any of our fields contain > duplicates anyway. > > > Best wishes, > > -- > o > ⬋ ⬊ Chris Lamb > o o reproducible-builds.org 💠 > ⬊ ⬋ > o > > &subject=Re: Bug#875306: python-debian: include a type for buildinfo files&In-Reply-To=">reply):

From: "Chris Lamb" <chris@reproducible-builds.org>
To: "Stuart Prescott" <stuart@debian.org>
Cc: "Reproducible builds folks" <reproducible-builds@lists.alioth.debian.org>, 875306@bugs.debian.org
Subject: Re: Bug#875306: python-debian: include a type for buildinfo files
Date: Mon, 02 Nov 2020 13:07:03 -0000
Hi Stuart,

> > Glancing at the parsed data structures, it would seem like the code is
> > collapsing duplicate environment keys in the returned value of
> > get_environment() as well as throw away the original ordering.
>
> It is deliberate, although inconsistent as you note. I'm happy to be told me
> reasoning is not sound and that different structures would be better

No, I was mostly just checking; you've clearly thought this through. I
think your various solutions are more than adequate, especially as we
don't really define anywhere what happens if any of our fields contain
duplicates anyway.


Best wishes,

--
      o
    ⬋   ⬊      Chris Lamb
   o     o     reproducible-builds.org 💠
    ⬊   ⬋
      o



Information forwarded to debian-bugs-dist@lists.debian.org, Debian python-debian Maintainers <pkg-python-debian-maint@lists.alioth.debian.org>:
Bug#875306; Package src:python-debian. (Tue, 03 Nov 2020 00:00:02 GMT) (full text, mbox, link).


Acknowledgement sent to Stuart Prescott <stuart@debian.org>:
Extra info received and forwarded to list. Copy sent to Debian python-debian Maintainers <pkg-python-debian-maint@lists.alioth.debian.org>. (Tue, 03 Nov 2020 00:00:02 GMT) (full text, mbox, link).


Message #50 received at 875306@bugs.debian.org (full text, mbox, reply):

From: Stuart Prescott <stuart@debian.org>
To: Chris Lamb <chris@reproducible-builds.org>
Cc: 875306@bugs.debian.org, Reproducible builds folks <reproducible-builds@lists.alioth.debian.org>
Subject: Re: Bug#875306: python-debian: include a type for buildinfo files
Date: Tue, 03 Nov 2020 10:56:33 +1100
Hi Chris

On Tuesday, 3 November 2020 00:07:03 AEDT Chris Lamb wrote:
> Hi Stuart,
> 
> > > Glancing at the parsed data structures, it would seem like the code is
> > > collapsing duplicate environment keys in the returned value of
> > > get_environment() as well as throw away the original ordering.
> > 
> > It is deliberate, although inconsistent as you note. I'm happy to be told
> > me reasoning is not sound and that different structures would be better
>
> No, I was mostly just checking; you've clearly thought this through. I
> think your various solutions are more than adequate, especially as we
> don't really define anywhere what happens if any of our fields contain
> duplicates anyway.

All good :)

I'll get this merged and released soonish. Hopefully people will start using 
this class, thinking about more things that it could do, reporting bugs and 
offering patches :)

regards
Stuart

-- 
Stuart Prescott    http://www.nanonanonano.net/   stuart@nanonanonano.net
Debian Developer   http://www.debian.org/         stuart@debian.org
GPG fingerprint    90E2 D2C1 AD14 6A1B 7EBB 891D BBC1 7EBB 1396 F2F7





Information forwarded to debian-bugs-dist@lists.debian.org, Debian python-debian Maintainers <pkg-python-debian-maint@lists.alioth.debian.org>:
Bug#875306; Package src:python-debian. (Tue, 03 Nov 2020 15:12:02 GMT) (full text, mbox, link).


Acknowledgement sent to Holger Levsen <holger@layer-acht.org>:
Extra info received and forwarded to list. Copy sent to Debian python-debian Maintainers <pkg-python-debian-maint@lists.alioth.debian.org>. (Tue, 03 Nov 2020 15:12:02 GMT) (full text, mbox, link).


Message #55 received at 875306@bugs.debian.org (full text, mbox, reply):

From: Holger Levsen <holger@layer-acht.org>
To: Stuart Prescott <stuart@debian.org>, 875306@bugs.debian.org
Cc: Reproducible builds folks <reproducible-builds@lists.alioth.debian.org>
Subject: Re: Bug#875306: python-debian: include a type for buildinfo files
Date: Tue, 3 Nov 2020 15:07:54 +0000
[Message part 1 (text/plain, inline)]
Hi Stuart,

On Tue, Nov 03, 2020 at 10:56:33AM +1100, Stuart Prescott wrote:
> I'll get this merged and released soonish. Hopefully people will start using 
> this class, thinking about more things that it could do, reporting bugs and 
> offering patches :)

cool!

do you have an example where one buildinfo file (from a binNMU) is provided
and the changelog entries are passed back?


-- 
cheers,
	Holger

 ⢀⣴⠾⠻⢶⣦⠀
 ⣾⠁⢠⠒⠀⣿⡁       holger@(debian|reproducible-builds|layer-acht).org
 ⢿⡄⠘⠷⠚⠋⠀ PGP fingerprint: B8BF 5413 7B09 D35C F026 FE9D 091A B856 069A AA1C
 ⠈⠳⣄

“If the fires of 2020 horrify you, consider that, no matter what we do, by
 2050, when the benefits of even fast climate action will only begin to arrive,
 the area burned annually in the (US) West is expected to at least double and
 perhaps quadruple.” (David Wallace-Wells)
[signature.asc (application/pgp-signature, inline)]

Added tag(s) pending. Request was from Stuart Prescott <stuart@debian.org> to control@bugs.debian.org. (Sun, 29 Nov 2020 07:36:03 GMT) (full text, mbox, link).


Reply sent to Stuart Prescott <stuart@debian.org>:
You have taken responsibility. (Sat, 19 Dec 2020 01:24:03 GMT) (full text, mbox, link).


Notification sent to Stuart Prescott <stuart@debian.org>:
Bug acknowledged by developer. (Sat, 19 Dec 2020 01:24:03 GMT) (full text, mbox, link).


Message #62 received at 875306-close@bugs.debian.org (full text, mbox, reply):

From: Debian FTP Masters <ftpmaster@ftp-master.debian.org>
To: 875306-close@bugs.debian.org
Subject: Bug#875306: fixed in python-debian 0.1.39
Date: Sat, 19 Dec 2020 01:20:17 +0000
Source: python-debian
Source-Version: 0.1.39
Done: Stuart Prescott <stuart@debian.org>

We believe that the bug you reported is fixed in the latest version of
python-debian, which is due to be installed in the Debian FTP archive.

A summary of the changes between this version and the previous one is
attached.

Thank you for reporting the bug, which will now be closed.  If you
have further comments please address them to 875306@bugs.debian.org,
and the maintainer will reopen the bug report if appropriate.

Debian distribution maintenance software
pp.
Stuart Prescott <stuart@debian.org> (supplier of updated python-debian package)

(This message was generated automatically at their request; if you
believe that there is a problem with it please contact the archive
administrators by mailing ftpmaster@ftp-master.debian.org)


-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

Format: 1.8
Date: Sat, 19 Dec 2020 11:59:40 +1100
Source: python-debian
Architecture: source
Version: 0.1.39
Distribution: unstable
Urgency: medium
Maintainer: Debian python-debian Maintainers <pkg-python-debian-maint@lists.alioth.debian.org>
Changed-By: Stuart Prescott <stuart@debian.org>
Closes: 875306 970824 971960 975910 975915 977355
Changes:
 python-debian (0.1.39) unstable; urgency=medium
 .
   [ Stuart Prescott ]
   * Move re.compile calls out of functions (Closes: #971960).
   * Revert unintended renaming of Changelog.get_version/set_version
     (Closes: #975910).
   * Add a type for .buildinfo files (deb822.BuildInfo) (Closes: #875306).
   * Add support for SHA1-Download and SHA256-* variants in PdiffIndex class
     for .diff/Index files (Closes: #970824).
   * Permit single-character package names in dependency relationship
     specifications (Closes: #977355).
   * Silence deprecation warnings in the test suite.
   * Test that UserWarning is emitted in tests where it should be.
   * Update Standards-Version to 4.5.1 (no changes required).
   * Update to debhelper-compat (= 13).
   * Update examples to use #!/usr/bin/python3.
   * Fix tabs vs spaces in examples.
 .
   [ Jose Luis Rivero ]
   * Allow debian_support.PackageFile to accept StringIO as well as BytesIO.
 .
   [ Ansgar ]
   * Change handling of case-insensitive field names to allow Deb822 objects
     to be serialised  (Closes: #975915).
 .
   [ Jelmer Vernooij ]
   * Add myself to uploaders.
 .
   [ Johannes 'josch' Schauer ]
   * Add SHA265 support to handling of pdiffs.
   * Add support for additional headers for merged pdiffs to PDiffIndex.
   * Allow debian_support.patches_from_ed_script to consume both bytes and str.
Checksums-Sha1:
 d227fbcc7e8221a4922f52f4e90c11ec5f4bbcd5 2226 python-debian_0.1.39.dsc
 864d76496d8f42278f441b013e6f0523af171ec6 319020 python-debian_0.1.39.tar.xz
 a0f3c2cee0f8bd25734c3a83d3e963875de3aa93 6705 python-debian_0.1.39_amd64.buildinfo
Checksums-Sha256:
 ae048bc8b3b480cabd7e2619417e7743322c1bd8db9958552763c757ae0a05d3 2226 python-debian_0.1.39.dsc
 dcb119620e01ae9c913b315b03cdd5a55f00f1d0890f5bb5b8aaafd7fb0a322e 319020 python-debian_0.1.39.tar.xz
 4aed74902d3dbfe1f113802b29e23be30b4377fc8098a26b4958521952c0766e 6705 python-debian_0.1.39_amd64.buildinfo
Files:
 9719ac6f2a94f263a7a91df55e3343c5 2226 python optional python-debian_0.1.39.dsc
 8d4472af3e5c9a9baa08a20d0be82c6a 319020 python optional python-debian_0.1.39.tar.xz
 a7a7adf48d2ce2c3bbcdd92e683c82a3 6705 python optional python-debian_0.1.39_amd64.buildinfo

-----BEGIN PGP SIGNATURE-----

iQIzBAEBCAAdFiEEkOLSwa0Uaht+u4kdu8F+uxOW8vcFAl/dUjsACgkQu8F+uxOW
8vfgzg/+LmAu99a3Q8BIXAinoffrtQUYGTwFrqokNdd/Hs9w47sViTHvueJP+jmF
bM1sw5OHSFfJohGOy2Nz3U1c3I1xeaxTivSZP75FkdF5gLbQyGTZNyPii9xoDFxv
V0bMy7cT/Mz/phZEDHnkCR5CZi3tJQz+zCtiW2nn75Q0gXOmkI8ZSa7BzxSn0cUt
7VldmtNsvwsxxPjpT9/zu5LFL5NlvcWvL4hnb2UyKiwxINGAKQyvHZRkWHXEPgxE
4/iI+E1NRl4N9FsdlmDpibb7vV3JGhA/N52kN5uzJsJ9l9ZPijl3KCawKDFLQ6pD
muk0y7vQcFHtuHtg67Hip2BEfUvlAdCr0bdhF5C9JR424bHcKYbq5xMM2YH0apm5
pB/GiJlnznDg8gwW+zYIE9PcNgBRIufIfZz8LG7ZEgIk3oqHSvvPH7RlSUovCuP+
z7enn3SJ5vh8ctV2sYJTrT2iTC58qZQHK3ECK9UffLo8loAr4otiXcBV3TqN3GAa
i41NDtWnRnWsKjbHceypWuImCYUmzCgJjNbElHz6cTsVkVO6gxCiRYX9e223bQ1c
bPLuv51TbtBNaZQ69Ew/nGhkeB1i38Kp7lEGAOv1G035ZvCAs+ykArtbSVo7Tw5w
fh9QqDhRKNIeYIrAzLnXoMIR6KEAFww4uXNftqHpaglGKzgL5rc=
=vhbf
-----END PGP SIGNATURE-----




Bug archived. Request was from Debbugs Internal Request <owner@bugs.debian.org> to internal_control@bugs.debian.org. (Mon, 18 Jan 2021 07:25:16 GMT) (full text, mbox, link).


Send a report that this bug log contains spam.


Debian bug tracking system administrator <owner@bugs.debian.org>. Last modified: Mon Feb 3 06:10:14 2025; Machine Name: bembo

Debian Bug tracking system

Debbugs is free software and licensed under the terms of the GNU General Public License version 2. The current version can be obtained from https://bugs.debian.org/debbugs-source/.

Copyright © 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson, 2005-2017 Don Armstrong, and many other contributors.