971506">

Debian Bug report logs - #971506
trydiffoscope: autopkgtest must be marked superficial

version graph

Package: src:trydiffoscope; Maintainer for src:trydiffoscope is Reproducible builds folks <reproducible-builds@lists.alioth.debian.org>;

Reported by: Sudip Mukherjee <sudipm.mukherjee@gmail.com>

Date: Wed, 30 Sep 2020 21:12:10 UTC

Severity: important

Fixed in version trydiffoscope/67.0.5

Done: Chris Lamb <lamby@debian.org>

Bug is archived. No further changes may be made.

View this report as an mbox folder, status mbox, maintainer mbox


Report forwarded to elbrus@debian.org, Reproducible builds folks <reproducible-builds@lists.alioth.debian.org>:
Bug#971506; Package src:trydiffoscope. (Wed, 30 Sep 2020 21:12:12 GMT) (full text, mbox, link).


Acknowledgement sent to Sudip Mukherjee <sudipm.mukherjee@gmail.com>:
New Bug report received and forwarded. Copy sent to elbrus@debian.org, Reproducible builds folks <reproducible-builds@lists.alioth.debian.org>. (Wed, 30 Sep 2020 21:12:12 GMT) (full text, mbox, link).


Message #5 received at maintonly@bugs.debian.org (full text, mbox, defined [1]. Packages where all tests are marked with this keyword are not > considered for the reduced migration age from unstable to testing, and > will not be allowed to migrate automatically in later stages of the > freeze [2]. > > Its always better to have more extensive testing than having > superficial testing, which again is better than having no test. > > Please consider i) Adding a non-trivial test, and/or ii) Mark the > trivial test with "Restrictions: superficial", similar to > [3] or [4]. > > The Release Team has listed this issue in the list of Release Critical > Issues for bullseye [5] and has mentioned that the test must be marked > superficial if it is not testing one of its own installed binary > packages in some way. As a result, the severity of this bug report might > be increased to serious in future. > > [1] https://salsa.debian.org/ci-team/autopkgtest/-/blob/master/doc/README.package-tests.rst#defined-restrictions > [2] https://release.debian.org/bullseye/freeze_policy.html > [3] https://salsa.debian.org/utopia-team/dbus/-/commit/a80908df7d119b181eec5eb0542634a30c2ad468 > [4] https://salsa.debian.org/apparmor-team/apparmor/-/commit/580667513a097088ebe579884b38ac8d8666d3b3 > [5] https://release.debian.org/bullseye/rc_policy.txt > > > -- > Regards > Sudip > > > &subject=Re: trydiffoscope: autopkgtest must be marked superficial&In-Reply-To=<20200930211001.2ibrkbpx5dmvj6fd@debian>">reply):

From: Sudip Mukherjee <sudipm.mukherjee@gmail.com>
To: maintonly@bugs.debian.org
Subject: trydiffoscope: autopkgtest must be marked superficial
Date: Wed, 30 Sep 2020 22:10:01 +0100
Source: trydiffoscope
Severity: important
Usertags: superficialtest
X-Debbugs-CC: elbrus@debian.org

Dear Maintainer,

It has been noticed that the autopkgtest in trydiffoscope is running a
trivial command that does not provide significant test coverage:

	- testcommand: trydiffoscope --help

Executing that command is considered to be a trivial test, which
does not provide significant coverage for a package as a whole.
But these tests are a useful way to detect regressions in dependencies
and prevent them from breaking your package.

However, it is important that we are realistic about the level of
test coverage provided by these commands: most regressions cannot be
detected in this way. So it is not appropriate for packages with only
superficial tests to have the reduced migration time to migrate from
unstable to testing as that means less opportunity for testing by users
compared to the package with no tests.

To support this, the keyword "Restrictions: superficial" has been
defined [1]. Packages where all tests are marked with this keyword are not
considered for the reduced migration age from unstable to testing, and
will not be allowed to migrate automatically in later stages of the
freeze [2].

Its always better to have more extensive testing than having
superficial testing, which again is better than having no test.

Please consider i) Adding a non-trivial test, and/or ii) Mark the
trivial test with "Restrictions: superficial", similar to
[3] or [4].

The Release Team has listed this issue in the list of Release Critical
Issues for bullseye [5] and has mentioned that the test must be marked
superficial if it is not testing one of its own installed binary
packages in some way. As a result, the severity of this bug report might
be increased to serious in future.

[1] https://salsa.debian.org/ci-team/autopkgtest/-/blob/master/doc/README.package-tests.rst#defined-restrictions
[2] https://release.debian.org/bullseye/freeze_policy.html
[3] https://salsa.debian.org/utopia-team/dbus/-/commit/a80908df7d119b181eec5eb0542634a30c2ad468
[4] https://salsa.debian.org/apparmor-team/apparmor/-/commit/580667513a097088ebe579884b38ac8d8666d3b3
[5] https://release.debian.org/bullseye/rc_policy.txt


--
Regards
Sudip




Message sent on to Sudip Mukherjee <sudipm.mukherjee@gmail.com>:
Bug#971506. (Thu, 01 Oct 2020 08:57:03 GMT) (full text, mbox, link).


Message #8 received at 971506-submitter@bugs.debian.org (full text, mbox, ------------------------------------------------------------------------ > > (this message was generated automatically) > -- > Greetings > > https://bugs.debian.org/971506 > > &subject=Re: Bug#971506 marked as pending in trydiffoscope&In-Reply-To=<5f75994aad2_3dd2b0396496744530b0@godard.mail>">reply):

From: Chris Lamb <lamby@debian.org>
To: 971506-submitter@bugs.debian.org
Subject: Bug#971506 marked as pending in trydiffoscope
Date: Thu, 01 Oct 2020 08:54:34 +0000
Control: tag -1 pending

Hello,

Bug #971506 in trydiffoscope reported by you has been fixed in the
Git repository and is awaiting an upload. You can see the commit
message below and you can check the diff of the fix at:

https://salsa.debian.org/reproducible-builds/trydiffoscope/-/commit/e6d983833f2abe7afbcc85bd061feaef28a33ffa

------------------------------------------------------------------------
Mark --help-only test as being "superficial". (Closes: #971506)
------------------------------------------------------------------------

(this message was generated automatically)
-- 
Greetings

https://bugs.debian.org/971506



Added tag(s) pending. Request was from Chris Lamb <lamby@debian.org> to 971506-submitter@bugs.debian.org. (Thu, 01 Oct 2020 08:57:03 GMT) (full text, mbox, link).


Reply sent to Chris Lamb <lamby@debian.org>:
You have taken responsibility. (Thu, 01 Oct 2020 09:09:05 GMT) (full text, mbox, link).


Notification sent to Sudip Mukherjee <sudipm.mukherjee@gmail.com>:
Bug acknowledged by developer. (Thu, 01 Oct 2020 09:09:05 GMT) (full text, mbox, link).


Message #15 received at 971506-close@bugs.debian.org (full text, mbox, - Add a (flaky, needs-internet) test that interacts with the > try.diffoscope.org service. > * Bump Standards-Version to 4.5.0. > * Bump debhelper compatibility level to 13. > Checksums-Sha1: > 0dd095a189a81b0208b41f9100a875362a37e528 1778 trydiffoscope_67.0.5.dsc > 24b69fc14cb082648dbd00127f8e442caa6ff9b8 8512 trydiffoscope_67.0.5.tar.xz > 91f21bf7e33fecec06c8bc6c6b846ae21579264c 5982 trydiffoscope_67.0.5_amd64.buildinfo > Checksums-Sha256: > c39893c6978ab4c3175d4d41a46f40d51806cbb3dcdfb523b8a64f2b3ba079cb 1778 trydiffoscope_67.0.5.dsc > 054f4989f781d13988632e5ba0414283d591f0bbf7f4d1139c110a1a7d732ebd 8512 trydiffoscope_67.0.5.tar.xz > da3b07b269cbaf5646f11e0285ce81e821a5848ed9b32dae21ec88aaad3a2adf 5982 trydiffoscope_67.0.5_amd64.buildinfo > Files: > 65a07084340eb485f5f6d1ea6a4f14b5 1778 devel optional trydiffoscope_67.0.5.dsc > 7d27c33aa18d6620cc0bf05201f17ef5 8512 devel optional trydiffoscope_67.0.5.tar.xz > 70b3ea923a9470d75606e91ae436b43e 5982 devel optional trydiffoscope_67.0.5_amd64.buildinfo > > -----BEGIN PGP SIGNATURE----- > &References=">reply):

From: Debian FTP Masters <ftpmaster@ftp-master.debian.org>
To: 971506-close@bugs.debian.org
Subject: Bug#971506: fixed in trydiffoscope 67.0.5
Date: Thu, 01 Oct 2020 09:05:05 +0000
Source: trydiffoscope
Source-Version: 67.0.5
Done: Chris Lamb <lamby@debian.org>

We believe that the bug you reported is fixed in the latest version of
trydiffoscope, which is due to be installed in the Debian FTP archive.

A summary of the changes between this version and the previous one is
attached.

Thank you for reporting the bug, which will now be closed.  If you
have further comments please address them to 971506@bugs.debian.org,
and the maintainer will reopen the bug report if appropriate.

Debian distribution maintenance software
pp.
Chris Lamb <lamby@debian.org> (supplier of updated trydiffoscope package)

(This message was generated automatically at their request; if you
believe that there is a problem with it please contact the archive
administrators by mailing ftpmaster@ftp-master.debian.org)


-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

Format: 1.8
Date: Thu, 01 Oct 2020 09:53:23 +0100
Source: trydiffoscope
Architecture: source
Version: 67.0.5
Distribution: unstable
Urgency: medium
Maintainer: Reproducible builds folks <reproducible-builds@lists.alioth.debian.org>
Changed-By: Chris Lamb <lamby@debian.org>
Closes: 971506
Changes:
 trydiffoscope (67.0.5) unstable; urgency=medium
 .
   * Autopkgtests:
     - Mark a "--help"-only test as being "superficial". (Closes: #971506)
     - Add a (flaky, needs-internet) test that interacts with the
       try.diffoscope.org service.
   * Bump Standards-Version to 4.5.0.
   * Bump debhelper compatibility level to 13.
Checksums-Sha1:
 0dd095a189a81b0208b41f9100a875362a37e528 1778 trydiffoscope_67.0.5.dsc
 24b69fc14cb082648dbd00127f8e442caa6ff9b8 8512 trydiffoscope_67.0.5.tar.xz
 91f21bf7e33fecec06c8bc6c6b846ae21579264c 5982 trydiffoscope_67.0.5_amd64.buildinfo
Checksums-Sha256:
 c39893c6978ab4c3175d4d41a46f40d51806cbb3dcdfb523b8a64f2b3ba079cb 1778 trydiffoscope_67.0.5.dsc
 054f4989f781d13988632e5ba0414283d591f0bbf7f4d1139c110a1a7d732ebd 8512 trydiffoscope_67.0.5.tar.xz
 da3b07b269cbaf5646f11e0285ce81e821a5848ed9b32dae21ec88aaad3a2adf 5982 trydiffoscope_67.0.5_amd64.buildinfo
Files:
 65a07084340eb485f5f6d1ea6a4f14b5 1778 devel optional trydiffoscope_67.0.5.dsc
 7d27c33aa18d6620cc0bf05201f17ef5 8512 devel optional trydiffoscope_67.0.5.tar.xz
 70b3ea923a9470d75606e91ae436b43e 5982 devel optional trydiffoscope_67.0.5_amd64.buildinfo

-----BEGIN PGP SIGNATURE-----

iQIzBAEBCAAdFiEEwv5L0nHBObhsUz5GHpU+J9QxHlgFAl91mUwACgkQHpU+J9Qx
HlgvhQ/6A5LzJ9cA0EQI9cOaSvGU6yWAzqbYg3qJYtgb+a6qw/j5ppxaMlIAzJok
eI9pdLShhNxsbxIkgm88EHYOQP3JVG6/z0xYLK+tuXh5qad2DI9lFkGTB96p3Lag
0ErVlxcZ9NNj/7PAU8EKbiodPffN80R5269g2bFDQgVz26Bpc4jNJ5piSXkYSgN5
+JLJ26JqgiXqA8vweQyljwQ2H5UzrPBlGIDG9iOwuaYVOq6RM21pVowv5tNt/JGt
zlnfm2qJ+RK1iDab291U1prfOwk5uOUQARM4Pw8RonBgNpGlxWeJJJBW9m7Tzq53
rYFSVWYFtS0tSCcK8obpYB3LQqyVQpqgpi8qN1deL91KhTjoMBH+eRLwudUze3rU
DdwnBLhhZQ6FDYExsF7XZniGwgHKvDEDfmeWWOjKcDnXnTBPR9P+834aFtx+Y6Lh
Dk8fB2H2ebETdUeigOUrrZyOC1t0IgX03g9jXv+V+03XNZM2M+jSwoUp8SLtUD9g
BLnJqz0WhVUS6w7rdkf1xXHbDzfzZf0qytkaRdLu78uLIicCTvupyoyd2aAppmNB
yACvk34qRL4eHxMhcuJMCZhXbRLpK2s06jBOyeYe5TynuLFyAFd1j1Uqx29tT4+/
4QXEjPSMoyTVHD7glFu2V5Ov2RYCDQEwwY+Qli2RjtPxEXGP21s=
=wgTw
-----END PGP SIGNATURE-----




Bug archived. Request was from Debbugs Internal Request <owner@bugs.debian.org> to internal_control@bugs.debian.org. (Tue, 03 Nov 2020 07:25:02 GMT) (full text, mbox, link).


Send a report that this bug log contains spam.


Debian bug tracking system administrator <owner@bugs.debian.org>. Last modified: Mon Feb 3 05:54:49 2025; Machine Name: bembo

Debian Bug tracking system

Debbugs is free software and licensed under the terms of the GNU General Public License version 2. The current version can be obtained from https://bugs.debian.org/debbugs-source/.

Copyright © 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson, 2005-2017 Don Armstrong, and many other contributors.