[guillem@debian.org: Fix typos in comments. ]

Ref: https://bugs.launchpad.net/bugs/2002582


Signed-off-by: Guillem Jover <guillem@debian.org>

The recently added lto handling for Ubuntu, did not take into account
that the DEB_BUILD_OPTIONS and DEB_BUILD_MAINT_OPTIONS environment
variables are parsed and their options applied just after setting the
defaults, and before applying any arch-specific mask.

We add a new init_build_features() internal method, which sits between
these two actions, so that the Ubuntu vendor module can modify the
defaults before any user or maintainer override are applied.

Fixes: commit df7627ac
Fixes: https://bugs.launchpad.net/bugs/2002582

The source ChangeLog.old file might have different permissions depending
on the umask during the source package unpack. Copy the file without
any flags, and then explicitly set known permissions on the result file.

This should fix the build on reproducible variants.

Fixes: commit 2c2f7066

When verifying a signature, we were using the internal dearmor() method,
as that makes it possible to not have gpg around when we are using gpgv.

The problem is that the internal dearmor() method does not handle
concatenated ASCII Armor blocks, and might then fail to find the
certificates. When using gpgv this is only a partial regression as
we were previously not verifying at all on minimal systems where gpg
was not available. But when it is available, now that has regressed.

In the future we might require no concatenated ASCII Armor blocks,
but for now let's mitigate this regression.

Fixes: commit a11d7340
Reported-by: Sven Joachim <svenjoac@gmx.de> (on IRC)

The gpgv command expects a trustedkeys.gpg keyring, and if there is none
it will try the trustedkeys.kbx one, but then will emit an error such as:

  gpgv: unknown type of key resource 'trustedkeys.kbx'
  gpgv: keyblock resource '<GPGHOME>/trustedkeys.kbx': General error

This error will only show if the gpgv invocation failed for other
reasons, but it is still an unnecessary distraction. Avoid that error,
by touching the trustedkeys.gpg keyring.

Closes: #1028981
Signed-off-by: Guillem Jover <guillem@debian.org>

The current GnuPG defaults with --openpgp cater for heavy backwards
compatibility at the cost of being insecure but potentially being
compatible with very old programs.

We care more about secure defaults than backwards compatibility with
ancient programs, so we pass our preferences to gpg when signing. This
should also cover the case for users that have created old keys with
insecure key preferences which might end up producing insecure
signatures.

Fixes: commit b83114da
Closes: #1028961

Update to 1184t.

Closes: #1008486

This adds support for .deb ZStandard compression and decompression.
The main reason for this addition is due to Ubuntu having forked the
.deb ecosystem when they added support for this unilaterally, so now
there are many .deb in the wild using this compression format, which
cannot be handled by the upstream dpkg-deb tool.

Although at least now the ZStandard format is widely used on many
projects, has been specified within the IETF as RFC8878, so backwards
compatibility and format stability are no longer a concern, and it has
good trade offs between size and speed.

This has been implemented from scratch, based on the initial prototype
code used during the early request evaluation. Using the new advanced
API, with support for multi-threading, and using an I/O loop resembling
the one used with liblzma, as the plan is to eventually switch all
compressors to use a single I/O loop implementation.

Closes: #892664

Use a new enum dpkg_stream_status to track the stream status, instead of
using the liblzma codes directly in the I/O loop, which will make it
possible in the future to generalize and use a single loop for all
compressors.

Stop handling the xz/lzma actions using the liblzma specific actions,
and use a generic enum, renamed from the old dpkg_stream_status, which
was in fact partially handling the action. This will make it easier in
the future to use the same action for all compressors.

Tracking the compression filter operation and its status merges two
distinct values into the same variable making things more difficult,
as we were using bits when these should really be mutually exclusive
values.

Instead of passing the dpkg_stream_action enum, pass the io_lzma struct,
so that we can use other members if needed.

The compression level max bound check is specific to each compressor,
and we cannot simply use a single max bound for every compressor. Move
the check into the compressor_check_params() function where we can check
whether it is coherent with the compressor requested.

The dpkg_options_parse_arg_int() already makes sure we get a sanitized
value that is 0 or larger but not larger than INT_MAX.

We were testing showing or listing the contents, but not explicitly
building the compressed packages nor extracting them.

Include the ChangeLog.old file which is big, and will better exercise
the compressors.

The regex was not correctly matching at the beginning of the line with
leading spaces, which could be left by a previous replacement from
the same s/// operator. Instead switch to split the flag value and
filter based on a hash, which means we do not need to care about such
space issues anymore.

This change should not change the semantics for space separated options
in the flags, as the code was already splitting the passed values on
spaces, and then remapping on the entire flag value.

Improve strip unit tests to cover all these cases.

Closes: #1028044

Signed-off-by: Johannes Schauer Marin Rodrigues <josch@debian.org>
Signed-off-by: Guillem Jover <guillem@debian.org>

It does not make much sense to ship the upstream «git log» changelog
which contains lots of fluff and metadata. At the same time the recent
debhelper trimming is not ideal as there's no easy way to fetch old
entries anymore. So stop shipping the upstream changelog in favor of
not trimming the debian/changelog one, which acts as a proper user
readable changelog.

Closes: #1027716

When bootstrapping a cross-compiler, it might not have some of the
required parts available, such as a libc. Catch the error, emit it as a
warning and return undef, like with the other conditions where we skip
the test. Because in the end this is just a tainting flag, and it is not
worth it failing the entire build due to that.

Closes: #1027966

Changelog: internal

The $file argument should be part of the error() arguments not the g_()
arguments.

The new version fixes the remaining issue that allows the code to use it
as a backend. Add it as a preferred implementation if available.

The handling for deb 0.x formats that relocates files around once
extracted was using a buffer with a hardcoded size, not taking into
account the length of the directory which would overflow it.

Switch to use a dynamically allocated buffer to handle any destination
directory length.

Reported-by: Georgy Yakovlev <gyakovlev@gentoo.org>

When xz does not have enough memory given the requested threads, it
might end up adjusting the memory usage and number of threads, while
emitting a warning, and then exiting with non-zero. Both of which we
do not want from the library.

If xz cannot meet the memory usage limit on multi-threaded mode, then
it can end up falling back to single-threaded mode, which would then
generate different output, making it non-reproducible. This has the
drawback that it can then error out.

When xz does not have enough memory given the requested threads, it
might end up adjusting the memory usage and number of threads, while
emitting a warning, and then exiting with non-zero. Both of which we
do not want from the library.