Merge branch '124-package-build-infrastructure-needs-documentation-2' into 'master'

Resolve "Package-build infrastructure needs documentation"

See merge request Librem5/developer.puri.sm!296

Contact.rst

@@ -16,3 +16,4 @@ and patches.
    Contact/Contributing
    Contact/Contributing/Translations
    Contact/Issues
+   Contact/Package_Building

Contact/Contributing.rst

@@ -91,25 +91,3 @@ Do NOT merge the request if any of these are true:
 * No project member tested the current version of the MR.
 * There are unresolved discussions.
 * A maintainer was asked to take a look at the MR, but didn't have the chance to speak yet.
-
-Submitting Repositories for Package Building
-********************************************
-
-.. note:: This section is mostly useful for core contributors.
-
-Some components are built by the Continuous Integration (CI) system as Debian packages to enable convenient distribution, installation and upgrade. Information about these components is held in the `deb-build-jobs`_ repository, where the ``jobs.yml`` file contains metadata for each of them.
-
-To add a new component, follow these steps:
-
-* Fork the `deb-build-jobs`_ repository into your own space on the server.
-* Clone your new fork locally.
-
-  * Make a feature branch that is a pristine copy of the default branch that is checked out after cloning; e.g. with ``git checkout -b add-component-name``
-  * Add metadata to the ``jobs.yml`` file for the new component, including the URL of the source code repository, the branch to build, and the target distributions to build packages for.
-  * Commit the changes to your branch.
-
-* When you are ready for the changes to be merged back into the production repository, push your branch back to your fork on the server.
-
-When pushing your changes, you should be automatically prompted to make a merge request and given a URL that you can visit to do this. Otherwise, you can manually create a merge request for your branch from the web interface on the server.
-
-.. include:: /links.txt

Contact/Package_Building.rst

+Package Building
+================
+
+.. note:: This section is mostly useful for core contributors.
+
+Some components are built by the Continuous Integration (CI) system as Debian packages to enable convenient distribution, installation and upgrade. This requires that the components have Debian packaging files that can be used by the CI system to build packages.
+
+.. contents::
+   :local:
+
+See Debian's advice on `Packaging Your Product`_ and the `Debian New Maintainers' Guide`_ if you need advice on creating packaging files.
+
+Build Jobs and Packages
+-----------------------
+
+Information about the components to be packaged is held in the `deb-build-jobs`_ repository, where the ``jobs.yml`` file contains metadata for each of them. This is extracted and passed to a script that runs the `git-buildpackage`_ tool to perform the package build.
+
+The packages built by the CI system are published on the `Debian Package Builds for the Librem 5`_ page and most of them are also published in the `Librem 5 scratch apt repository`_.
+
+Metadata Format
+---------------
+
+The metadata takes the following form::
+
+   <component_name>:
+      url: <URL of git repository>
+      archs: <list of target architectures>
+      branch: <git branch name>
+      dists: <list of distributions to build for>
+
+Only the ``url`` parameter is required, but the ``branch`` parameter should also be set. Defaults for the other parameters will be used if they are not specified. The following example shows metadata for the ``phosh`` component::
+
+   phosh:
+     url: https://source.puri.sm/Librem5/phosh.git
+     dists: ['buster+ci']
+     branch: 'master'
+     deb_build_options: ['nocheck']
+
+Parameters other than ``url``, ``archs``, ``branch`` and ``dists`` can also be used to configure the build process for a component. The following list describes some of the parameters currently in use:
+
+``archs``
+   The architectures the component will be built for, specified as a Python-style list. It is useful to restrict this choice if building a component that is only suitable for a particular architecture. |br|
+   Default value: ``['aarch64', 'armhf', 'amd64']``
+``branch``
+   The branch in the git repository to use when building. The default value is based on the operating system and distribution being built for. It is better to explicitly specify a branch such as ``master`` than to rely on the default value.
+``deb_build_options``
+   Options for the ``DEB_BUILD_OPTIONS`` environment variable, specified as a Python-style list. These are passed to the ``gdp`` tool when a package is built. |br| No default value.
+``deb_build_profiles``
+   Options for the ``DEB_BUILD_PROFILES`` environment variable, specified as a Python-style list. These are passed to the ``gdp`` tool when a package is built. |br| No default value.
+``dists``
+   The distributions to build the package for, specified as a Python-style list. Generally, you should choose the same distribution as the core components. |br|
+   Default value: ``['buster']``
+``repo``
+   Name of the aptly repository used to host the build artifacts. |br|
+   Default value: ``scratch``
+``submodule_update``
+   Whether the repository's submodules are updated before building. |br|
+   Default value: ``true``
+
+A full, up-to-date list of parameters and their default values can be found in the `script <groovy_script_>`_ used to process the ``jobs.yml`` file.
+
+It can also be useful to look at `previous merge requests <deb-build-jobs_merged_>`_ to get an idea of which values are most suitable for a given component.
+
+Adding a New Component
+----------------------
+
+To add a new component, follow these steps:
+
+* Fork the `deb-build-jobs`_ repository into your own space on the server.
+* Clone your new fork locally.
+
+  * Make a feature branch that is a pristine copy of the default branch that is checked out after cloning; e.g. with ``git checkout -b add-component-name``
+  * Add metadata to the ``jobs.yml`` file for the new component, including the relevant fields described in the previous section. Note that the components are listed in alphabetical order, so ensure that the new component maintains the correct order.
+  * Commit the changes to your branch.
+  * Test the changes locally -- see :ref:`contributing_testing`.
+* When you are ready for the changes to be merged back into the production repository, push your branch back to your fork on the server.
+
+When pushing your changes, you should be automatically prompted to make a merge request and given a URL that you can visit to do this. Otherwise, you can manually create a merge request for your branch from the web interface on the server.
+
+.. _contributing_testing:
+
+Testing a New Component Locally
+-------------------------------
+
+Before submitting a merge request to add a new component for the CI infrastructure to build, it is best to ensure that packages for the component can be built correctly. This is typically done on the developer's workstation by following the usual steps to build a Debian package.
+
+.. include:: /links.txt
+.. _deb-build-jobs_merged: https://source.puri.sm/Librem5/deb-build-jobs/merge_requests?scope=all&utf8=%E2%9C%93&state=merged
+.. _groovy_script: https://source.puri.sm/Librem5/deb-build-jobs/blob/master/jobs.groovy

links.txt

@@ -6,6 +6,8 @@
 .. _`community/librem-5-devkit`: https://matrix.to/#/#librem-5-devkit:talk.puri.sm
 .. _`coredumpctl man page`: https://www.freedesktop.org/software/systemd/man/coredumpctl.html
 .. _`Debian`: https://www.debian.org
+.. _`Debian New Maintainers' Guide`: https://www.debian.org/doc/manuals/maint-guide/
+.. _`Debian Package Builds for the Librem 5`: https://arm01.puri.sm/job/debs/
 .. _`Debian package submission procedures`: https://www.debian.org/doc/manuals/distribute-deb/distribute-deb.html#adding-packages-to-debian
 .. _`deb-build-jobs`: https://source.puri.sm/Librem5/deb-build-jobs
 .. _`Desktop Entry`: https://specifications.freedesktop.org/desktop-entry-spec/desktop-entry-spec-latest.html
@@ -107,6 +109,7 @@
 .. _libhandy website : https://source.puri.sm/Librem5/libhandy
 .. _`Librem 5 developer documentation repository`: https://source.puri.sm/Librem5/developer.puri.sm
 .. _`Librem 5 examples project`: https://source.puri.sm/Librem5/example-apps/
+.. _`Librem 5 scratch apt repository`: https://ci.puri.sm/dists/scratch/
 .. _`librem5-devkit-tools`: https://source.puri.sm/Librem5/librem5-devkit-tools
 .. _`linux-emcraft repository`: https://source.puri.sm/Librem5/linux-emcraft/
 .. _`linux-emcraft issue 2`: https://source.puri.sm/Librem5/linux-emcraft/issues/2
@@ -117,6 +120,7 @@
 .. _`net.hadess.SensorProxy`: https://developer.gnome.org/iio-sensor-proxy/1.0/gdbus-net.hadess.SensorProxy.html
 .. _`Nightly GNOME Apps`: https://wiki.gnome.org/Apps/Nightly
 .. _`Ninja`: https://ninja-build.org/
+.. _`Packaging Your Product`: https://www.debian.org/doc/manuals/distribute-deb/distribute-deb.html#packaging
 .. _`Plasma Mobile`: https://www.plasma-mobile.org
 .. _`Plasma Mobile application development`: https://docs.plasma-mobile.org/AppDevelopment.html
 .. _`Issues for Plasma Mobile on Librem 5`: https://source.puri.sm/groups/Librem5/Plasma/-/issues