Copr - look back at 2022
Let me sum up what the Copr team did during 2022.
- We did six releases of Mock Starting with a major upgrade to 3.x that dropped python2 support and EL7 as the host platform.
- We added
--list-chrootsoption. Allow better customization of used tar binary and adapt to the new split of qemu-user-static package.
- We also added lots of new chroots: AlmaLinux, RockyLinux, EuroLinux, OpenEuler, and a few others…
- We did nine releases of Copr and upgraded Copr servers to Fedora 37.
- We wrote two “4 cool new projects to try in Copr” articles for Fedora Magazine
Beside previously built of all gems from Rubygems.org we built all modules from PyPI as RPM packages [Thousands of PyPI and RubyGems RPMs now available for RHEL 9 Red Hat Developer](https://developers.redhat.com/articles/2022/06/07/thousands-pypi-and-rubygems-rpms-now-available-rhel-9) Big thanks to Karolina Surma from Python team on cooperation on this. As a side effect, we introduced pyp2spec as a second option to build directly from PyPI.
- We presented at Fedora Nest.
- We dropped APIv2. And provided guidance how to migrate your scripts https://fedora-copr.github.io/posts/api3-migration-helper
- We added Kerberos authentication to command line tools and API https://fedora-copr.github.io/posts/how-to-use-kerberos-in-copr
- We cooperated with Packit on building SRPM in Copr https://packit.dev/postcs/copr-srpms/
- We started using IBM Cloud for native s390x builders https://pavel.raiskup.cz/blog/fedora-copr-uses-ibm-cloud.html
As a side effect we packaged python modules for managing resources in IBM Cloud.
- We spent lots of time optimizing the scheduler in Copr. E.g.
- Builds from webhooks are now background jobs https://docs.pagure.org/copr.copr/release-notes/2022-02-03.html#webhook-rebuilds-are-background-jobs-now
- We improved the throughput when the queue is bigger than 70k jobs https://docs.pagure.org/copr.copr/release-notes/2022-03-21.html#large-queue-improvements
- And we were able to increase quota of parallel builds from one user from 35 to 45.
- We started using SHA256 for signing packages https://docs.pagure.org/copr.copr/release-notes/2022-03-21.html#signing-packages-with-sha256
- We started using OpenPGP v4 signatures and we were one of the first to discover issues with new Sequoia backend of RPM with older version of signatures.
- We created a webUI statistics page that shows the utilization of resalloc resources https://docs.pagure.org/copr.copr/release-notes/2022-06-22.html#resalloc-webui
- We (finally) were able to count download statistics from CDN https://docs.pagure.org/copr.copr/release-notes/2022-08-18.html#rpm-download-statistics
- You can submit more builds at once from command line https://docs.pagure.org/copr.copr/release-notes/2022-07-27.html#submitting-multiple-builds-at-once-via-copr-cli
- We upgraded aarch64 builders to stronger Graviton3 machines https://docs.pagure.org/copr.copr/release-notes/2022-11-28.html#updated-aarch64-builders-to-graviton3-processors
- We migrated our git project to GitHub https://docs.pagure.org/copr.copr/release-notes/2022-11-28.html#development-moved-to-github
- We migrated to new storage as we hit 16 TB hard limit for one volume in AWS. It took us few weeks of preparation to make just a few hour outage for building packages and almost no outage for yum repositories availability.
- Fedora Infrastructure helped us to provision a new Power9 box (for ppc64le builds) which hosts 25 builders (which complements the set of Power9 machines in OSUOSL). Note that Power9+ is needed for Enterprise Linux 9 builds.
- We created OpenShift and Kustomize deployment script.
- Copr run 1 251 633 builds (one package for different chroots is conted as one)
- People created 23 053 new projects.
- We participated in SPDX License migration phase 1 https://fedoraproject.org/wiki/Changes/SPDX_Licenses_Phase_1
- We announced the initial release of Package Review service https://firstname.lastname@example.org/message/E4TT2PEOSITJ4PJP44L2GQUU4CA6R6B3/
- We contributed to fedora-review to produce machine-readable output https://pagure.io/FedoraReview/pull-request/463
- One modulemd-tools release.
- FIXME another notable fedora-review contribution?
- We did three releases of Tito
- We did seven releases of Resalloc
- RPM Spec Wizard got a nice front page https://xsuchy.github.io/rpm-spec-wizard/
- Copr users are known to build the same NEVRAs multiple times in the same project, which confuses DNF and other tools. We contributed to createrepo_c https://github.com/rpm-software-management/createrepo_c/pull/325 which is the first step to deterministically resolve the problem.
- The “differential PyLint checker” we used for the last few years was separated from the Copr base code into https://github.com/fedora-copr/vcs-diff-lint and it can now be easily used by other projects. We also provide a “glue” GitHub Action wich makes it’s use trivial, see integration example https://github.com/rpm-software-management/mock/blob/main/.github/workflows/python-diff-lint.yml
Outlook for 2023
- Integration with Koshei - automatic rebuild of your package in your project when dependency changes (inherited from the previous year - not even started yet)
- Hopefully start using ImageBuilder.
- We are investigating the usage of Pulp as a backend for storing RPM packages.
Ideas we have for 2023 - some of them are inherited from the previous year:
Mock --chainto try to set %bootstrap when the standard loop fails. When the set succeeds, rebuild the bootstrapped package again without the %bootstrap macro.
- Contribute to fedpkg/koji to have machine-readable output. FIXME
- Include packages from Copr in results of https://packages.fedoraproject.org/
- Allow running various tools right after the build - e.g., rpminspect, swidtags.
- Automatically verify if your package’s license can be used in Copr.
The Community Packaging Team consists of Pavel Raiskup, Jakub Kadlcik, Jiří Kyjovský (who replaced Silvie Chlupova), and me.