Over the last year, the Copr team has put a lot of effort into supporting Pulp as a storage for Copr build results. It is now time to realize the investment.

Status Quo

At this very moment, Copr hosts ~35k projects from ~8k Fedora users. The data consumption sits around ~37TB with only ~3TB of free disk space. With a very few exceptions, everything is stored in our “Copr backend” storage.

Our current storage is a RAID array of block devices with the Ext4 filesystem. For various reasons, AWS block devices larger than 16TB are either hard to maintain or expensive. We plan to solve this problem by migrating to an S3 object storage and Pulp, which is a mature and durable-by-default solution for managing software repositories.

We have already migrated all Copr team projects to Pulp to dogfood the integration on ourselves, and we have already migrated one large project to avoid running out of disk space.

The Goal

Our goal is to stop using the current “Copr backend” storage for all new projects and migrate all existing project results to Pulp.

Impact On Users

  • No change to your workflows should be needed.
  • Copr repositories enabled on user machines should continue working.
  • While your projects are being migrated, all your builds and actions will be stuck in the “pending” state until the migration is finished.
  • After the migration of your projects is finished, you shouldn’t notice any difference from the status quo.
  • For technical reasons, we won’t notify users directly that their projects are about to be migrated. Please see the Pulp Migration Schedule to get a rough estimate of when you could be affected.
  • Every user should be affected for at most 24 hours.

To summarize, the change should be transparent to you, there are no UI/UX changes, and your builds may be paused while your project is being migrated.

Our Plan

For a detailed, step-by-step plan with dates and tracked progress, please see the Pulp Migration Schedule. Roughly, the plan is as follows:

  1. Set the default storage for all new Packit projects to Pulp.
  2. Set the default storage for all new projects to Pulp.
  3. Migrate large projects (17 projects that are 100GB+).
  4. Migrate all projects with a few exceptions.
  5. Migrate everything that remains.

Contact

If you have any questions or worries regarding our migration of data to Pulp, please let us know: