Skip to content

WIP

This page is still work in progress.

Tagging a new CTA release

Info

Whenever possible, the next official CTA release (date, content and version) should always be discussed with the rest of the CTA team.

Info

New releases can be created for testing purposes. These can be created on a ad-hoc basis and must clearly be identified as testing. They must be removed once no longer needed.

The procedure for tagging a new CTA release, despite being mostly automatized, still requires a few manual steps. In this page we give some detailed guidelines on how to perform them.

1. Choose a version number for the new CTA release

CTA versions are in the format:

<xrootd>.<major>.<minor>.<patch>-<package>.

Example:

5.10.11.0-1

Where - <xrootd>: - XrootD version. - Currently only 5 is allowed. - <major>: - Major number, bumped up every time a new catalogue is released. - All releases with a certain <major> value depend on the same CTA Catalogue schema version. - <minor>: - Minor number, bumped up for every release that does not require a new Catalogue version. - Backward compatible, regarding Catalogue schema. - Does not guarantee to be backward compatible with other systems (i.e. EOS). Dependency software versions, such as EOS, should follow the versioning defined in versionlock.list. - <patch>: - Patch number, bumped when applying a fix to an old version (including public releases). - Should not include new features, only fixes. - <package>: - Package number, for any changes in the packaging/built that do require the code logic to be modified.

All DB schema releases should bump up the <major> version number (eg.: 4.10.0.0-1). Non-BD schema releases (just code updated) should bump up only the <minor> version number (eg.: 4.10.1.0-1).

Optionally, we can append a <suffix> to the CTA version:

<xrootd>.<major>.<minor>.<patch>-<package>.<suffix>.

Any value [a-zA-Z0-9]+ is accepted for <suffix>. It can be used to include extra information in the tag, without affecting how the CI works.

Some examples/conventions:

5.10.11.0-1.el9 # Alma 9 release, only needed until CentOS-7 reaches EOL

5.10.11.0-1.rc1 # Release canditate, for tagging test releases that will never be used in production.

5.10.11.0-1.test1 # Used for testing the tagging procedure itself

1.1 Special case: Catalogue release

As mentioned above, all DB schema releases should bump up the <major> version.

These releases should contain no code changes, unless this can't be avoided.

They can only contain the following changes:

  • Changing the cta-catalogue-schema submodule commit, in order to reference a new tagged CTA Catalogue release.
  • It's important that the CTA Catalogue version is tagged, and includes all the migration scrips.
  • Updating the supported Catalogue versions in catalogue/CTARefCatalogueSchemaVersion.cmake (click for link).
  • A CTA catalogue release, must be compatible with both the old and the new catalogue version.
  • Example:
    • If 5.11.0.0-1 introduces the new CTA Catalogue version 15.0 (updated from 14.0) then it must be compatible with both 14.0 and 15.0 versions, to allow for the upgrade to go smoothly.

2. Create a dev issue with the label "Release"

All CTA releases must be tracked by a dev issue with the label Release (click to list current releases).

The CTA project already provides a template for Release issues (click for link).

The developer should indicate on the release issue which commit and branch (preferably main) are going to be tagged. These values can later be modified, if necessary.

We should not be blocking development while the release is ongoing. The main branch should be able to receive new commits which will not be included in the release.

If necessary (ex: additional commits needed for a fix, patch release, etc.), the release can be done on a separate branch.

3. Verify that the Release Notes and cta.spec.in files are up-to-date

Before tagging a release, the developer should check that the Release Notes are up-to-date and contain all the features, bug fixes and improvements for this release. Each one of the entries in the Release Notes file should contain a link to an issue in the CTA project.

In addition to this, add a summary of the Release to the %changelog section of cta.spec.in (see below for more details).

In case of doubt about any commit, contact directly the creator of such commit.

Finally, commit these changes and update the new commit ID on the Release issue.

3.1 cta.spec.in %changelog section

The detailed changelog for each Release is in ReleaseNotes.md.

The cta.spec.in %changelog should be a brief summary and should only include important new features, major bug fixes and updates to dependencies. This is a summary for end-users who will install the RPM.

The text in the %changelog section of cta.spec.in is packaged with the RPM and can be viewed with tools like rpm --changelog and repoquery --changelog. It has to be formatted in a precise way as the %changelog is parsed by RPM package management tools. This is the format:

  • One line to declare a new Release:
* DoW Mon dd yyyy Firstname Lastname <firstname.lastname@cern.ch> - vX.Y.Z-R
  • Below it, one or more lines short summary of what is in the Release. Each line should start with -. If the description extends over more than one line (discouraged), the second and subsequent lines should be indented with two spaces. Examples:
- Major feature or new tool
  Continuation line for very long summaries
- Fixes major bug (cta/CTA#1234)
- Catalogue schema version X.Y
- Updates EOS to vX.Y.Z
- Updates XRootD to vX.Y.Z
- Updates Ceph to vX.Y.Z
  • A single blank line to separate each Release.

4. Run all important system tests

Check that all important system tests run successfully in the CI.

Besides all the mandatory tests, some of the optional ones can also be useful (if relevant):

  • liquibase-update: Important only for CTA Catalogue releases. Checks that the migration scripts from catalogue version X to Y result into a correct version Y schema.
  • external-tape-formats: Performs a test with the OSM tape format.

5. Perform a stress test

The release candidate should be stress tested.

The stress test will be available in a container-based deployment, instructions to follow (TODO).

Results from the stress test should be added to the release issue by the person performing the test.

Any issues arising from the stress test should be addressed. This will result in a new commit ID.

6. Tag the release, to trigger the release pipeline

Once all the above is complete, the new release can be tagged in Gitlab.

Tagging a release will trigger a "Release" pipeline, which will automatically build the RPMs for the new CTA release. Optionally, it also allows some system tests to be done.

7. Publish the RPMs

After the CI is complete, the developer can trigger two manual stages:

  • publish_tagged_rpm:
  • Publishes the RPMs to the CTA team internal repo. These become available for deployments at CERN.
  • release_tagged_rpm:
  • Publishes the RPMs to the CTA public repo (external). These become available for other sites to use.

Warning

The public release step should be taken only after the internal release has been in use in production for a few weeks, in order to guarantee that it's stable and that no buggy RPMs are released to the public. If needed, a testing release can be done to make the new RPMs immediately available to the public.

Info

For the moment, testing releases are not published automatically by the CI.

8. Deployment

Once the Release has been tagged, open a deployment operations ticket, indicating the tagged version that should be deployed.

This is usually performed by a CTA operator.