How to Cut a Release

Due to the length of time that this process takes, it is recommended to do the release over two days to allow for errors to be corrected and because some of the koji commands take a long time to run when a large number of rpms are being released.

This document doesn't discuss the policy for deciding what goes into a release. The release policy is discussed elsewhere.

NOTE Interactive scripts can be found at and can be checked out with SVN.

Pick the version number

This page will help you get the various commands exactly right. To do this, enter the exact version number (such as 3.0.5) and click submit. If you do not do this, the commands below will be missing essential text.

Version: Upcoming:

Day 0

Step 1: Update the osg-version RPM

Edit the osg-version RPM's spec file to update the version number to VERSION. Build it in koji:

# If building for the latest release out of trunk:
osg-build koji osg-version
# If building for an older release out of a branch:
osg-build koji --repo=VERSION osg-version

Step 2: Generate the tentative release list

The release manager often needs a preliminary list of packages slated for the next release a few weeks in advance. This is done by finding the package differences between what's in testing and what's in the current release. There is a script in SVN that handles this:

./0-generate-pkg-list VERSION

Day 1: Populate Pre-Release and Testing

Step 1: Populate Pre-Release

Populate the pre-release. This requires you to understand which packages should be in the release. This is a question of policy. If you do not know which packages should be in the release, talk to the OSG Release Coordinator.

./1-populate-prerelease VERSION

There shouldn't be any packages in prerelease and you will prompted to untag any packages that are still in there before continuing. Afterwards, you will need to manually edit the package list to make sure the proper packages get promoted (consult the Release Manager for details).

NOTE: Edit pkgs-to-release-* to remove packages not intended for this release. Please verify that the osg-version RPM is in your set of packages for the release! Also verify that if there is a new version of the osg-tested-internal RPM, then it is included in the release as well!

Step 2: Test Pre-Release in VM Universe

Make sure you meet the pre-requisites required to submit VM Universe jobs on After that's done, you can prepare the test suite by running:

osg-run-tests 'Testing OSG pre-release'

After you cd into the directory specified in the output of the previous command, you will need to edit test-parameters.yaml to reflect the tests that you will want to run i.e. clean installs, upgrade installs and upgrade installs between OSG versions. The standard set of test parameters looks like the following:

  - centos_5_x86_64
  - centos_6_x86_64
  - rhel_5_x86_64
  - rhel_6_x86_64
  - sl_5_x86_64
  - sl_6_x86_64

  - 3.1; osg-prerelease                      # Tests clean installs of 3.1 pre-release
  - 3.1; osg > osg-prerelease                # Tests update installs of 3.1 pre-release
  - 3.2; osg-prerelease                      # Tests clean installs of 3.2 pre-release
  - 3.2; osg > osg-prerelease                # Tests update installs of 3.2 pre-release
  - 3.1; osg > 3.2/osg-prerelease            # Tests update installs between 3.1 and 3.2 pre-release
  - 3.1; osg-prerelease > 3.2/osg-prerelease # Tests update installs between 3.1 pre-release and 3.2 pre-release

  - [osg-tested-internal]

Once you're satisfied with your list of parameters, submit the dag:

condor_submit_dag master-run.dag

NOTE: If there are failures, consult the release-manager before proceeding.

Step 2: Create the client tarballs

Create the client tarballs using the scripts in software/tarball-client/trunk from our SVN, and then copy them into the UW's AFS. Do this as root on an EL6 fermicloud machine. Change the username and machine name as appropriate. Note: the prerelease repos must have been populated and regenerated for this to work; normally that happens in Step 2.

yum -y install subversion yum-priorities
mkdir -p /tmp/tarball-client
cd /tmp/tarball-client
svn export
trunk/make-client-tarball --osgver=VERSION --prerelease --all
scp *.tar.gz $ruser@$rmachine:/p/vdt/public/html/tarball-client/
You should get 8 tarballs, 30-55 megs each. They should all have VERSION in the name.

NOTE If you are running into 404 errors concerning specific packages, this may be due to stale build repos. To regenerate the build repos, run:

[user@client ~]$ osg-koji regen-repo osg-<osg version>-<distro version>-release-build

For example, run the following command to regenerate the OSG 3.2 EL6 build repo:

[user@client ~]$ osg-koji regen-repo osg-3.2-el6-release-build

Step 3: Briefly test the client tarballs

As an unprivileged user, extract each tarball into a separate directory. Make sure osg-post-install works. Make sure 'osgrun osg-version' works.

for client in osg-client osg-wn-client; do
    for rhel in el5 el6; do
        for arch in i386 x86_64; do
            mkdir -p $rhel-$arch
            pushd $rhel-$arch
            tar xzf ../$file
            $client/osgrun osg-version

If you have time, try some of the binaries, such as grid-proxy-init.

NOTE: We need to automate this and have it run on the proper architectures and version of RHEL.

Step 4: Update the UW AFS installation of the tarball client

The UW keeps an install of the tarball client in /p/vdt/workspace/tarball-client on the UW's AFS. There is a script to update it called afs-install-tarball-client. Run it as follows:

/p/vdt/workspace/tarball-client/afs-install-tarball-client VERSION

Step 5: Remove old UW AFS installations of the tarball client

To keep space usage down, remove tarball client installations and symlinks under /p/vdt/workspace/tarball-client that are more than 2 months old. The following command can list them:

find  /p/vdt/workspace/tarball-client  -maxdepth 1  -mtime +60  -name 3\*  -ls

Step 6: Wait

Wait for clearance. The OSG Release Coordinator (in consultation with the Software Team and any testers) need to sign off on the update before it is released. If you are releasing things over two days, this is a good place to stop for the day.

Day 2: Pushing the Release

NOTE: For the second phase of the release, try to complete it earlier in the day rather than later. The GOC would like to send out the release announcement prior to 3 p.m. their time (Eastern time zone), which means that our procedure needs to be completed before then.

Step 1: Push from pre-release to release

This script moves the packages into release, clones release into a new VERSION specific release repo, locks the new repo and regenerates it. Afterwards, it produces *release-note* files that should be given to whomever's writing the release notes.

./2-create-release VERSION

*.txt files are also created and it should be verified that they've been moved to /p/vdt/public/html/release-info/ on UW's AFS. There is also a section of the release notes that lists the packages that have been updated. To get this list, run:

list-package-updates VERSION

Step 2: Upload the client tarballs

Ask Mat or someone with privileges on the repo servers to upload the tarballs with the following procedure:

On a CS machine

cd /p/vdt/public/html/tarball-client
ssh mkdir /tmp/VERSION/
scp *VERSION*gz


scp -r /tmp/VERSION repo1:/tmp/
scp -r /tmp/VERSION repo2:/tmp/
rm -rf /tmp/VERSION

On repo1/repo2 (as root)

You can ssh to repo1 and repo2 from; you will need to do this procedure on both systems.
sudo su -
mv /tmp/VERSION /usr/local/repo/tarball-install/VERSION/
rm -f /usr/local/repo/tarball-install/VERSION/*latest*
ls -l /usr/local/repo/tarball-install/VERSION/*latest* # verify the symlinks are correct

Step 3: Install the tarballs into OASIS

You must be an OASIS manager of the 'mis' VO to do these steps. Known managers as of 2014-07-22: Mat, Tim C, Tim T, Brian L. Get the uploader script from SVN and run it with osgrun from the UW AFS install of the tarball client you made earlier. On a UW CSL machine:

svn cat file:///p/vdt/workspace/svn/software/tarball-client/trunk/upload-tarballs-to-oasis > /tmp/upload-tarballs-to-oasis
chmod +x /tmp/upload-tarballs-to-oasis
/p/vdt/workspace/tarball-client/current/sys/osgrun /tmp/upload-tarballs-to-oasis VERSION
The script will automatically ssh you to and give you instructions to complete the process.

Step 4: Announce the release

The following instructions are meant for the release manager (or interim release manager). If you are not the release manager, let the release manager know that they can announce the release.

  1. The release manager writes the release announcement and send it out. Here is a sample:

    Subject: Announcing OSG Software version VERSION

    We are pleased to announce OSG Software version VERSION!

    This is the new OSG Software distributed via RPMs for:

    * Scientific Linux 5 and 6
    * CentOS 5 and 6
    * Red Hat Enterprise Linux 5 and 6

    This release affects the SET-OF-METAPACKAGES (client, compute element, etc...). Changes include:

    * Major change 1
    * Major change 2
    * Major change 3

    Release notes and pointers to more documentation can be found at:

    Need help? Let us know:

    We welcome feedback on this release!

  2. The release manager emails the announcement to
  3. The release manager asks the GOC to distribute the announcement by opening a ticket
  4. The release manager closes the tickets marked 'Ready for Release' in the release's JIRA filter
  5. The release manager updates the recent/scheduled release tables on the Software/Release homepage

Topic revision: r93 - 10 Mar 2015 - 20:32:02 - BrianLin
Hello, TWikiGuest


TWiki | Report Bugs | Privacy Policy

This site is powered by the TWiki collaboration platformCopyright by the contributing authors. All material on this collaboration platform is the property of the contributing authors..