How to Cut a Release
Due to the length of time that this process takes, it is recommended to do the release over two days to allow for errors to be corrected and because some of the koji commands take a long time to run when a large number of rpms are being released.
This document doesn't discuss the policy for deciding what goes into a release. The release policy is discussed elsewhere
Interactive scripts can be found at https://vdt.cs.wisc.edu/svn/software/release-tools/
and can be checked out with SVN.
This page will help you get the various commands exactly right. To do this, enter the exact version number (such as 3.0.5
) and click submit. If you do not do this, the commands below will be missing essential text.
Step 1: Update the osg-version RPM
Edit the osg-version RPM's spec file to update the version number to VERSION. Build it in koji:
# If building for the latest release out of trunk:
osg-build koji osg-version
# If building for an older release out of a branch:
osg-build koji --repo=VERSION osg-version
Step 2: Generate the tentative release list
The release manager often needs a preliminary list of packages slated for the next release a few weeks in advance. This is done by finding the package differences between what's in testing and what's in the current release. There is a script in SVN that handles this:
Step 1: Populate Pre-Release
Populate the pre-release. This requires you to understand which packages should be in the release. This is a question of policy. If you do not know which packages should be in the release, talk to the OSG Release Coordinator.
There shouldn't be any packages in prerelease and you will prompted to untag any packages that are still in there before continuing. Afterwards, you will need to manually edit the package list to make sure the proper packages get promoted (consult the Release Manager for details).
to remove packages not intended for this release. Please verify that the
RPM is in your set of packages for the release! Also verify that if there is a new version of the
RPM, then it is included in the release as well!
Step 2: Test Pre-Release in VM Universe
Make sure you meet the pre-requisites
required to submit VM Universe jobs on
. After that's done, you can prepare the test suite by running:
osg-run-tests 'Testing OSG pre-release'
into the directory specified in the output of the previous command, you will need to edit
and remove all other yaml files in that directory. The standard set of test parameters looks like the following:
- 3.2; osg-prerelease # Tests clean installs of 3.2 pre-release
- 3.2; osg > osg-prerelease # Tests update installs of 3.2 pre-release
- 3.1; osg > 3.2/osg-prerelease # Tests update installs between 3.1 and 3.2 pre-release
Once you're satisfied with your list of parameters, submit the dag:
If there are failures, consult the release-manager before proceeding.
Step 2: Create the client tarballs
Create the client tarballs using the scripts in software/tarball-client/trunk from our SVN, and then copy them into the UW's AFS.
Do this as root on an EL6 fermicloud machine.
Change the username and machine name as appropriate.
Note: the prerelease repos must have been populated and regenerated for this to work;
normally that happens in Step 2.
yum -y install subversion yum-priorities
mkdir -p /tmp/tarball-client
svn export https://vdt.cs.wisc.edu/svn/software/tarball-client/trunk
trunk/make-client-tarball --osgver=VERSION --prerelease --all
scp *.i386.tar.gz $ruser@$rmachine:/p/vdt/public/html/tarball-client/VERSION/i386
scp *.x86_64.tar.gz $ruser@$rmachine:/p/vdt/public/html/tarball-client/VERSION/x86_64
You should get 8 tarballs, 30-55 megs each.
They should all have VERSION in the name.
If you are running into 404 errors concerning specific packages, this may be due to stale build repos. To regenerate the build repos, run:
[user@client ~]$ osg-koji regen-repo osg-<osg version>-<distro version>-release-build
For example, run the following command to regenerate the OSG 3.2 EL6 build repo:
[user@client ~]$ osg-koji regen-repo osg-3.2-el6-release-build
Step 3: Briefly test the client tarballs
As an unprivileged user
, extract each tarball into a separate directory.
Make sure osg-post-install works. Make sure 'osgrun osg-version' works.
for client in osg-client osg-wn-client; do
for rhel in el5 el6; do
for arch in i386 x86_64; do
mkdir -p $rhel-$arch
tar xzf ../$file
If you have time, try some of the binaries, such as grid-proxy-init.
We need to automate this and have it run on the proper architectures and version of RHEL.
Step 4: Update the UW AFS installation of the tarball client
The UW keeps an install of the tarball client in
on the UW's AFS.
There is a script to update it called
. Run it as follows:
Step 5: Remove old UW AFS installations of the tarball client
To keep space usage down, remove tarball client installations and symlinks under
that are more than 2 months old.
The following command can list them:
find /p/vdt/workspace/tarball-client -maxdepth 1 -mtime +60 -name 3\* -ls
Step 6: Wait
Wait for clearance. The OSG Release Coordinator (in consultation with the Software Team and any testers) need to sign off on the update before it is released. If you are releasing things over two days, this is a good place to stop for the day.
For the second phase of the release, try to complete it earlier in the day rather than later. The GOC would like to send out the release announcement prior to 3 p.m. their time (Eastern time zone), which means that our procedure needs to be completed before then.
This script moves the packages into release, clones release into a new VERSION specific release repo, locks the new repo and regenerates it. Afterwards, it produces *release-note* files that should be given to whomever's writing the release notes.
*.txt files are also created and it should be verified that they've been moved to /p/vdt/public/html/release-info/ on UW's AFS. There is also a section of the release notes that lists the packages that have been updated. To get this list, run:
Step 2: Upload the client tarballs
Ask Mat or someone with privileges on the grid.iu.edu repo servers to upload the tarballs with the following procedure:
On a CS machine
ssh jump.grid.iu.edu mkdir /tmp/VERSION/
scp *VERSION*gz jump.grid.iu.edu:/tmp/VERSION/
scp -r /tmp/VERSION repo1:/tmp/
scp -r /tmp/VERSION repo2:/tmp/
rm -rf /tmp/VERSION
On repo1/repo2 (as root)
You can ssh to repo1 and repo2 from jump.grid.iu.edu; you will need to do this procedure on both systems.
sudo su -
mv /tmp/VERSION /usr/local/repo/tarball-install/VERSION/
rm -f /usr/local/repo/tarball-install/VERSION/*latest*
ls -l /usr/local/repo/tarball-install/VERSION/*latest* # verify the symlinks are correct
Step 3: Install the tarballs into OASIS
You must be an OASIS manager of the 'mis' VO to do these steps.
Known managers as of 2014-07-22: Mat, Tim C, Tim T, Brian L.
Get the uploader script from SVN and run it with osgrun from the UW AFS install of the tarball client you made earlier.
On a UW CSL machine:
svn cat file:///p/vdt/workspace/svn/software/tarball-client/trunk/upload-tarballs-to-oasis > /tmp/upload-tarballs-to-oasis
chmod +x /tmp/upload-tarballs-to-oasis
/p/vdt/workspace/tarball-client/current/sys/osgrun /tmp/upload-tarballs-to-oasis VERSION
The script will automatically ssh you to oasis-login.opensciencegrid.org and give you instructions to complete the process.
Step 4: Announce the release
The following instructions are meant for the release manager (or interim release manager). If you are not the release manager, let the release manager know that they can announce the release.
- The release manager writes the release announcement and send it out. Here is a sample:
Subject: Announcing OSG Software version VERSION
- The release manager emails the announcement to
- The release manager asks the GOC to distribute the announcement by opening a ticket
- The release manager closes the tickets marked 'Ready for Release' in the release's JIRA filter
- The release manager updates the recent/scheduled release tables on the Software/Release homepage