How to Cut a Release

Due to the length of time that this process takes, it is recommended to do the release over two days to allow for errors to be corrected and because some of the koji commands take a long time to run when a large number of rpms are being released.

This document doesn't discuss the policy for deciding what goes into a release. The release policy is discussed elsewhere.

NOTE Interactive scripts can be found at https://vdt.cs.wisc.edu/svn/software/release-tools/ and can be checked out with SVN.

Pick the version number

This page will help you get the various commands exactly right. To do this, enter the exact version number (such as 3.0.5) and click submit. If you do not do this, the commands below will be missing essential text.

Version: Upcoming:

Day 0

Step 1: Update the osg-version RPM

Edit the osg-version RPM's spec file to update the version number to VERSION. Build it in koji:

# If building for the latest release out of trunk:
osg-build koji osg-version
# If building for an older release out of a branch:
osg-build koji --repo=VERSION osg-version

Step 2: Generate the tentative release list

The release manager often needs a preliminary list of packages slated for the next release a few weeks in advance. This is done by finding the package differences between what's in testing and what's in the current release. There is a script in SVN that handles this:

./0-generate-pkg-list VERSION

Day 1: Populate Pre-release and Testing

Step 1: Create four clean VMs per OSG version

Make four VMs for each OSG version (upcoming, 3.1, 3.2, etc.), probably in FermiCloud, so you can do testing. Two of them will be used to test fresh installations on a clean machine (one EL5, one EL6), while the other two will be used to test an update from the current release (one EL5, one EL6).

Step 2: Populate pre-release

Populate the pre-release. This requires you to understand which packages should be in the release. This is a question of policy. If you do not know which packages should be in the release, talk to the OSG Release Coordinator.

./1-populate-prerelease VERSION

There shouldn't be any packages in prerelease and you will prompted to untag any packages that are still in there before continuing. Afterwards, you will need to manually edit the package list to make sure the proper packages get promoted (consult the Release Manager for details).

Edit pkgs-to-release-* to remove packages not intended for this release. Please verify that the osg-version RPM is in your set of packages for the release! Also verify that if there is a new version of the osg-tested-internal RPM, then it is included in the release as well!

Step 3: Test fresh installs of VERSION

You will do a fresh installation. Note: you should do this for each distribution. The osg-test command below will enable the osg-prerelease repository, and the osg repository will already be enabled. So packages will be taken from the osg repository unless there is a newer version in the osg-prerelease repository. Note, due to some oddities in the tests, you'll need to be in a directory that the vdttest user has access to before running osg-test. /tmp should suffice for this. The bestman tests will break if you do not do this.

wget --quiet http://vdt.cs.wisc.edu/native/bootstrap-osg-test
chmod 0755 bootstrap-osg-test
./bootstrap-osg-test VERSION testing
cd /tmp
osg-test -var osg-prerelease -i osg-tested-internal

All tests should pass. If any tests fail, discuss with the OSG Release Coordinator.

Step 4: Test updates of VERSION

Now you will do an installation just from the osg repository, then update it from the osg-prerelease repository. Note: you should do this for each distribution. This is an important double-check, because we want to know that updates work smoothly. Note, due to some oddities in the tests, you'll need to be in a directory that the vdttest user has access to before running osg-test. /tmp should suffice for this. The bestman tests will break if you do not do this.

wget --quiet http://vdt.cs.wisc.edu/native/bootstrap-osg-test
chmod 0755 bootstrap-osg-test
./bootstrap-osg-test VERSION testing
cd /tmp
osg-test -va -i osg-tested-internal -g osg-prerelease

All tests should pass. If any tests fail, discuss with the OSG Release Coordinator.

Step 5: Create the client tarballs

Create the client tarballs using the scripts in software/tarball-client/trunk from our SVN, and then copy them into the UW's AFS. Do this as root on an EL6 fermicloud machine. Change the username and machine name as appropriate. Note: the prerelease repos must have been populated and regenerated for this to work; normally that happens in Step 2.

ruser=matyas
rmachine=library.cs.wisc.edu
mkdir -p /tmp/tarball-client
cd /tmp/tarball-client
svn export https://vdt.cs.wisc.edu/svn/software/tarball-client/trunk
trunk/make-client-tarball --osgver=VERSION --prerelease --all
scp *.tar.gz $ruser@$rmachine:/p/vdt/public/html/tarball-client/
You should get 8 tarballs, 20-45 megs each. They should all have VERSION in the name.

Step 6: Briefly test the client tarballs

As an unprivileged user, extract each tarball into a separate directory. Make sure osg-post-install works. Make sure 'osgrun osg-version' works.

for client in osg-client osg-wn-client; do
    for rhel in el5 el6; do
        for arch in i386 x86_64; do
            file=$client-VERSION-1.$rhel.$arch.tar.gz
            mkdir $rhel-$arch
            pushd $rhel-$arch
            tar xzf ../$file
            $client/osg/osg-post-install
            $client/osgrun osg-version
            popd
        done
    done
done

If you have time, try some of the binaries, such as grid-proxy-init.

NOTE: We need to automate this and have it run on the proper architectures and version of RHEL.

Step 7: Update the UW AFS installation of the tarball client

The UW keeps an install of the tarball client in /p/vdt/workspace/tarball-client on the UW's AFS. There is a script to update it called afs-install-tarball-client. Run it as follows:

/p/vdt/workspace/tarball-client/afs-install-tarball-client VERSION

Step 8: Wait

Wait for clearance. The OSG Release Coordinator (in consultation with the Software Team and any testers) need to sign off on the update before it is released. If you are releasing things over two days, this is a good place to stop for the day.


Day 2: Pushing the Release

NOTE: For the second phase of the release, try to complete it earlier in the day rather than later. The GOC would like to send out the release announcement prior to 3 p.m. their time (Eastern time zone), which means that our procedure needs to be completed before then.

Step 1: Push from pre-release to release

This script moves the packages into release, clones release into a new VERSION specific release repo, locks the new repo and regenerates it. Afterwards, it produces *release-note* files that should be given to whomever's writing the release notes.

./2-create-release VERSION

*.txt files are also created and it should be verified that they've been moved to /p/vdt/public/html/release-info/ on UW's AFS.

Step 2: Upload the client tarballs

Ask Mat or someone with privileges on the grid.iu.edu repo servers to upload the tarballs with the following procedure:

On a CS machine

cd /p/vdt/public/html/tarball-client
ssh jump.grid.iu.edu mkdir /tmp/VERSION/
scp *VERSION*gz jump.grid.iu.edu:/tmp/VERSION/

On jump.grid.iu.edu

scp -r /tmp/VERSION repo1:/tmp/
scp -r /tmp/VERSION repo2:/tmp/
rm -rf /tmp/VERSION

On repo1/repo2 (as root)

You can ssh to repo1 and repo2 from jump.grid.iu.edu; you will need to do this procedure on both systems.
sudo su -
mv /tmp/VERSION /usr/local/repo/tarball-install/VERSION/
rm -f /usr/local/repo/tarball-install/VERSION/*latest*
/root/mk-sims.sh
ls -l /usr/local/repo/tarball-install/VERSION/*latest* # verify the symlinks are correct

Step 3: Install the tarballs into OASIS

You must be an OASIS manager of the 'mis' VO to do these steps. Known managers as of 2013-08-13: Mat, Tim C, Tim T. Get the uploader script from SVN and run it with osgrun from the UW AFS install of the tarball client you made earlier. On a UW CSL machine:

svn cat file:///p/vdt/workspace/svn/software/tarball-client/trunk/upload-tarballs-to-oasis > /tmp/upload-tarballs-to-oasis
chmod +x /tmp/upload-tarballs-to-oasis
/p/vdt/workspace/tarball-client/current/sys/osgrun /tmp/upload-tarballs-to-oasis VERSION
The script will automatically ssh you to oasis-login.opensciencegrid.org and give you instructions to complete the process.

Step 4: Announce the release

  1. The release manager writes the release announcement and send it out. Here is a sample:

    Subject: Announcing OSG Software version VERSION

    We are pleased to announce OSG Software version VERSION!

    This is the new OSG Software distributed via RPMs for:

    * Scientific Linux 5 and 6
    * CentOS 5 and 6
    * Red Hat Enterprise Linux 5 and 6

    This release affects the SET-OF-METAPACKAGES (client, compute element, etc...). Changes include:

    * Major change 1
    * Major change 2
    * Major change 3

    Release notes and pointers to more documentation can be found at:

    https://www.opensciencegrid.org/bin/view/Documentation/Release3/ReleaseVERSION

    Need help? Let us know:

    https://www.opensciencegrid.org/bin/view/Documentation/Release3/HelpProcedure

    We welcome feedback on this release!

  2. The release manager emails the announcement to vdt-discuss@opensciencegrid.org
  3. The release manager asks the GOC to distribute the announcement by opening a ticket

Topic revision: r73 - 06 Mar 2014 - 17:21:41 - MatyasSelmeci
Hello, TWikiGuest
Register

 
TWIKI.NET

TWiki | Report Bugs | Privacy Policy

This site is powered by the TWiki collaboration platformCopyright by the contributing authors. All material on this collaboration platform is the property of the contributing authors..