How to Cut a Release

This document details the process for releasing new OSG Release version(s). This document does NOT discuss the policy for deciding what goes into a release, which can be found here.

Due to the length of time that this process takes, it is recommended to do the release over three or more days to allow for errors to be corrected and tests to be run.

Requirements

  • User certificate registered with OSG's koji with build and release team privileges
  • An account on UW CS machines (e.g. library, ingwe) to access UW's AFS
  • Convenience scripts from GitHub
  • osg-build scripts in your PATH (installed via OSG yum repos or source)

Pick the Version Number

Enter the verions numbers for each OSG version to be released (e.g. 3.3.12 3.2.38 upcoming) and click submit to

Versions:

Day 0: Generate Preliminary Release List

The release manager often needs a tentative list of packages to be released. This is done by finding the package differences between osg-testing and the current release.

Step 1: Update the osg-version RPM

For each release (excluding upcoming), update the version number in the osg-version RPM's spec file and build it in koji:

# If building for the latest release out of trunk:
osg-build koji osg-version
# If building for an older release out of a branch:
osg-build koji --repo=MAJOR_VERSION osg-version

Where MAJOR_VERSION is of the format x.y (e.g. 3.2).

Step 2: Promote osg-version and generate the release list

Run 0-generate-pkg-list from a machine that has your koji-registered user certificate:

[user@client ~]$ git clone https://github.com/opensciencegrid/release-tools.git
[user@client ~]$ cd release-tools
[user@client ~]$ 0-generate-pkg-list VERSIONS

Day 1: Verify Pre-Release and Generate Tarballs

This section is to be performed 1-2 days before the release (as designated by the release manager) to perform last checks of the release and create the client tarballs.

Step 1: Verify Pre-Release

Compare the list of packages already in pre-release to the final list for the release put together by the OSG Release Coordinator (who should have updated release-list in git). To do this, run the 1-verify-prerelease script from git:

[user@client ~]$ 1-verify-prerelease VERSIONS

If there are any discrepancies consult the release manager. You may have to tag or untag packages with the osg-koji tool.

NOTE: Please verify that the osg-version RPM is in your set of packages for the release! Also verify that if there is a new version of the osg-tested-internal RPM, then it is included in the release as well!

Step 2: Test Pre-Release in VM Universe

To test pre-release, you will be kicking off a manual VM universe test run from osghost.chtc.wisc.edu.

  1. Ensure that you meet the pre-requisites for submitting VM universe test runs
  2. Prepare the test suite by running:
    osg-run-tests 'Testing OSG pre-release'
    
  3. cd into the directory specified in the output of the previous command
  4. cd into parameters.d and remove all files within it except for osg32.yaml, osg33-el6.yaml, and osg33-el7.yaml
  5. Edit osg32.yaml so that it reads:
    platform:
      - centos_5_x86_64
      - centos_6_x86_64
      - rhel_5_x86_64
      - rhel_6_x86_64
      - sl_5_x86_64
      - sl_6_x86_64
    
    sources:
      - opensciencegrid:master; 3.2; osg-prerelease
      - opensciencegrid:master; 3.2; osg > osg-prerelease
      - opensciencegrid:master; 3.1; osg > 3.2/osg-prerelease
    
    packages:
      - All: [java-1.7.0-openjdk-devel, osg-java7-compat, osg-java7-devel-compat, osg-tested-internal]
    
  6. Edit osg33-el6.yaml so that it reads:
    platform:
      - centos_6_x86_64
      - rhel_6_x86_64
      - sl_6_x86_64
    
    sources:
      - opensciencegrid:master; 3.3; osg-prerelease
      - opensciencegrid:master; 3.3; osg > osg-prerelease
      - opensciencegrid:master; 3.2; osg > 3.3/osg-prerelease
      - opensciencegrid:master; 3.1; osg > 3.3/osg-prerelease
     
    
    packages:
      - All: [java-1.7.0-openjdk-devel, osg-java7-compat, osg-java7-devel-compat, osg-tested-internal]
      - All + GRAM: [java-1.7.0-openjdk-devel, osg-java7-compat, osg-java7-devel-compat, osg-tested-internal,
                     globus-gatekeeper, globus-gram-client-tools, globus-gram-job-manager, globus-gram-job-manager-fork,
                     globus-gram-job-manager-fork-setup-poll, gratia-probe-gram, globus-gram-job-manager-scripts,
                     globus-gram-job-manager-condor, globus-gram-job-manager-pbs-setup-seg]
    
  7. Edit osg33-el7.yaml so that it reads:
    platform:
      - centos_7_x86_64
      - rhel_7_x86_64
      - sl_7_x86_64
    
    sources:
      - opensciencegrid:master; 3.3; osg-prerelease
      - opensciencegrid:master; 3.3; osg > osg-prerelease
     
    
    packages:
      - All: [java-1.7.0-openjdk-devel, osg-java7-compat, osg-java7-devel-compat, osg-tested-internal]
      - All + GRAM: [java-1.7.0-openjdk-devel, osg-java7-compat, osg-java7-devel-compat, osg-tested-internal,
                     globus-gatekeeper, globus-gram-client-tools, globus-gram-job-manager, globus-gram-job-manager-fork,
                     globus-gram-job-manager-fork-setup-poll, gratia-probe-gram, globus-gram-job-manager-scripts,
                     globus-gram-job-manager-condor, globus-gram-job-manager-pbs-setup-seg]
    
  8. cd back into the root directory of the test run (e.g. cd ..)
  9. Submit the DAG:
    condor_submit_dag master-run.dag
    

NOTE: If there are failures, consult the release-manager before proceeding.

Step 3: Regenerate the build repositories

To avoid 404 errors when retrieving packages, it's necessary to regenerate the build repositories. Run the following script from a machine with your koji-registered user certificate:

[user@client ~]$ 1-regen-repos VERSIONS

Step 4: Create the client tarballs

Create the client tarballs as root on an EL6 fermicloud machine using the relevant script from git:

[root@client ~]$ git clone https://github.com/opensciencegrid/release-tools.git
[root@client ~]$ cd release-tools
[root@client ~]$ 1-client-tarballs VERSIONS
You should get up to 8 tarballs, 25-55 megs each. They should all have VERSIONS in the name.

Step 5: Briefly test the client tarballs

As an unprivileged user, extract each tarball into a separate directory. Make sure osg-post-install works. Make sure 'osgrun osg-version' works by running the following tests, replacing highlighted text as necessary:

If you have time, try some of the binaries, such as grid-proxy-init.

NOTE: We need to automate this and have it run on the proper architectures and version of RHEL.

Step 6: Update the UW AFS installation of the tarball client

The UW keeps an install of the tarball client in /p/vdt/workspace/tarball-client on the UW's AFS. To update it, run the following commands:

for ver in VERSIONS; do
    /p/vdt/workspace/tarball-client/afs-install-tarball-client $ver
done

Step 7: Wait

Wait for clearance. The OSG Release Coordinator (in consultation with the Software Team and any testers) need to sign off on the update before it is released. If you are releasing things over two days, this is a good place to stop for the day.

Day 2: Pushing the Release

NOTE: For the second phase of the release, try to complete it earlier in the day rather than later. The GOC would like to send out the release announcement prior to 3 p.m. Eastern time.

Step 1: Push from pre-release to release

This script moves the packages into release, clones releases into new version-specific release repos, locks the repos and regenerates them. Afterwards, it produces *release-note* files that should be used to update the TWiki release note pages. Clone it from the github repo and run the script:

[user@client ~]$ 2-create-release VERSIONS

*.txt files are also created and it should be verified that they've been moved to /p/vdt/public/html/release-info/ on UW's AFS.

Step 2: Get the new CA certificates pushed to software.grid.iu.edu

If this release contains a new version of osg-ca-certs or igtf-ca-certs, submit a GOC ticket to have them propagate the CA certificates from the recently updated release repositories to software.grid.iu.edu and assign this ticket to Scott Teige:

Please run /opt/bin/install-latest-cache-files-new.sh on software1/2

Once the GOC has run their script, run the following command to update the CA certificates in the tarball installations and verify that the version of the CA certificates match the version that was promoted to release.

[user@client ~]$ /p/vdt/workspace/tarball-client/current/amd64_rhel6/osgrun osg-update-certs
[user@client ~]$ /p/vdt/workspace/tarball-client/current/amd64_rhel7/osgrun osg-update-certs

Step 3: Upload the client tarballs

Ask Tim Theisen, Brian Lin, or someone with privileges on the grid.iu.edu repo servers to upload the tarballs with the following procedure:

On a CS machine

for ver in VERSIONS; do
    major_ver=`sed 's/.[0-9]*$//' <<< $ver`
    cd /p/vdt/public/html/tarball-client
    ssh jump.grid.iu.edu mkdir /tmp/$ver/
    scp -p $major_ver/*/*$ver*gz jump.grid.iu.edu:/tmp/$ver/
done

On jump.grid.iu.edu

for ver in VERSIONS; do
    rm -f /tmp/$ver/*-afs-*
    scp -pr /tmp/$ver repo1:/tmp/
    scp -pr /tmp/$ver repo2:/tmp/
    rm -rf /tmp/$ver
done

On repo1/repo2 (as root)

You can ssh to repo1 and repo2 from jump.grid.iu.edu; you will need to do this procedure on both systems.
sudo su -
for ver in VERSIONS; do
    major_ver=`sed 's/.[0-9]*$//' <<< $ver`
    mv /tmp/$ver /usr/local/repo/tarball-install/$major_ver/
    rm -f /usr/local/repo/tarball-install/$major_ver/*latest*
done
/root/mk-sims.sh
for ver in VERSIONS; do
    major_ver=`sed 's/.[0-9]*$//' <<< $ver`
    ls -l /usr/local/repo/tarball-install/$major_ver/*latest* # verify the symlinks are correct
done

Step 4: Install the tarballs into OASIS

You must be an OASIS manager of the mis VO to do these steps. Known managers as of 2014-07-22: Mat, Tim C, Tim T, Brian L. Get the uploader script from Git and run it with osgrun from the UW AFS install of the tarball client you made earlier. On a UW CSL machine:

cd /tmp
git clone --depth 1 file:///p/vdt/workspace/git/repo/tarball-client.git
for ver in VERSIONS; do
    /p/vdt/workspace/tarball-client/current/sys/osgrun /tmp/tarball-client/upload-tarballs-to-oasis $ver
done
The script will automatically ssh you to oasis-login.opensciencegrid.org and give you instructions to complete the process.

Step 5: Remove old UW AFS installations of the tarball client

To keep space usage down, remove tarball client installations and symlinks under /p/vdt/workspace/tarball-client on UW's AFS that are more than 2 months old. The following command can list them:

find  /p/vdt/workspace/tarball-client  -maxdepth 1  -mtime +60  -name 3\*  -ls

Step 6: Announce the release

The following instructions are meant for the release manager (or interim release manager). If you are not the release manager, let the release manager know that they can announce the release.

  1. The release manager writes the release announcement and send it out. Here is a sample, replace highlighted text with the appropriate values:
    Subject: Announcing OSG Software version <VERSION(S)>
    We are pleased to announce OSG Software version <VERSION(S)>!
    
    This is the new OSG Software distributed via RPMs for:
    
    * Scientific Linux 5 and 6
    * !CentOS 5 and 6
    * Red Hat Enterprise Linux 5 and 6
    
    This release affects the <SET-OF-METAPACKAGES (client, compute element, etc...)>. Changes include:
    
    * Major change 1
    * Major change 2
    * Major change 3
    
    Release notes and pointers to more documentation can be found at:
    
    https://www.opensciencegrid.org/bin/view/Documentation/Release3/Release%SHORTVERSION%
    
    Need help? Let us know:
    
    https://www.opensciencegrid.org/bin/view/Documentation/Release3/HelpProcedure
    
    We welcome feedback on this release!
  2. The release manager emails the announcement to vdt-discuss@opensciencegrid.org
  3. The release manager asks the GOC to distribute the announcement by opening a ticket
  4. The release manager closes the tickets marked 'Ready for Release' in the release's JIRA filter
  5. The release manager updates the recent/scheduled release tables on the Software/Release homepage

Topic revision: r126 - 12 Jul 2016 - 18:34:51 - BrianLin
Hello, TWikiGuest
Register

 
TWIKI.NET

TWiki | Report Bugs | Privacy Policy

This site is powered by the TWiki collaboration platformCopyright by the contributing authors. All material on this collaboration platform is the property of the contributing authors..