Release Process

Note

Make sure to follow the latest documentation when doing a release.

Outcomes

By the end of the release process we will have:

Prerequisites

Software

Access

Preparing For a Release

The following steps should be carried out on a Flocker development machine. Log into the machine using SSH agent forwarding so that you can push changes to GitHub using the keys from your workstation.

vagrant ssh -- -A
  1. Choose a version number according to the Flocker version numbering policy.

  2. Export the version number as an environment variable for later use:

    export VERSION=0.1.2
    
  3. Create an issue:

    1. Set the title to “Release flocker $VERSION”
    2. Assign it to yourself
  4. Create a clean, local working copy of Flocker with no modifications:

    git clone [email protected]:ClusterHQ/flocker.git "flocker-${VERSION}"
    
  5. Create a branch for the release and push it to GitHub:

    git checkout -b release/flocker-${VERSION} origin/master
    git push origin --set-upstream release/flocker-${VERSION}
    
  6. Check that all required versions of the dependency packages are built:

    1. Inspect the package versions listed in the install_requires section of setup.py.
    2. Compare it to the package versions listed in python-flocker.spec.in.
    3. If there are any mismatches, change python-flocker.spec.in appropriately, commit the changes, and update the archive.clusterhq.com repository by following the steps in Appendix: Pre-populating RPM Repository, adding any missing package names to the lists of downloaded packages.

    Note

    XXX: Automate the checking of package versions. See https://github.com/ClusterHQ/flocker/issues/881.

  7. Back port features from master (optional)

    The release may require certain changes to be back ported from the master branch. See Appendix: Back Porting Changes From Master.

  8. Update the version numbers in:

  9. Ensure the release notes in NEWS are up-to-date:

    XXX: Process to be decided. See https://github.com/ClusterHQ/flocker/issues/523

    git commit -am "Updated NEWS"
    
  10. Ensure copyright dates in LICENSE are up-to-date:

    XXX: Process to be decided. See https://github.com/ClusterHQ/flocker/issues/525

    git commit -am "Updated copyright"
    
  11. Push the changes:

    git push
    
  12. Ensure all the tests pass on BuildBot:

    Go to the BuildBot web status and force a build on the just-created branch.

  13. Make a pull request on GitHub

    The pull request should be for the release branch against master, with a Fixes #123 line in the description referring to the release issue that it resolves.

    Wait for an accepted code review before continuing.

    Warning

    Add a note to the pull request description explaining that the branch should not be merged until the release process is complete.

Review Process

  1. Do the acceptance tests:

    XXX: See https://github.com/ClusterHQ/flocker/issues/315

    You’ll need to build a tutorial vagrant image using the BuildBot RPM packages from the release branch.

    The RPM version will not yet correspond to the release version, because we haven’t yet created a tag.

    To find the version, visit the BuildBot build results page and navigate to the flocker-rpms build, then click on stdio from the build-sdist step.

    At the top, you should find a line beginning got version which contains the version string.

    Then run the tutorial image build script as follows, substituting the --branch and --flocker-version values:

    vagrant/tutorial/build --flocker-version=0.2.1-378-gb59b886 --branch=release/flocker-0.3.0dev1
    

    Then add the resulting box to vagrant using the following command:

    vagrant box add --name='clusterhq/flocker-tutorial'  flocker-tutorial-0.2.1-378-gb59b886.box
    

    You should now see that box listed:

    $ vagrant box list
    clusterhq/fedora20-updated (virtualbox, 2014.09.19)
    clusterhq/flocker-dev      (virtualbox, 0.2.1.263.g572d20f)
    clusterhq/flocker-tutorial (virtualbox, 0)
    

    Finally follow the BuildBot built tutorial documentation from the release branch, but modify the config.vm.box_version line in docs/gettingstarted/tutorial/Vagrantfile version to 0 before running vagrant up.

    $ cat flocker-tutorial/Vagrantfile
    ...
    Vagrant.configure(VAGRANTFILE_API_VERSION) do |config|
        config.vm.box = "clusterhq/flocker-tutorial"
        config.vm.box_version = "= 0"
    ...
    

Note

It is the reviewer’s job to also review the Homebrew pull request which is created in the following release steps. See the “Update the Homebrew recipe” step below which explains how to test the new Homebrew recipe from a branch.

Warning

The branch should not be merged yet. It should only be merged once it has been tagged, in the next series of steps.

Release

Warning

The following steps should be carried out on a Flocker development machine. Log into the machine using SSH agent forwarding so that you can push changes to GitHub using the keys from your workstation.

vagrant ssh -- -A
  1. Change your working directory to be the Flocker release branch working directory.

  2. Create (if necessary) and activate the Flocker release virtual environment:

    Note

    The following instructions use virtualenvwrapper but you can use virtualenv directly if you prefer.

    mkvirtualenv flocker-release-${VERSION}
    pip install --editable .[release]
    
  3. Tag the version being released:

    git tag --annotate "${VERSION}" "release/flocker-${VERSION}" -m "Tag version ${VERSION}"
    git push origin "${VERSION}"
    
  4. Go to the BuildBot web status and force a build on the tag.

    Force a build on a tag by putting the tag name (e.g. 0.2.0) into the branch box (without any prefix).

    Note

    We force a build on the tag as well as the branch because the RPMs built before pushing the tag won’t have the right version. Also, the RPM upload script currently expects the RPMs to be built from the tag, rather than the branch.

  5. Build python packages and upload them to archive.clusterhq.com

    python setup.py sdist bdist_wheel
    gsutil cp -a public-read \
        "dist/Flocker-${VERSION}.tar.gz" \
        "dist/Flocker-${VERSION}-py2-none-any.whl" \
        gs://archive.clusterhq.com/downloads/flocker/
    

    Note

    Set up gsutil authentication by following the instructions from the following command:

    $ gsutil config
    
  6. Build RPM packages and upload them to archive.clusterhq.com

    admin/upload-rpms "${VERSION}"
    
  7. Build and upload the tutorial vagrant box.

    Warning

    This step requires Vagrant and should be performed on your own workstation; not on a Flocker development machine.

  8. Update the Homebrew recipe

    The aim of this step is to provide a version specific Homebrew recipe for each release.

    • Checkout the homebrew-tap repository.

      git clone [email protected]:ClusterHQ/homebrew-tap.git
      
    • Create a release branch

      git checkout -b release/flocker-${VERSION} origin/master
      git push origin --set-upstream release/flocker-${VERSION}
      
    • Create a flocker-${VERSION}.rb file

      Copy the last recipe file and rename it for this release.

    • Update recipe file

      • Update the version number

        The version number is included in the class name with all dots and dashes removed. e.g. class Flocker012 < Formula for Flocker-0.1.2

      • Update the URL

        The version number is also included in the url part of the recipe.

      • Update the sha1 checksum.

        sha1sum "dist/Flocker-${VERSION}.tar.gz"
        ed03a154c2fdcd19eca471c0e22925cf0d3925fb  dist/Flocker-0.1.2.tar.gz
        
      • Commit the changes and push

        git commit -am "Bumped version number and checksum in Homebrew recipe"
        git push
        
    • Test the new recipe on OS X with Homebrew installed

      Try installing the new recipe directly from a GitHub link

      brew install https://raw.githubusercontent.com/ClusterHQ/homebrew-tap/release/flocker-${VERSION}/flocker-${VERSION}.rb
      brew test flocker-${VERSION}.rb
      
    • Make a pull request

      Make a homebrew-tap pull request for the release branch against master, with a Refs #123 line in the description referring to the release issue that it resolves.

      Include the brew install line from the previous step, so that the reviewer knows how to test the new recipe.

    • Do not continue until the pull request is merged.

      Otherwise the documentation will refer to an unavailable Homebrew recipe.

  9. Build tagged docs at Read the Docs:

    1. Force Read the Docs to reload the repository

      There is a GitHub webhook which should notify Read The Docs about changes in the Flocker repository, but it sometimes fails. Force an update by running:

      curl -X POST http://readthedocs.org/build/flocker
      
    2. Go to the Read the Docs dashboard Versions section.

    3. Enable the version being released.

    4. Wait for the documentation to build. The documentation will be visible at http://docs.clusterhq.com/en/${VERSION} when it has been built.

    5. Set the default version and latest version to that version:

      Warning

      Skip this step for weekly releases and pre-releases. The features and documentation in weekly releases and pre-releases may not be complete and may not have been tested. We want new users’ first experience with Flocker to be as smooth as possible so we direct them to the tutorial for the last stable release. Other users choose to try the weekly releases, by clicking on the latest weekly version in the ReadTheDocs version panel.

  10. Merge the release branch

    Merge release branch and close the release pull request.

Appendix: Back Porting Changes From Master

XXX: This process needs documenting. See https://github.com/ClusterHQ/flocker/issues/877

Appendix: Pre-populating RPM Repository

Warning

This only needs to be done if the dependency packages for Flocker (e.g. 3rd party Python libraries) change; it should not be done every release. If you do run this you need to do it before running the release process above as it removes the flocker-cli etc. packages from the repository!

These steps must be performed from a Flocker development environment because it has the HybridLogic Copr repository pre-installed.

mkdir repo
mkdir srpm

# Download all the latest binary and source packages from the Copr repository.
yumdownloader --disablerepo='*' --enablerepo=tomprince-hybridlogic --destdir=repo python-characteristic python-eliot python-idna python-netifaces python-service-identity python-treq python-twisted python-docker-py python-psutil
yumdownloader --disablerepo='*' --enablerepo=tomprince-hybridlogic --destdir=srpm --source python-characteristic python-eliot python-idna python-netifaces python-service-identity python-treq python-twisted python-docker-py python-psutil

# Create local repositories.
createrepo repo
createrepo srpm

# Upload to Google Cloud Storage using ``gsutil``.
gsutil cp -a public-read -R repo gs://archive.clusterhq.com/fedora/20/x86_6
gsutil cp -a public-read -R srpm gs://archive.clusterhq.com/fedora/20/SRPMS