Improve the bob release instructions

bob/devtools/scripts/release.py

@@ -143,12 +143,6 @@ def release(changelog, group, package, resume, dry_run):
 
     gl = get_gitlab_instance()
 
-    # if we are releasing 'bob' metapackage, it's a simple thing, no GitLab
-    # API
-    if package == "bob":
-        release_bob(changelog)
-        return
-
     # traverse all packages in the changelog, edit older tags with updated
     # comments, tag them with a suggested version, then try to release, and
     # wait until done to proceed to the next package

bob/devtools/scripts/update_bob.py

@@ -12,21 +12,23 @@ logger = get_logger(__name__)
 
 
 @click.command(
-    epilog="""
+    epilog="""\b
 Examples:
-
-bdt gitlab update-bob -vv
+    bdt gitlab update-bob -vv
+    bdt gitlab update-bob -vv --stable
 """
 )
-@ref_option()
+@click.option(
+    "--stable/--beta", help="To use the stable versions in the list and pin packages."
+)
 @verbosity_option()
 @bdt.raise_on_error
-def update_bob(ref):
+def update_bob(stable):
     """Updates the Bob meta package with new packages.
     """
     import tempfile
     from ..ci import read_packages
-    from ..release import get_gitlab_instance, download_path
+    from ..release import get_gitlab_instance, download_path, get_latest_tag_name
 
     gl = get_gitlab_instance()
 
@@ -34,7 +36,7 @@ def update_bob(ref):
     nightlies = gl.projects.get("bob/bob.nightlies")
 
     with tempfile.NamedTemporaryFile() as f:
-        download_path(nightlies, "order.txt", f.name, ref=ref)
+        download_path(nightlies, "order.txt", f.name, ref="master")
         packages = read_packages(f.name)
 
     # find the list of public packages
@@ -60,13 +62,22 @@ def update_bob(ref):
         "The following packages were not public:\n%s", "\n".join(private_packages)
     )
 
-    # modify conda/meta.yaml and requirements.txt in bob/bob
+    # if requires stable versions, add latest tag versions to the names
+    if stable:
+        logger.info("Getting latest tag names for the public packages")
+        tags = [
+            get_latest_tag_name(gl.projects.get(f"bob/{pkg}"))
+            for pkg in public_packages
+        ]
+        public_packages = [f"{pkg} =={tag}" for pkg, tag in zip(public_packages, tags)]
 
+
+    # modify conda/meta.yaml and requirements.txt in bob/bob
     logger.info("Updating conda/meta.yaml")
     start_tag = "# LIST OF BOB PACKAGES - START"
     end_tag = "# LIST OF BOB PACKAGES - END"
 
-    with open("conda/meta.yaml", "r+") as f:
+    with open("conda/meta.yaml") as f:
         lines = f.read()
         i1 = lines.find(start_tag) + len(start_tag)
         i2 = lines.find(end_tag)
@@ -75,7 +86,7 @@ def update_bob(ref):
             lines[:i1] + "\n    - ".join([""] + public_packages) + "\n    " + lines[i2:]
         )
 
-        f.seek(0)
+    with open("conda/meta.yaml", "w") as f:
         f.write(lines)
 
     logger.info("Updating requirements.txt")

doc/release.rst

@@ -69,67 +69,69 @@ To manually update the changelog, follow these guidelines:
 Releasing the Bob meta package
 ==============================
 
-.. todo:: These instructions may be outdated!!
-
 Here are the instructions to release Bob meta package:
 
 * Run:
 
   .. code-block:: sh
 
-     $ bdt gitlab getpath bob/bob.nightlies order.txt
-     $ bdt gitlab visibility order.txt
+     $ cd bob
+     $ bdt gitlab update-bob -vvv --stable
 
-* Put the list of public packages in ../../bob/requirements.txt
-* Run ``bdt gitlab changelog`` first:
+* The script above cannot identify linux only packages. After running the script,
+  **you need to manually tag linux only packages** in both ``conda/meta.yaml`` and
+  ``requirements.txt``. For example, in ``conda/meta.yaml``::
 
-  .. code-block:: sh
+  .. code-block:: yaml
 
-     $ bdt gitlab changelog ../../bob/requirements.txt bob_changelog.md
+    - bob.ip.binseg ==1.1.0  # [linux]
 
-* Put the beta of version of the intended release version in
-  ``../../bob/version.txt``
+  and, in ``requirements.txt``::
 
-  * For example do ``$ echo 5.0.0b0 > version.txt`` for bob 5.0.0 release.
-  * Commit only this change to master: ``$ git commit -m "prepare for bob 5 release" version.txt``
+    bob.ip.binseg ==1.1.0 ; sys_platform == 'linux'
 
-* Get the pinnings (``--bob-version`` needs to be changed):
+* Test the conda recipe of bob. You may want to cancel the
+  command below once it reaches the nosetests.:
 
   .. code-block:: sh
 
-     $ bdt gitlab release -p bob -c bob_changelog.md --bob-version 5.0.0 -- TOKEN
+     $ bdt build -vv --stable
 
-* Put the pinnings below in requirements.txt and meta.yaml (like ``bob.buildout
-  == 2.1.6``) and meta.yaml (like ``bob.buildout 2.1.6``)
+* Commit the changes and push:
 
-  * Make sure you add ``  # [linux]`` to Linux only packages.
+  .. code-block:: sh
+
+     $ git commit -m "Pinning packages for the next release. [skip ci]" conda/meta.yaml requirements.txt
+     $ git push
 
-* Test the conda recipe:
+
+* Tag the package using the same changelog mechanism that you used to tag other
+  packages. Assuming the changelog has a ``* bob/bob`` entry:
 
   .. code-block:: sh
 
-     $ cd ../../bob
-     $ conda render -m ../bob.admin/gitlab/conda_build_config.yaml -c https://www.idiap.ch/software/bob/conda conda
+     $ bdt gitlab release -vvv CHANGELOG --package bob/bob
+
+* When the script says ``Waiting for the pipeline *** of "bob/bob" to finish``, cancel
+  it. You can check the progress online.
 
-* Update the badges and version.txt to point to this version of Bob.
-* Commit, push and tag a new version manually:
+* To revert the pins while in beta run::
 
   .. code-block:: sh
 
-     $ git commit -am "Increased stable version to 4.0.0"
-     $ git tag v4.0.0
-     $ git push
-     $ git push --tags
+     $ git pull --rebase
+     $ bdt gitlab update-bob -vvv --beta
 
-* Put ``bob_changelog.md`` inside bob's tag description.
-* Cancel the pipeline for master and make sure that tag pipeline passes before
-  continuing.
-* Remove pinnings from bob's requirement.txt and meta.yaml and revert changes
-  that went in ``README.rst`` back to master version.
-* Commit and push the following (not verbatim):
+* Like before, **tag the linux only packages manually**.
+
+* Commit and push the changes:
 
   .. code-block:: sh
 
-     $ echo 4.0.1b0 > version.txt
-     $ git commit -am "Increased latest version to 4.0.1b0 [skip ci]"
+     $ git commit -m "Remove package pins while in beta. [skip ci]" conda/meta.yaml requirements.txt
      $ git push
+
+You can see that if we could identify linux only packages automatically, the whole
+release process would have been only to run
+``bdt gitlab release -vvv CHANGELOG --package bob/bob``.
+Do you want to help fix that?