Generate changelog
To generate a changelog that shows changes (tags and commits) in all packages since last release, please run the following script:
$ python generate_changelog.py <Gitlab Private Access Token> > changelog_since_last_release.txtThe script will go through all packages listed in https://gitlab.idiap.ch/bob/bob.nightlies/blob/master/order.txt file. The date of the latest tag of bob package, will be taken as the last release date. All changes including tags and commits will be logged for each package. The resulted log file should be manually updated, so the changes descriptions become more concise. Once updated, the changelog should be put in https://gitlab.idiap.ch/bob/bob/tags.
Manually update changelog
The changelog since the last release can be found in changelog_since_last_release.txt. The structure is the following:
* package name
* tag1 name (date of the tag).
* tag description. Each line of the tag description starts with `*` character.
- commits (from earliest to latest). Each line of the commit starts with `-` character.
* tag2 name (date of the tag).
* tag description. Each line of the tag description starts with `*` character.
- commits (from earliest to latest). Each line of the commit starts with `-` character.
* patch
- leftover not-tagged commits (from earliest to latest)To manually update the changelog, follow these steps:
1. For each tag, summarize the commits into several descriptive lines. These summaries become
tag descriptions and they should extend/update the existing tag description. Therefore, each line
of the summary should also start with `*` character like the tag descriptions.
2. The last tag name is called `patch`. This indicates that a patch version of this package will
be automatically updated during the next tag/release. You can change `patch` to `minor` or `major`,
and the package will be then tagged/released with either minor or major version bump.
3. Once all commits were changed to corresponding tag descriptions (no more lines start with `-` characters),
this package is ready for release and you can continue to another package in the changelog. Release Bob
Here are some instructions to do a massive Bob release and scripts to help you release Bob:
Create a buildout configuration file that checks out all packages from
bob.nightlies. Here is an example:
[buildout]
parts = scripts
eggs = bob.extension
bob.blitz
bob.core
...
bob # don't forget bob
...
bob.bio.video
bob.bio.vein
bob.db.voxforge
bob.pad.base
extensions = bob.buildout
mr.developer
auto-checkout = *
develop = src/bob.extension
src/bob.blitz
src/bob.core
...
src/bob.bio.video
src/bob.bio.vein
src/bob.db.voxforge
src/bob.pad.base
; options for bob.buildout extension
debug = false
verbose = true
newest = false
[sources]
bob.extension = git git@gitlab.idiap.ch:bob/bob.extension
bob.blitz = git git@gitlab.idiap.ch:bob/bob.blitz
bob.core = git git@gitlab.idiap.ch:bob/bob.core
...
bob.bio.video = git git@gitlab.idiap.ch:bob/bob.bio.video
bob.bio.vein = git git@gitlab.idiap.ch:bob/bob.bio.vein
bob.db.voxforge = git git@gitlab.idiap.ch:bob/bob.db.voxforge
bob.pad.base = git git@gitlab.idiap.ch:bob/bob.pad.base
[scripts]
recipe = bob.buildout:scriptsList and order of packages come from bob.nightlies; in core.txt and
extra.txt. Don't forget to add bob after core.txt.
Create a conda environment with bob-devel:
$ conda create -n bob-devel-27 python=2 bob-develYou should not install any other package and run:
$ buildout
$ bin/nosetests -sv src/*Make sure all the tests are passing.
Go through all packages and write down what has changed since their last
tag and write it down into a file (changelog.md for example). Here is an
example:
bob.extension (Minor)
* !37 !38 !39 !41 : Improved documentation for a better development guide
* !42 : Improve the new version script in terms of functionality and documentation
bob.blitz (Patch)
* Maintenance release.This is the most important part!!!! Do not write this changelog easily. You really have to dig through the history of all packages since the last release. Write down what has changed by not copying and pasting. Changelogs are read by humans and must be generated by humans.
PROPERLY identify if a package needs a patch, minor, or a major release bump. See http://semver.org for information. Ask the package's maintainer if you are not sure.
Then you can use release.sh and modify its release list depending on what you
have in your changelog.md. Some packages need a patch release and some need a
minor release and some a major release.
Hold off `bob` for now.
Don't release private packages!
Read bin/bob_new_version.py --helpRun that script. Release the first package. Wait for its CI to finish. Merge
its merge request on bob.conda. wait for that to finish in master of bob.conda.
Press y to go to the next package.
Finally read bob s readme and release that too.
You are done here no need to read the rest of this file.
You may want to disable the runners for bob.conda so that you can run it once everything is merged into master. To merge everything and cancel all pipelines in bob.conda. Read here: https://python-gitlab.readthedocs.io/en/stable/index.html
Then:
import gitlab
gl = gitlab.Gitlab.from_config()
bob_conda = gl.projects.search('bob.conda')[0]
# get all merge requests and merge them
mrs = bob_conda.mergerequests.list(all=True, state='opened')
print(mrs)
for mr in mrs:
try:
mr.merge()
except Exception:
print('failed for ', mr.name)
pass
# get all pipelines and merge them
pipelines = bob_conda.pipelines.list(all=True, scope='running')
print(len(pipelines))
for p in pipelines:
try:
p.cancel()
except Exception:
print('failed for ', p)
passThis will merge all merge requests. But sometimes the branch is created but there is no merge request for that. Merge those branches manually! and cancel the pipelines again.
Run the pipeline for master of bob.conda once and fix till every conda package is released.
Release bob.
import gitlab
import distutils.version
import re
gl = gitlab.Gitlab.from_config()
# bob's new requirementst.txt file:
path = '..../git/bob/requirements.txt'
bob_group = gl.groups.search('bob')[0]
def get_max_version(versions):
try:
v = list(reversed(sorted([distutils.version.StrictVersion(k)
for k in versions])))
final = [k for k in v if not k.prerelease]
if final:
return final[0]
return v[0]
except Exception:
v = list(reversed(sorted([distutils.version.LooseVersion(k)
for k in versions])))
final = [k for k in v if not re.search(r'[a-z]', k.vstring)]
if final:
return final[0]
return v[0]
pkgs = {}
for line in open(path):
pkg, version = line.split('==')
pkgs[pkg.strip()] = version.strip()
# verify bob is pointing to the latest tags of core packages.
for pkg, version in pkgs.items():
projects = bob_group.projects.search(pkg)
project = [p for p in projects if p.name == pkg][0]
tags = project.tags.list(all=True)
versions = [tag.name[1:] for tag in tags]
max_version = get_max_version(versions)
max_version = '{}.{}.{}'.format(*max_version.version)
if max_version != version:
print(pkg, versions)