Skip to content
Snippets Groups Projects
Commit fcefbb9b authored by André Anjos's avatar André Anjos :speech_balloon:
Browse files

Merge branch 'move-ci-install-to-bob-private' into 'master' 

[doc] Moves CI installation information to bob/private>

See merge request !331
parents 41c8ade3 d39769ba
No related branches found
No related tags found
1 merge request!331[doc] Moves CI installation information to bob/private>
Pipeline #65862 passed
Stage: qa
Stage: build
Stage: deploy
Hide whitespace changes
Inline Side-by-side
Showing
with 0 additions and 726 deletions
doc/ci.rst deleted 100644 → 0
+ 0
− 23
View file @ 41c8ade3
.. vim: set fileencoding=utf-8 :
.. _bob.devtools.ci:
==========================================
Instructions for CI machine installation
==========================================
This document contains instructions for CI machine installation from sources.
.. warning::
Idiap has throttling rules that are typically applied to all machines in the
lab network. To avoid issues for newly installed CI nodes, ensure you
request throttling to be disabled for new CI machines.
.. toctree::
:maxdepth: 2
linux
macos
doc/index.rst
+ 0
− 1
View file @ fcefbb9b
......@@ -20,7 +20,6 @@ Documentation
templates
release
api
ci
Indices and tables
......
doc/linux.rst deleted 100644 → 0
+ 0
− 260
View file @ 41c8ade3
.. vim: set fileencoding=utf-8 :
.. _bob.devtools.ci.linux:
============================
Deploying a Linux-based CI
============================
This document contains instructions to build and deploy a new bare-OS CI for
Linux. Instructions for deployment assume a freshly installed machine, with
Idiap's latest Debian distribution running. Our builds use Docker images. We
also configure docker-in-docker to enable to run docker builds (and other
tests) within docker images.
.. warning::
Idiap has throttling rules that are typically applied to all machines in the
lab network. To avoid issues for newly installed CI nodes, ensure you
request throttling to be disabled for new CI machines.
Docker and Gitlab-runner setup
------------------------------
Base docker installation:
https://secure.idiap.ch/intranet/system/software/docker
Ensure to add/configure for auto-loading the ``overlay`` kernel module in
``/etc/modules``. Then update/create ``/etc/docker/daemon.json`` to contain
the entry ``"storage-driver": "overlay2"``.
To ensure that you can control memory and CPU usage for launched docker
containers, you'll need to enable "cgroups" on your machine. In essence,
change ``/etc/default/grub`` to contain the line
``GRUB_CMDLINE_LINUX="cgroup_enable=memory swapaccount=1"``. Then, re-run
``update-grub`` after such change.
To install docker at Idiap, you also need to follow the security guidelines.
If you do not follow these guidelines, the machine will not be accessible from
outside via the login gateway, as the default docker installation conflicts with
Idiap's internal setup. You may also find other network connectivity issues.
Also, you want to place ``/var/lib/docker`` on a **fast** disk. Normally, we
have a scratch partition for this. Follow the instructions at
https://linuxconfig.org/how-to-move-docker-s-default-var-lib-docker-to-another-directory-on-ubuntu-debian-linux
for this step:
.. code-block:: sh
$ mkdir /scratch/docker
$ chmod g-rw,o-rw /scratch/docker
$ service docker stop
$ rsync -aqxP /var/lib/docker/ /scratch/docker
$ rm -rf /var/lib/docker
$ vim /etc/docker/daemon.json # add data-root -> /scratch/docker
$ service docker start
Reboot the machine to ensure everything works fine.
Hosts section
=============
We re-direct calls to www.idiap.ch to our internal server, for speed. Just add
this to `/etc/hosts`:
.. code-block:: sh
$ echo "" >> /etc/hosts
$ echo "#We fake www.idiap.ch to keep things internal" >> /etc/hosts
$ echo "What is the internal server IPv4 address?"
$ read ipv4add
$ echo "${ipv4add} www.idiap.ch" >> /etc/hosts
$ echo "What is the internal server IPv6 address?"
$ read ipv6add
$ echo "${ipv6add} www.idiap.ch" >> /etc/hosts
.. note::
You should obtain the values of the internal IPv4 and IPv6 addresses from
inside the Idiap network. We cannot replicate them in this manual for
security reasons.
Gitlab runner configuration
===========================
Once that is setup, install gitlab-runner from https://docs.gitlab.com/runner/install/linux-repository.html, and then register it https://docs.gitlab.com/runner/register/.
We are currently using this kind of configuration (notice you need to replace
the values of ``<internal.ipv4.address>`` and ``<token>`` on the template below):
.. code-block:: ini
concurrent = 20
check_interval = 10
[session_server]
session_timeout = 1800
[[runners]]
name = "<machine-name>"
output_limit = 102400
url = "https://gitlab.idiap.ch/"
token = "<token>"
executor = "shell"
shell = "bash"
builds_dir = "/scratch/builds"
cache_dir = "/scratch/cache"
[[runners]]
name = "bp-srv01"
output_limit = 102400
url = "https://gitlab.idiap.ch/"
token = "<token>"
executor = "docker"
builds_dir = "/scratch/builds"
cache_dir = "/scratch/cache"
[runners.docker]
tls_verify = false
image = "continuumio/conda-concourse-ci"
privileged = false
disable_entrypoint_overwrite = false
oom_kill_disable = false
disable_cache = false
volumes = ["/scratch/cache"]
shm_size = 0
extra_hosts = ["www.idiap.ch:<internal.ipv4.address>"]
[runners.cache]
Insecure = false
.. note::
You must make both ``/scratch/builds`` and ``/scratch/cache`` owned by the
user running the ``gitlab-runner`` process. Typically, it is
``gitlab-runner``. These commands, in this case, are in order to complete
the setup:
.. code-block:: sh
$ mkdir /scratch/builds
$ chown gitlab-runner:gitlab-runner /scratch/builds
$ mkdir /scratch/cache
$ chown gitlab-runner:gitlab-runner /scratch/cache
Once the configuration is done, add the gitlab-runner user to the docker group
so it can do tasks related to Docker (images pulling, python client call,
etc.):
.. code-block:: sh
$ usermod -a -G docker gitlab-runner
Access to Idiap's docker registry
=================================
If you want the Idiap docker registry (docker.idiap.ch) to be accessible from
the shell executors, you must also ensure Idiap registry certificates are
available on the host. You may copy the contents of ``docker.idiap.ch``
directory in this documentation set for that purpose, to the directory
``/etc/docker/certs.d``. Then, ensure to use something like: ``docker login -u
gitlab-ci-token -p $CI_JOB_TOKEN docker.idiap.ch`` on the (global)
``before_script`` phase in jobs requiring access to the registry.
Repository cloning from CI jobs
===============================
If you'd like to allow the (shell-based) runner to clone repositories other
than the one being built, you need to ensure the following is configured at
``~/.ssh/config`` of the user running the ``gitlab-runner`` process
(typically ``gitlab-runner``):
.. code-block:: text
Host gitlab.idiap.ch
ForwardX11 no
ForwardX11Trusted no
ForwardAgent yes
StrictHostKeyChecking no
ControlMaster auto
ControlPath /tmp/%r@%h-%p
ControlPersist 600
Compression yes
Make sure to use an "https" git-clone strategy in your recipes.
Git
===
The version of git (2.11) shipped with Debian Stretch (9.x) is broken. The
git-clean command does not honour the ``--exclude`` passed via the
command-line. I advise you install the most recent version from debian
backports by enabling this repository or configuring it with instructions from
https://backports.debian.org. To install the newest git version, after an
``apt update``, just run the following command as root:
.. code-block:: sh
$ apt-get -t stretch-backports install "git" "git-lfs"
X11
===
Some utilities such as ``dot`` (graphviz) require X11 support. If you intend
to make use of the ``shell`` builder and ``graphviz``, you must install basic
X11 support. Just run the following command as root to fix this:
.. code-block:: sh
$ apt install libxrender1 libxext6
Crontabs
========
.. code-block:: sh
# crontab -l
MAILTO=""
0 12 * * SUN /usr/share/gitlab-runner/clear-docker-cache
Conda and shared builds
=======================
To avoid problems with conda and using shared builders, consider creating the
directory ``~gitlab-runner/.conda`` and touching the file
``environments.txt`` in that directory, setting a mode of ``444`` (i.e., make
it read-only).
Extra packages
==============
List of extra packages to ensure are installed on the shell environment:
* rsync
* libgl1
libgl1 is required to run the beat/beat.editor> tests. While the offscreen
plugin is used and therefor no X11 server is required, the libraries still needs
the OpenGL symbols.
Locale
======
Ensure to set the default locale as ``C.UTF-8`` by re-running
``dpkg-reconfigure locales``. The click (python) package `requires it
<https://click.palletsprojects.com/en/7.x/python3/>`_.
doc/macos-ci-install/admin-install.sh deleted 100755 → 0
+ 0
− 37
View file @ 41c8ade3
#!/usr/bin/env bash
# Installs basic software on a fresh macOS installation that requires admin
# priviledges.
# VARIABLES - edit to your requirements
MACOS_VERSION="${1}"
USERNAME="${2}"
# --------------------
# Don't edit past this
# --------------------
# gets the current path leading to this script
DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" >/dev/null && pwd )"
echo "Setting up date and time..."
${DIR}/datetime.sh
echo "Updating the current system and disabling automatic updates..."
${DIR}/system-update.sh
echo "Installing Xcode Command-Line (CLI) tools..."
${DIR}/xcode-cli-tools.sh
echo "Installing macOS ${MACOS_VERSION} SDK..."
${DIR}/install-sdk.sh ${MACOS_VERSION}
echo "Setting up special idiap.ch host..."
${DIR}/idiap-host.sh
echo "Installing homebrew and build-time dependencies...":w
${DIR}/setup-paths.sh
${DIR}/install-homebrew.sh ${USER}
echo "Installing (or updating) gitlab runner..."
{DIR}/install-gitlab-runner.sh
doc/macos-ci-install/datetime.sh deleted 100755 → 0
+ 0
− 12
View file @ 41c8ade3
#!/usr/bin/env bash
# run more easily.
# -e will cause the script to exit immediately if any command within it exits non 0
# -o pipefail : this will cause the script to exit with the last exit code run.
# In tandem with -e, it will return the exit code of the first
# failing command.
set -eox pipefail
systemsetup -setusingnetworktime on
systemsetup -settimezone Europe/Zurich
systemsetup -setnetworktimeserver time.euro.apple.com.
doc/macos-ci-install/idiap-host.sh deleted 100755 → 0
+ 0
− 15
View file @ 41c8ade3
#!/usr/bin/env bash
if [[ `grep -c "www.idiap.ch" /etc/hosts` != 0 ]]; then
echo "Not updating /etc/hosts - www.idiap.ch is already present..."
else
echo "Updating /etc/hosts..."
echo "" >> /etc/hosts
echo "#We fake www.idiap.ch to keep things internal" >> /etc/hosts
echo "What is the internal server IPv4 address?"
read ipv4add
echo "${ipv4add} www.idiap.ch" >> /etc/hosts
echo "What is the internal server IPv6 address?"
read ipv6add
echo "${ipv6add} www.idiap.ch" >> /etc/hosts
fi
doc/macos-ci-install/install-gitlab-runner.sh deleted 100755 → 0
+ 0
− 3
View file @ 41c8ade3
#!/usr/bin/env bash
set -eox pipefail
brew services start gitlab-runner
doc/macos-ci-install/install-homebrew.sh deleted 100755 → 0
+ 0
− 22
View file @ 41c8ade3
#!/usr/bin/env bash
set -x
if [[ $EUID == 0 ]]; then
# changes path setup for all users, puts homebrew first
sed -e '/^\/usr\/local/d' -i .orig /etc/paths
echo -e "/usr/local/bin\n/usr/local/sbin\n/usr/local/opt/coreutils/libexec/gnubin\n$(cat /etc/paths)" > /etc/paths
# restarts to install brew as non-root user
exec su ${1} -c "$(which bash) ${0}"
fi
/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)"
brew=/usr/local/bin/brew
if [ ! -x ${brew} ]; then
brew=/opt/homebrew/bin/brew
fi
${brew} install curl git coreutils highlight neovim tmux htop python@3 pygments imagemagick gitlab-runner
${brew} services list #forces the installation of "services" support
${brew} install --cask mactex
doc/macos-ci-install/install-sdk.sh deleted 100755 → 0
+ 0
− 18
View file @ 41c8ade3
#!/usr/bin/env bash
# Installs the relevant SDK
# gets the current path leading to this script
DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" >/dev/null && pwd )"
if [ ! -r "MacOSX${1}.sdk.tar.xz" ]; then
echo "Downloading macOS ${1} SDK..."
curl -L -o ${DIR}/MacOSX${1}.sdk.tar.xz https://github.com/phracker/MacOSX-SDKs/releases/download/10.13/MacOSX${1}.sdk.tar.xz
else
echo "File MacOSX${1}.sdk.tar.xz is already here, skip download"
fi
if [ ! -d /opt ]; then mkdir /opt; fi
cd /opt
tar xfJ "${DIR}/MacOSX${1}.sdk.tar.xz"
ln -s /opt/MacOSX${1}.sdk /Library/Developer/CommandLineTools/SDKs/
doc/macos-ci-install/reconfig-gitlab-runner.sh deleted 100755 → 0
+ 0
− 17
View file @ 41c8ade3
#!/usr/bin/env bash
set -x
if [[ $EUID == 0 ]]; then
# restarts to reconfigure as gitlab user
exec su gitlab -c "$(which bash) ${0}"
fi
# the command above is bogus - it will use the "admin" user home dir
# you need to reconfigure it to fix this
cfgfile="/Users/gitlab/Library/LaunchAgents/homebrew.mxcl.gitlab-runner.plist"
/bin/launchctl stop $cfgfile
/bin/launchctl unload $cfgfile
/usr/bin/sed -i~ 's/admin/gitlab/g' $cfgfile
/bin/launchctl load $cfgfile
/bin/launchctl start $cfgfile
doc/macos-ci-install/setup-paths.sh deleted 100755 → 0
+ 0
− 16
View file @ 41c8ade3
#!/usr/bin/env bash
set -x
# changes path setup for all users, puts homebrew first
sed -e '/^\/usr\/local/d' -i .orig /etc/paths
sed -e '/^\/Library\/TeX\/texbin/d' -i .orig /etc/paths
echo -e "/usr/local/bin\n/usr/local/sbin\n/usr/local/opt/coreutils/libexec/gnubin\n/Library/TeX/texbin\n$(cat /etc/paths)" > /etc/paths
# ensures no cross-talking happens between miniconda installations on
# the shared builder - see bob/bob.devtools#12 and bob/bob.devtools!8
condadir=/Users/${USER}/.conda
mkdir ${condadir}
touch ${condadir}/environments.txt
chown -R ${USER}:staff ${condadir}
chmod a-w ${condadir}/environments.txt
doc/macos-ci-install/system-update.sh deleted 100755 → 0
+ 0
− 39
View file @ 41c8ade3
#!/bin/bash
set -eox pipefail
# read into array ${RECOMMENDED[@]}, a filtered list of available and
# recommended software updates
self=`basename $0`
TMPFILE=`mktemp -t ${self}` || exit 1
softwareupdate -l | grep -e '^\s\+\*' | sed -e 's/^[ \*]*//g' > ${TMPFILE}
# starts of strings of installables we don't care about
BLACKLIST=()
BLACKLIST+=('iBook')
BLACKLIST+=('iTunes')
BLACKLIST+=('Install macOS')
cat ${TMPFILE} | while read k; do
update="yes"
for l in "${BLACKLIST[@]}"; do
if [[ "${k}" == ${l}* ]]; then
update="no"
fi
done
if [[ "${update}" == "yes" ]]; then
echo "Updating $k..."
touch /tmp/.com.apple.dt.CommandLineTools.installondemand.in-progress
softwareupdate --verbose --install "${k}"
rm /tmp/.com.apple.dt.CommandLineTools.installondemand.in-progress
else
echo "Ignoring update for $k..."
softwareupdate --ignore "${k}"
fi
done
rm -f ${TMPFILE}
# We don't want our system changing on us or restarting to update. Disable
# automatic updates.
echo "Disable automatic updates…"
softwareupdate --schedule off
doc/macos-ci-install/update-ci.sh deleted 100644 → 0
+ 0
− 31
View file @ 41c8ade3
#!/usr/bin/env bash
# Update CI installation
brew=/usr/local/bin/brew
pip=/usr/local/bin/pip3
if [ ! -x ${brew} ]; then
brew=/opt/homebrew/bin/brew
pip=/opt/homebrew/bin/pip3
fi
echo "[update-ci.sh] Updating homebrew..."
${brew} update
echo "[update-ci.sh] Upgrading homebrew (outdated) packages..."
${brew} upgrade
# A cask upgrade may require sudo, so we cannot do this
# with an unattended setup
#echo "[update-ci.sh] Updating homebrew casks..."
#${brew} cask upgrade
echo "[update-ci.sh] Cleaning-up homebrew..."
${brew} cleanup
# Updates PIP packages installed
function pipupdate() {
echo "[update-ci.sh] Updating ${1} packages..."
[ ! -x "${1}" ] && return
${1} list --outdated --format=freeze | grep -v '^\-e' | cut -d = -f 1 | xargs -n1 ${1} install -U;
}
pipupdate ${pip}
doc/macos-ci-install/xcode-cli-tools.sh deleted 100755 → 0
+ 0
− 14
View file @ 41c8ade3
#!/bin/sh
set -eox pipefail
# create the placeholder file that's checked by CLI updates' .dist code
# in Apple's SUS catalog
touch /tmp/.com.apple.dt.CommandLineTools.installondemand.in-progress
# find the CLI Tools update
PROD=$(softwareupdate -l | grep "\*.*Command Line" | tail -n 1 | awk -F"*" '{print $2}' | sed -e 's/^ *//' | tr -d '\n')
# install it
softwareupdate -i "$PROD" --verbose
rm /tmp/.com.apple.dt.CommandLineTools.installondemand.in-progress
doc/macos.rst deleted 100644 → 0
+ 0
− 218
View file @ 41c8ade3
.. vim: set fileencoding=utf-8 :
.. _bob.devtools.ci.macos:
============================
Deploying a macOS-based CI
============================
This document contains instructions to build and deploy a new bare-OS CI for
macOS. Instructions for deployment assume a freshly installed machine.
.. note::
For sanity, don't use an OS with lower version number than the macOS SDK
code that will be installed (currently 10.9). There may be undesired
consequences. You may use the latest OS version in case of doubt, but by
default we recommend the one before the last stable version, for stability.
So, if the current version is 10.14, a good base install would use 10.13.
.. warning::
Idiap has throttling rules that are typically applied to all machines in the
lab network. To avoid issues for newly installed CI nodes, ensure you
request throttling to be disabled for new CI machines.
Building the reference setup
----------------------------
0. Make sure the computer name is correctly set or execute the following on the
command-line, as an admin user::
$ sudo scutil --get LocalHostName
...
$ sudo scutil --get HostName
...
$ sudo scutil --get ComputerName
...
# if applicable, run the following commands
$ sudo scutil --set LocalHostName "<hostname-without-domain-name>"
$ sudo scutil --set HostName "<fully-qualified-domain-name>"
$ sudo scutil --set ComputerName "<fully-qualified-domain-name>"
1. Disable all energy saving features. Go to "System Preferences" then "Energy
Saver":
- Enable "Prevent computer from sleeping..."
- Disable "Put hard disks to sleep when possible"
- Leave "Wake for network access" enabled
- You may leave the display on sleep to 10 minutes
2. To be able to send e-mails from the command-line (e.g., when completing
cronjobs), via the Idiap SMTP, you will need to modify the postfix
configuration:
- Edit the file ``/etc/postfix/main.cf`` to add a line stating ``relayhost =
[smtp.lab.idiap.ch]`` (all e-mails should be routed by this SMTP host)
- Edit the file ``/etc/postfix/generic`` to add a line stating
``admin@hostname.lab.idiap.ch hostname@lab.idiap.ch`` (all e-mails leaving
the lab infrastruture need to have ``@lab.idiap.ch`` addresses)
- Run ``postmap /etc/postfix/generic`` as root (required to update the
internal postfix aliases)
3. Create a new user (without administrative priviledges) called ``gitlab``.
Choose a password to protect access to this user. In "Login Options",
select this user to auto-login, type its password to confirm
4. Enable SSH access to the machine by going on ``System Preferences``,
``Sharing`` and then selecting ``Remote Login`` (for ssh) and ``Screen
Sharing`` (for remote desktop connections). Make sure only users on the
``Administrators`` group can access the machine.
5. Create as many ``Administrator`` users as required to manage the machine
6. Login as administrator of the machine (so, not on the `gitlab` account). As
that user, run the ``admin-install.sh`` script (after copying this repo from
https://gitlab.idiap.ch/bob/bob.devtools via a zip file download)::
$ cd
$ curl -o bob.devtools-master.zip https://gitlab.idiap.ch/bob/bob.devtools/-/archive/master/bob.devtools-master.zip
$ unzip ~/Downloads/bob.devtools-master.zip
$ cd bob.devtools-master/doc/macos-ci-install
$ sudo ./admin-install.sh 10.9 gitlab
Check that script for details on what is installed and the order. You may
execute pieces of the script by hand if something fails. In that case,
please investigate why it fails and properly fix the scripts so the next
install runs more smoothly.
7. Check the maximum number of files that can be opened on a shell session
with the command ``launchctl limit maxfiles``. If smaller than 4096, set
the maximum number of open files to 4096 by creating the file
``/Library/LaunchDaemons/limit.maxfiles.plist`` with the following
contents::
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
<key>Label</key>
<string>limit.maxfiles</string>
<key>ProgramArguments</key>
<array>
<string>launchctl</string>
<string>limit</string>
<string>maxfiles</string>
<string>4096</string>
<string>unlimited</string>
</array>
<key>RunAtLoad</key>
<true/>
<key>ServiceIPC</key>
<false/>
</dict>
</plist>
At this occasion, verify if the kernel limits are not lower than this value
using::
$ sysctl kern.maxfilesperproc
10240 #example output
$ sysctl kern.maxfiles
12288 #example output
If that is the case (i.e., the values are lower than 4096), set those values
so they are slightly higher than that new limit with ``sudo sysctl -w
kern.maxfilesperproc=10240`` and ``sudo sysctl -w kern.maxfiles=12288``
respectively, for example.
8. Install oh-my-zsh_ for both the admin and gitlab users. Set ZSH theme "ys".
Add the following bits to ``.zshrc`` to ensure completions work::
# Enables homebrew auto-completions for zsh (add: right at the top!)
if type brew &>/dev/null; then
FPATH=$(brew --prefix)/share/zsh/site-functions:$FPATH
ZSH_DISABLE_COMPFIX="true"
fi
...
# plugins (add: just before sourcing oh-my-zsh)
plugins=()
plugins+=(docker)
plugins+=(git)
plugins+=(gitfast)
plugins+=(python)
plugins+=(themes)
plugins+=(z)
plugins+=(zsh-syntax-highlighting)
plugins+=(history-substring-search)
9. Enter as gitlab user and install/configure the `gitlab runner`_:
Configure the runner for `shell executor`_, with local caching. As
``gitlab`` user, execute on the command-line::
# notice that running `brew services gitlab-runner start or restart` will
# break the configuration of the service once more. Execute the following
# to correct for it:
$ /bin/bash <(curl -s https://gitlab.idiap.ch/bob/bob.devtools/raw/master/doc/macos-ci-install/reconfig-gitlab-runner.sh)
Once that is set, your runner configuration (``~/.gitlab-runner/config.toml``) should look like this (remove comments if gitlab does not like them)::
concurrent = 8 # set this to the number of cores available
check_interval = 10 # do **not** leave this to zero
[[runners]]
name = "<runner-name>" # use a suggestive name
output_limit = 102400 # this value is in kb, so we mean 100 mb
url = "https://gitlab.idiap.ch" # this is our gitlab service
token = "abcdefabcdefabcdefabcdefabcdef" # this is specific to the conn.
executor = "shell" # select this
builds_dir = "/Users/gitlab/builds" # set this or bugs occur
cache_dir = "/Users/gitlab/caches" # this is optional, but desirable
shell = "bash"
10. So conda works properly on a shared builder, as the ``gitlab`` user, make
sure to create an empty, read-only file named
``~/.conda/environments.txt``. Failure to create this file and make it
read-only to the gitlab user, will create a concurrence issue on the shared
builder, w.r.t. to conda.
11. While at the gitlab user, install `Docker for Mac`_. Ensure to set it up to
start at login. In "Preferences > Filesystem Sharing", ensure that
`/var/folders` is included in the list (that is the default location for
temporary files in macOS).
12. Once installed, go the the settings and in "General" uncheck the option
"Use gRPC FUSE for file sharing". At the time of writing (04.10.2021), the
gRPC Fuse system does not work well with beat/beat.core> testing.
13. Import Idiap's self-signed root certificate::
$ curl -o cert.crt -s https://pki.idiap.ch/download/Idiap_2016_Root-cacert.crt
$ sudo security add-trusted-cert -d -r trustRoot -k /Library/Keychains/System.keychain cert.crt
$ rm -f cert.crt
14. Reboot the machine. At this point, the gitlab user should be auto-logged
and the runner process should be executing. Congratulations, you're done!
Running regular updates
-----------------------
We recommend that the CI machine to have homebrew and installed pip packages
updated regularly (once a week). To do so, setup a cronjob like the following,
for the ``admin`` user:
.. code-block:: text
SHELL=/bin/bash
MAILTO="your.email@idiap.ch"
00 12 * * 0 /bin/bash <(curl -s https://gitlab.idiap.ch/bob/bob.devtools/raw/master/doc/macos-ci-install/update-ci.sh)
And one line the following for the ``gitlab`` user, about 30 minutes later, to
give time for the machine updating to be performed. The second cronjob will
re-spawn the gitlab-runner, which may be necessary if it was updated on the
previous step:
.. code-block:: text
SHELL=/bin/bash
MAILTO="your.email@idiap.ch"
30 12 * * 0 /bin/bash <(curl -s https://gitlab.idiap.ch/bob/bob.devtools/raw/master/doc/macos-ci-install/reconfig-gitlab-runner.sh)
.. include:: links.rst
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment