diff --git a/install/docker-ci/README.md b/install/docker-ci/README.md new file mode 100644 index 0000000000000000000000000000000000000000..1d1ebaefd88677851cf7f6829a085950e26ace71 --- /dev/null +++ b/install/docker-ci/README.md @@ -0,0 +1,102 @@ +# Gitlab Docker CI Machine Installation + +This guide assumes you want to install a new CI machine from scratch, with +`docker` and `docker build` support, for Bob. + + +## Steps + +1. Install a base Debian 8.x distribution +2. Install the following packages: + ```sh + $ apt-get update + $ apt-get install openssh-server curl vim-nox htop + ``` +3. Make sure to copy access keys for personel at `/root/.ssh/authorized_keys` +4. Access your machine via ssh from this point onwards, make sure it is OK +5. Follow the steps in section "Kernel Support" below for extended CPU and + memory control on docker containers, in case you need those +6. If there are locale problems, install the missing locales with: + ```sh + $ dpkg-reconfigure locales + ``` +7. Install docker: + ```sh + $ curl -sSL https://get.docker.com/ | sh + ``` +8. Install Gitlab CI: + ```sh + $ curl -L https://packages.gitlab.com/install/repositories/runner/gitlab-ci-multi-runner/script.deb.sh | bash + ``` +9. Add the `gitlab-runner` user to the docker group: + ```sh + $ usermod -aG docker gitlab-runner + ``` +10. Register the runner and fine-tune its configuration: + ```sh + $ gitlab-ci-multi-runner register + $ vim /etc/gitlab-runner/config.toml + $ service gitlab-runner restart + ``` + If you'd like to user `docker build`, you'll need to register 2 runners: one + with the `docker` executor and another with a `shell` executor (that can + access `docker build`. More information here: + https://docs.gitlab.com/ce/ci/docker/using_docker_build.html +11. Configure docker to access Idiap's registry: + ```sh + $ mkdir -p /etc/docker/certs.d/docker.idiap.ch + $ scp ./docker.crt root@<machine>:/etc/docker/certs.d/docker.idiap.ch/ + $ scp ./gitlab.crt root@<machine>:/etc/docker/certs.d/docker.idiap.ch/ + ``` + + +## Kernel Support + +If you need memory and CPU quota control over your Docker containers, you'll +ned to adjust a few parameters on the running machine and, possibly, recompile +the its kernel. + +Here are some tests you can run to test if both are available: + +```sh +$ for the memory check +$ docker run -it --rm -m 128M --memory-swap=128M progrium/stress --vm 1 --vm-bytes 130M --vm-hang 0 +$ for CPU quota check +$ docker run -it --rm --cpu-quota=50000 --cpu-period=100000 progrium/stress --cpu 3 +``` + +In each case, you should not see any warning message and the docker engine must +respect the imposed limits (which you can verify with docker stats). If you see +warning messages claiming a feature is not enabled, you must either enable the +features either by recompiling the kernel or passing the right boot parameters +to enable them. On a Debian system, you have to do the following: + +Add `cgroup_enable=memory swapaccount=1` as boot parameters. Reboot with those +options. After rebooting, the first command-line above should start working. Reference: https://docs.docker.com/engine/installation/linux/ubuntulinux/#/enable-memory-and-swap-accounting + +Recompile the stock kernel and add `CPU bandwidth provisioning for +FAIR_GROUP_SCHED` under `General setup -> Control group support -> Group CPU +scheduler`. Reboot with that modified kernel. After rebooting, the second +command-line above should start working correctly. Reference: +http://www.blaess.fr/christophe/2012/01/07/linux-3-2-cfs-cpu-bandwidth-english-version/ + +N.B.: Idiap Debian kernels have both enabled by default. In case you're +running on a virtual machine with a single CPU, we have noticed the CFS can be +a bit imprecise (e.g. allocate more than 50% CPU even if you set it to have +only 50%). It works better if there are more CPUs assigned to the virtual +machine in this case. + +Recompilation on a stock Debian distribution: + +```sh +$ apt-get install fakeroot kernel-package linux-source libncurses5-dev +$ cd /usr/src +$ tar xf linux-source-3.16.tar.xz +$ cd linux-source-3.16 +$ xzcat ../linux-config-3.16/config.amd64_none_amd64.xz > .config +$ make menuconfig +# Do as explained above +# Save the new configuration +$ fakeroot make-kpkg --initrd --revision=1.0.bob kernel_image +$ dpkg -i ../linux-image-*.dpkg +``` diff --git a/install/docker-ci/docker.crt b/install/docker-ci/docker.crt new file mode 100644 index 0000000000000000000000000000000000000000..5bcee722eab44a9424041441bf34cf744db5216f --- /dev/null +++ b/install/docker-ci/docker.crt @@ -0,0 +1,40 @@ +-----BEGIN CERTIFICATE----- +MIIG7zCCBNegAwIBAgIJAP2rGWTQbd8bMA0GCSqGSIb3DQEBCwUAMIGYMQswCQYD +VQQGEwJDSDELMAkGA1UECBMCVlMxETAPBgNVBAcTCE1hcnRpZ255MSEwHwYDVQQK +ExhJZGlhcCBSZXNlYXJjaCBJbnN0aXR1dGUxDDAKBgNVBAsTA1BLSTEbMBkGA1UE +AxMSSWRpYXAgUm9vdCBDQSAyMDE2MRswGQYJKoZIhvcNAQkBFgxwa2lAaWRpYXAu +Y2gwHhcNMTYwMTI3MTU1MzAxWhcNNDYwMTMwMTU1MzAxWjCBmDELMAkGA1UEBhMC +Q0gxCzAJBgNVBAgTAlZTMREwDwYDVQQHEwhNYXJ0aWdueTEhMB8GA1UEChMYSWRp +YXAgUmVzZWFyY2ggSW5zdGl0dXRlMQwwCgYDVQQLEwNQS0kxGzAZBgNVBAMTEklk +aWFwIFJvb3QgQ0EgMjAxNjEbMBkGCSqGSIb3DQEJARYMcGtpQGlkaWFwLmNoMIIC +IjANBgkqhkiG9w0BAQEFAAOCAg8AMIICCgKCAgEAydOC2EzaT+6bPcUFin8BawnS +OxfFzDh18536O+PDAuyycOY1cIefpC3PvLk7XM83hIlUY1Q02/yt4h0iXkAUs/RI +VE6PyIh7pTxabyjIqIZ7xlVkK9cfGhlUm+ofd/Limo6WBbaH64jf/Jm6KdRtu2n4 +q1brII31nCIwBvf8uVBuo5Y4NMS8bWcaBxj93S+C16sV4E2gz28FYCuSpAnU5DeP +0PwNsuBkVFgHUwc2VxfUyCmQEs+qlWb+6W7ULWvtD3K28414tygVxOSXkl9ZVJGX +KBwevSs2TTlH25Q3TAY4mXweJ2RweFwlnbzpA0YYrQUDy5MTVslD5Sl0c5vO6HZH +DgT4jyWbL87akczTaz028zmUQvC7RoGPKXqhHlaO3uDu1fBC7aoA/L1akqfYGnoO +x4xU7gouGi9Hck23DKLsW1HP2PzaZ/ME++IZPr2I049aKBadz3vCbAh2bHgosdyp +a69e6SMiq9guloQXCqFTSk+P6CwU9wWil3SdGojRafyocKyoBL67hKIFkJQOdjhq +edi3WfoSU+1kDwEyXU2fCsTjgg0q1BsJFctSUFs7QoVMAoyXXauDCf1YuojlXuXM +FrQmYLBzSFA7OFFynWbnn1mHMzsHEiAzAr0+7ecIreBwN1gJqx1+7hxkSQzDm2oH +jTPGGwJfbJnej4RduBkCAwEAAaOCATgwggE0MB0GA1UdDgQWBBR6H/NwUn5K3lVV +IVgpNR41/bxdtTCBzQYDVR0jBIHFMIHCgBR6H/NwUn5K3lVVIVgpNR41/bxdtaGB +nqSBmzCBmDELMAkGA1UEBhMCQ0gxCzAJBgNVBAgTAlZTMREwDwYDVQQHEwhNYXJ0 +aWdueTEhMB8GA1UEChMYSWRpYXAgUmVzZWFyY2ggSW5zdGl0dXRlMQwwCgYDVQQL +EwNQS0kxGzAZBgNVBAMTEklkaWFwIFJvb3QgQ0EgMjAxNjEbMBkGCSqGSIb3DQEJ +ARYMcGtpQGlkaWFwLmNoggkA/asZZNBt3xswDwYDVR0TAQH/BAUwAwEB/zAJBgNV +HRIEAjAAMBcGA1UdEQQQMA6BDHBraUBpZGlhcC5jaDAOBgNVHQ8BAf8EBAMCAQYw +DQYJKoZIhvcNAQELBQADggIBAJmXqtgmHj1XXUptloVVsCwCYBU8ykf1dZz2Kxrx +oe0dnDO24CA6w3D3TCt8rncT2lFNRTbc/4HO32xl1IDNiWh5P/ZPNpptwd6XjGR1 +EgDjpIBKNotf+6WWvcKrs23mj9UwNPHDwNA251LAMVXaoMN2iOflzj2BbIcasY3P +IcYeshd3CChy8QqltE1M8mjwb7brkIzwcPI5QEhW9NmfYUfbijILZrE2kgo6oOFH +mRZIDoexrd19hHLWFLxoe0IPj6R1GFajBHi8Ttt3tPQOPjwjGQvNfVPRhWh3/kAF +UrWZposffDDIc+8TNlrhkx+YKucYH56Tyuh6Y1Po7FCkvp2/G/JxKWeAEqKpI2+g +8Hsl0XjSOJ9bOhs+R0wMzeBzntDk8k+6ar3KYGJD24gQ+QDy4klE/rsdC/Gp6dEi +tSIPvH4VIvN0lLICWKj3IFhBv6IOJ12Xq5IMquDq5BZ6O2+yqoROIYQyhwHq+xhn +rqqR6TsFMl/F5R0j14oGzg+VdB8VsIrg7rTJx+oDD9r+Pa2hua4DRmQsw+CJgnHz +NqU3Xei/78W+eLh9HZvVqXpi4s/fF6z+lXKDHpqVRh5kNAKJbYQUfcV2H7FEtCux +NIDS6J1GnHJKCmYPuwFSrQ5VXM/1p7w+A9MkJktsxw2kxsRUvJn7Ka+bp7M6wERU +JHsX +-----END CERTIFICATE----- diff --git a/install/docker-ci/gitlab.crt b/install/docker-ci/gitlab.crt new file mode 100644 index 0000000000000000000000000000000000000000..756d6dd03e11ff9b990665dc413e4b571fd5d03e --- /dev/null +++ b/install/docker-ci/gitlab.crt @@ -0,0 +1,35 @@ +-----BEGIN CERTIFICATE----- +MIIGFzCCA/+gAwIBAgIUftbnnMmtgcTIGT75XUQodw40ExcwDQYJKoZIhvcNAQEL +BQAwSDELMAkGA1UEBhMCQk0xGTAXBgNVBAoTEFF1b1ZhZGlzIExpbWl0ZWQxHjAc +BgNVBAMTFVF1b1ZhZGlzIFJvb3QgQ0EgMiBHMzAeFw0xMjExMDYxNDUwMThaFw0y +MjExMDYxNDUwMThaME0xCzAJBgNVBAYTAkJNMRkwFwYDVQQKExBRdW9WYWRpcyBM +aW1pdGVkMSMwIQYDVQQDExpRdW9WYWRpcyBHbG9iYWwgU1NMIElDQSBHMzCCAiIw +DQYJKoZIhvcNAQEBBQADggIPADCCAgoCggIBANf8Od17be6c6lTGJDhEXpmkTs4y +Q39Rr5VJyBeWCg06nSS71s6xF3sZvKcV0MbXlXCYM2ZX7cNTbJ81gs7uDsKFp+vK +EymiKyEiI2SImOtECNnSg+RVR4np/xz/UlC0yFUisH75cZsJ8T1pkGMfiEouR0EM +7O0uFgoboRfUP582TTWy0F7ynSA6YfGKnKj0OFwZJmGHVkLs1VevWjhj3R1fsPan +H05P5moePFnpQdj1FofoSxUHZ0c7VB+sUimboHm/uHNY1LOsk77qiSuVC5/yrdg3 +2EEfP/mxJYT4r/5UiD7VahySzeZHzZ2OibQm2AfgfMN3l57lCM3/WPQBhMAPS1jz +kE+7MjajM2f0aZctimW4Hasrj8AQnfAdHqZehbhtXaAlffNEzCdpNK584oCTVR7N +UR9iZFx83ruTqpo+GcLP/iSYqhM4g7fy45sNhU+IS+ca03zbxTl3TTlkofXunI5B +xxE30eGSQpDZ5+iUJcEOAuVKrlYocFbB3KF45hwcbzPWQ1DcO2jFAapOtQzeS+MZ +yZzT2YseJ8hQHKu8YrXZWwKaNfyl8kFkHUBDICowNEoZvBwRCQp8sgqL6YRZy0uD +JGxmnC2e0BVKSjcIvmq/CRWH7yiTk9eWm73xrsg9iIyD/kwJEnLyIk8tR5V8p/hc +1H2AjDrZH12PsZ45AgMBAAGjgfMwgfAwEgYDVR0TAQH/BAgwBgEB/wIBATARBgNV +HSAECjAIMAYGBFUdIAAwOgYIKwYBBQUHAQEELjAsMCoGCCsGAQUFBzABhh5odHRw +Oi8vb2NzcC5xdW92YWRpc2dsb2JhbC5jb20wDgYDVR0PAQH/BAQDAgEGMB8GA1Ud +IwQYMBaAFO3nb3Zav2DsSVvGpXe7chZxm8Q9MDsGA1UdHwQ0MDIwMKAuoCyGKmh0 +dHA6Ly9jcmwucXVvdmFkaXNnbG9iYWwuY29tL3F2cmNhMmczLmNybDAdBgNVHQ4E +FgQUsxKJtalLNbwVAPCA6dh4h/ETfHYwDQYJKoZIhvcNAQELBQADggIBAFGm1Fqp +RMiKr7a6h707M+km36PVXZnX1NZocCn36MrfRvphotbOCDm+GmRkar9ZMGhc8c/A +Vn7JSCjwF9jNOFIOUyNLq0w4luk+Pt2YFDbgF8IDdx53xIo8Gv05e9xpTvQYaIto +qeHbQjGXfSGc91olfX6JUwZlxxbhdJH+rxTFAg0jcbqToJoScWTfXSr1QRcNbSTs +Y4CPG6oULsnhVvrzgldGSK+DxFi2OKcDsOKkV7W4IGg8Do2L/M588AfBnV8ERzpl +qgMBBQxC2+0N6RdFHbmZt0HQE/NIg1s0xcjGx1XW3YTOfje31rmAXKHOehm4Bu48 +gr8gePq5cdQ2W9tA0Dnytb9wzH2SyPPIXRI7yNxaX9H8wYeDeeiKSSmQtfh1v5cV +7RXvm8F6hLJkkco/HOW3dAUwZFcKsUH+1eUJKLN18eDGwB8yGawjHvOKqcfg5Lf/ +TvC7hgcx7pDYaCCaqHaekgUwXbB2Enzqr1fdwoU1c01W5YuQAtAx5wk1bf34Yq/J +ph7wNXGvo88N0/EfP9AdVGmJzy7VuRXeVAOyjKAIeADMlwpjBRhcbs9m3dkqvoMb +SXKJxv/hFmNgEOvOlaFsXX1dbKg1v+C1AzKAFdiuAIa62JzASiEhigqNSdqdTsOh +8W8hdONuKKpe9zKedhBFAvuxhDgKmnySglYc +-----END CERTIFICATE-----