Updating GitLab in a Docker Container

I have an instance of GitLab that I am using for a project that is suffering under an NDA.  I have it installed in a Docker container at home.  We have just bought some server space so I am migrating the service to the new server.  The GitLab I have at home is an older version than the latest.

It makes sense to update GitLab in place before doing the migration.  So, I need to back it up first before upgrading it and then migrating it to the new server.

A list:

  • I use a user to store docker-compose.yml’s
  • cd GitLab
  • docker-compose down
  • su root -c “tar cfj Backup/gitlab-20181213.tbz2 volumes/”
  • docker-compose up -d
  • docker commit -p gitlab_web_1 backup-gitlab-20181213
  • docker save -o Backup/docker-gitlab-20181213.tar backup-gitlab-20181213
  • docker exec -i -t gitlab_web_1 bash
  • gitlab-rake gitlab:backup:create STRATEGY=copy
  • exit
  • su root -c “cp volumes/data/backups/1544700734_2018_12_13_11.3.1_gitlab_backup.tar Backup/”
  • docker exec -i -t gitlab_web_1 bash

Strange, I have the same version of Ubuntu in the Docker container but ‘apt-get install gitlabce‘ won’t update to “gitlabce/xenial,now 11.5.3-ce.0″.  Let’s do this on both containers:

  • cd /tmp
  • curl -LO https://packages.gitlab.com/install/repositories/gitlab/gitlab-ce/script.deb.sh
  • bach script.deb.sh
  • apt-get update
  • apt-get install gitlab-ce

That last one takes a LONG time but look:

I’m happy with that.  I’ve checked the site and everything seems to be working.  I did notice that the “apt-get install gitlabce” did a back up too.  Belt and braces though, Belt AND braces.

WordPress in a Docker container

Running WordPress in a container is tricky.  It is tricky because WordPress will hardcode the site’s domain in URLs.  WordPress is happy with this situation.  The container does not know the virtual name it will be given and so URLs contain a domain string i.e. 192.168.0.3.  I have failed to do this several times but I think I have got close this time.

I use Let’s Encrypt, TLS and SNI with Apache to point to services running in Docker containers.  I prefer that services do not encrypt services because my poor server should not have to encrypt twice.  The technique I use to point to containers is called a reverse proxy.

At the interface of the reverse proxy we can change all sorts of things.  This is my Apache config for WordPress:

VirtualHost *:80>
  ServerAdmin williams@c3iq.co.uk
  ServerName bob.domain.om
  ErrorLog /usr1/home/frank/virtual/bob/logs/error_log
  CustomLog /usr1/home/frank/virtual/bob/logs/access_log common

  Redirect / https://bob.domain.com/
</VirtualHost>

<VirtualHost *:443>
  Protocols h2 http/1.1
  ServerAdmin williams@c3iq.co.uk
  ServerName bob.domain.com
  ErrorLog /usr1/home/frank/virtual/bob/logs/error_log
  CustomLog /usr1/home/frank/virtual/bob/logs/access_log common

  #AddType application/x-httpd-php .php

  ProxyRequests Off
  ProxyPreserveHost Off

  #SSLProxyCheckPeerCN off
  #SSLProxyCheckPeerExpire off
  #SSLProxyVerify none
  #SSLProxyEngine on

  Substitute "s|http://192.168.0.3:8002|https://bob.domain.com|n"
  Substitute "s|http:\/\/192.168.0.3:8002|https:\/\/bob.domain.com|n"
  Substitute "s|http%3A%2F%2F192.168.0.3%3A8002|http%3A%2F%2Fbob.domain.com|n"
  Substitute "s|http://0.gravatar.com|https://0.gravatar.com|n"
#  Substitute "s|http://|https://|n"

  FilterDeclare NEWPATHS
  FilterProvider NEWPATHS SUBSTITUTE "%{Content_Type} =~ m|^text/html|"
  FilterProvider NEWPATHS SUBSTITUTE "%{Content_Type} =~ m|^text/css|"
  FilterProvider NEWPATHS SUBSTITUTE "%{Content_Type} =~ m|^text/javascript|"
  FilterProvider NEWPATHS SUBSTITUTE "%{Content_Type} =~ m|^application/javascript|"
  FilterChain NEWPATHS

  Header edit Location ^http://192.168.0.3 https://bob.domain.com
  Header set X-Frame-Options "SAME-ORIGIN"

  <Location "/">
    ProxyPass http://192.168.0.3:8002/
    ProxyPassReverse http://192.168.0.3:8002/
    ProxyPassReverseCookieDomain 192.168.0.3 bob.domain.com
    RequestHeader unset Accept-Encoding

    #AllowMethods GET POST OPTIONS PUT

    AllowOverride All
    Require all granted

    #Require ip 192.168
  </Location>

  SSLEngine on
  SSLProtocol -ALL +TLSv1.2
  SSLCipherSuite ALL:!aNULL:!ADH:!eNULL:!LOW:!EXP:!RC4:!DES:!3DES:+HIGH:+MEDIUM
  SSLCertificateKeyFile /etc/letsencrypt/live/c3iq.co.uk/privkey.pem
  SSLCertificateFile /etc/letsencrypt/live/c3iq.co.uk/fullchain.pem

  <Files ~ "\.(cgi|shtml|phtml|php?)$">
     SSLOptions +StdEnvVars
  </Files>
</VirtualHost>

Please, comment if you see a mistake or know of a better pattern.

When asked to produce a Gantt chart and you are already using GitLab…

I have never been asked to create a Gantt chart using a spreadsheet before.  I’ve rarely been asked to create anything using spreadsheets.  Software engineers are in the business of automating mundane tasks.  Use the right tool for the job.  Right?

Modern software engineering creates or fixes software in tight cycles depending on the users’ needs.  Users’ needs are described in issues, these are discussed and may become tasks.  Many will represent a few hours work and hence will not be useful in a Gantt chart.  The process is almost organic and so the waterfall approach is a poor fit.

I am currently involved with a company who is starting up a business that will rent out space in an app on Android and iOS.  The business may look at PWAs later.  In the app business, I am starting to understand, if the unique point of the app is not a must have the app must evolve over time.  The features must improve and new ones be added.  It is not good enough that the software is updated once a year.  Add to this that security issues must be fixed as and when they are discovered we have another reason why we should not describe our project’s effort using a waterfall methodology.

In this relationship, there are gaps on both sides.  I do not have a lot of experience with spreadsheets.  To some this is odd but I have always used tools or created them to avoid the overhead of maintaining a spreadsheet.  My colleagues are new to software/app development and maintenance.  However, they are used to project management where waterfall methods do work.  They have insisted that I create a Gannt chart.

My first attempt to do this quickly used GanttLab which uses the GitLab API to pull out details of a project to produce the chart.  The result is not very friendly and this attempt to tick a box did not go down very well.

This week I have been learning how to colour in cells in a spreadsheet but still to automate the process.  The following is version 1.

How to create a Gantt chart from GitLab

The spreadsheet will take issues/tasks against Milestones in GitLab and create a Gantt chart from them.

I have GitLab installed in a Docker container. This is how I retrieve the issues:

  • $ docker exec -i -t gitlab_web_1 bash
  • $ gitlab-psql -d gitlabhq_production
  • $ Copy (select issues.iid, m.title, issues.title, issues.description, time_estimate, issues.due_date from issues, milestones as m where issues.state = ‘opened’ and issues.time_estimate > 0 and issues.milestone_id = m.id ORDER BY issues.due_date ASC) To ‘/tmp/issues.csv’ With CSV DELIMITER ‘;’;

Back at the host:

  • docker cp gitlab_web_1:/tmp/issues.csv .

I then delete everything in the “issues” sheet of the spreadsheet and make sure there is a row at the top that can say anything but I label them in bold:

  • Issues
  • Milestone
  • Title
  • Description
  • Estimate
  • Due date

The Gantt sheet describes everything in a milestone that has an estimate greater than zero.

You can get the spreadsheet from GitHub.

Learning Locker in a Docker container

A list to help install Learning Locker:

  • Apache on the host supports TLS and SNI:
<VirtualHost *:80>
  ServerAdmin email@somedomain.com
  ServerName lrs.somedomain.com
  ErrorLog /path/to/my/error_log
  CustomLog /path/to/access_log common

  Redirect / https://lrs.somedomain.com/
</VirtualHost>

<VirtualHost *:443>
  Protocols h2 http/1.1
  ServerAdmin email@somedomain.com
  ServerName lrs.somedomain.com
  ErrorLog /path/to/error_log
  CustomLog /path/to/access_log common

  AddType application/x-httpd-php .php

  ProxyRequests On
  <Location "/">
    ProxyPass http://localhost:8001/
    ProxyPassReverse http://localhost:8001/
#    AllowMethods GET POST OPTIONS
    AllowOverride All
    Require all granted
  </Location>

  SSLEngine on
  #SSLProtocol -ALL +TLSv1.1 +TLSv1.2
  SSLProtocol -ALL +TLSv1.2
  SSLCipherSuite ALL:!aNULL:!ADH:!eNULL:!LOW:!EXP:!RC4:!DES:!3DES:+HIGH:+MEDIUM
  SSLCertificateKeyFile /etc/letsencrypt/live/c3iq.co.uk/privkey.pem
  SSLCertificateFile /etc/letsencrypt/live/c3iq.co.uk/fullchain.pem

  <Files ~ "\.(cgi|shtml|phtml|php?)$">
     SSLOptions +StdEnvVars
  </Files>
</VirtualHost>

Let’s Encrypt is a beautiful thing.

  • I use a user called docker to corral configs for my docker containers:
  • git clone https://github.com/michzimny/learninglocker2-docker
  • mv learninglocker2-docker LRS
  • In LRS, mkdir volumes
  • edit .env
  • edit app/Dockerfile to use the latest version of Learning Locker
  • edit nginx.conf.template to reflect that Apache on the host does the redirect from 80 to 443

./build-dev.sh and docker-compose up -d.

While a default of HTTPS is a good thing it took me a while to find the redirect in NGINX.  That’s my job. 🙂  I assumed there was an issue with versions when the server reported Service Not Available.  It took several iterations and some debugging to get right.

This is a test of Gutenberg on my Gemini PDA

So, this is a standard paragraph. and it has some text.

Gemini PDA with an Illy cup
Gemini PDA with an Illy cup

This is another text block.  This seems to be working quite well.  I think, that the speed of typing with a small keyboard is dependant on how high the table is compared to where my elbows are and the angle of them.  I’m currently sitting on my sofa with the PDA and I’m typing quite quickly.  My back is hurting so maybe…