How to use Docker to learn Jenkins

Often in software engineering we are expected to develop using new tools; and so, it is incredibly helpful to be able to learn at least somewhat quickly. In this article, I aim to show how you can use Docker to create a local instance of something (in this case Jenkins), in order to safely experiment with moderately complicated tooling.

Original Problem: My team wanted to add a new build into our Jenkins pipeline, which involved a somewhat complicated process of passing secrets in.

It fell on me to develop a solution; with 2 caveats:

1. I didn’t know how Jenkins works.
2. It is always a terrible idea to develop/test in production.

I won’t describe how I solved the particular problem, but rather, how I set up a Jenkins “sandbox”.

Note: while I specifically focus on Jenkins, the process of spinning up a lightweight, local environment should apply for any moderately complex system.

Who is the intended audience? Many moderate to large sized companies have development process entirely abstracted away, so my target audience is primarily individuals and small teams working on technical products, trying to get things working.

Plan of Attack

In order to test Jenkins, we want to :

1. Spin-up a Docker container running Jenkins on our machine (ideally with the Jenkins GUI ported to our browser)
2. Configure the container to some base-state with configured to replicate our production environment, and save a ‘save-point’
3. Spin up a new container from the ‘save-point’
4. Break things
5. Repeat steps 3 and 4

In this article, I’ll go through steps 1–3, and leave the steps 4 and 5 up to you.

Quick Terms

What is Jenkins: If you don’t know what Jenkins is or does, that shouldn’t stop this article from being helpful to you. Jenkins is a CI/CD automation tool (like CircleCI or Travis) that runs configurable jobs in a server based on triggers (i.e. running tests on each git commit), to ensure that changes you have don’t break the code base or to deploy new changes.

What is Docker?: Docker is a container platform.

What is a Container: “A container is a standard unit of software that packages up code and all its dependencies so the application runs quickly and reliably from one computing environment to another.” — Docker.com

What is an Image: An image what forms the basis for the container. To make an analogy to OOP: a class is to an instance, as a image is to a container. In other words, a container is like a specific instance of an image.

Let’s Get Started

Note: I run Ubuntu 18.04, but this shouldn’t change anything.

Let’s make a local instance of Jenkins:

To start, optionally run: docker search jenkins (which pulls from https://hub.docker.com/_/jenkins/) to see what Docker images are out there to start. We are going to pick jenkins/jenkins:lts.

Next, run:

docker run --rm -it --name jenkins_medium_demo --publish 8080:8080 --publish 50000:50000 --volume jenkins_home:/var/jenkins_home jenkins/jenkins:lts /bin/bash

What is going on here:

• Docker run image_name : this spins up a container from an image (it will pull from hub.docker.com if we don’t have the image downloaded locally already).
• --rm: this is optional, but advised, as it removes the container after closing. I have found it very easy to forget to remove old containers after I’m done with them.
• -it … /bin/bash : this is technically two commands (effectively, it stands for ‘interactive terminal’), but together it allows us to access the running container interactively from terminal (in this case, using bash).
• --name : this names our running container, so we can easily refer to it later.
• --publish #A:#B: this publishes port #A of the container to port #B of your host computer. This will allow us to access the Jenkins GUI from our host machine by going to localhost:#B in our favorite web browser.
• --volume jenkins_home:/var/jenkins_home: this create a volume called jenkins_home and mount it to /var/jenkins_home inside the container.

Once we get it running, we can access the GUI by going to the url localhost:8080 on a browser on our host computer.

We should be prompted for a password, which we should be able to get 3 ways:

• The simplest way to get the password is we can look at the terminal for the container we are running, which should look something like this:

You know my one-time password now!

• We can attach to the container from another instance ( docker exec -it jenkins_medium_demo /bin/bash) , and run cat /var/jenkins_home/secrets/initialAdminPassword . Which is a process to access the password from inside the container.
• And lastly (this is not advised), we can run sudo cat /var/lib/docker/volumes/jenkins_home/_data/secrets/intialAdminPassword on our host-machine. Which is directly observing the secrets file by looking inside the container’s volume as it is stored on our host machine.

Now we set up our Jenkins container through the GUI (I recommend installing the default plugins).

Be careful not to sign away your first-born, while clicking through default-installs

Then you should load up to :

my password is password, hack me.

Now we can play!

At this point, we have a local instance of something that is all set up, and we can easily play with, without breaking things. I strongly advise “committing” the container here.

Docker commit is effectively the opposite of Docker run. Docker run takes an image and makes a container. Docker commit takes a container and makes an image. This means that next time you want to start running your container, you don’t have to start from scratch, you can start with all your credentials processed and installations installed.

In this demo, I accidentally did not make my user a part of the “docker” group; you shouldn’t use sudo here

We have just went through a basic process that should enable you to relatively easily set something up which you can play with, and not be afraid to muck up.

To Recap:

1. We pulled a Docker image.
2. We ran the Docker image interactively.
3. We interacted with the Docker container through a GUI in our browser.
4. And lastly we committed our container so we could access it again.

The core of the article is over, you’re free !

Optional: Running Docker inside Docker

If you’re interested, I want to go into a bit of an advanced tooling — in Jenkins, you might want to run a container inside a container if you want to test a Jenkins pipeline which runs a container. But even outside of the Jenkins use-case, we can imagine many use-cases for containers inside containers, like with testing container orchestration or a data-engineering pipeline.

Ok — Let’s get started again

In order for your Jenkins container to call Docker inside the container, we need to share both Docker and the Docker socket with the Jenkins container:

docker run -it --rm -p 8080:8080 -p 50000:50000 -v jenkins_home:/var/jenkins_home -v $(which docker):/usr/bin/docker -v /var/run/docker.sock:/var/run/docker.sock jenkins_post_setup:base-install

(I bolded the new parts.)

But, unfortunately, if we try to run Docker inside of our Jenkins container, we will have permissions issues.

Docker gives each container an easily written name if you don’t do it yourself. In this case, “silly_albattani”

Why? Because we are passing the Docker application and Docker socket into the Jenkins container, which the container does not have permissions to. We need to give the user in the docker container permissions to the resources we are passing in.

Specifically: we need to create a group inside Jenkins with the same group-id as docker.sock.

An important note: It is not enough to simply create a group “docker” and add the Jenkins user to it, because while the name “docker” might be the same, the group ids will almost certainly not be.

On the host computer: ls -l /var/run/docker.sock

here docker is group 128

In another terminal, attach to the Jenkins container as root:

1. docker exec -it -u root container_name /bin/bash
2. groupadd -g 128 docker :128 is the groupid of ‘docker’
3. usermod -a -G docker jenkins
4. Restart the jenkins container. But careful not to delete the container here — since you might be running with --rm flag. So, commit the existing container to an image, and rundocker run on that new image.

Now you can create Docker containers within your Docker containers. You can repeat this process, and make it turtles all the way down.