Building container images from Jenkins using buildah (part 1)

Sep 12, 2021 · 4 min read

Jenkins is considered by most people the best automation server around, it can automate almost any task using its plugins: can connect to any ssh capable hosts and run commands, execute ansible playbooks, run maven command, push packages or images to a remote repository, etc.

jenkins image

But this time, I focused on using jenkins to trigger container image builds.

Deploying Jenkins

Jenkins came packaged as docker image so it is very easy to run, the only thing needed was a place to save the data:

  jenkins:
    container_name: jenkins
    image: jenkins/jenkins
    volumes:
      - "$PWD/data:/var/jenkins_home"
    ports:
      - "8080:8080"

Once the container was deployed, I pointed my browser to http://localhost:8080 and I was asked for the initial password, than could be found in the secrets/initialAdminPassword file inside the mounted volume.

On the initial configuration wizard, I selected several plugins in order to test them, but the important here was SSH Build Agents plugin which allowed running jenkins agents using ssh connections.

Deploying a buildah container

I wanted to try buildah for building container image, because it supported building images without accessing the docker daemon. Buildah also came prepackaged as docker image, but the official image lacked the ssh server, so I had to build a custom image:

FROM quay.io/buildah/stable:latest
RUN dnf update -y \
    && dnf install -y openssh-server java-1.8.0-openjdk-devel java-1.8.0-openjdk\
    && dnf clean all \
    && install -d /home/build/.ssh --mode 700 --owner build --group build\
    && install -d /home/build/workspace --mode 700 --owner build --group build\
    && /usr/libexec/openssh/sshd-keygen ecdsa \
    && /usr/libexec/openssh/sshd-keygen rsa

EXPOSE 22
ENTRYPOINT ["/usr/sbin/sshd"]
CMD ["-D", "-e"]

DISCLAIMER: This is not ready for production in an exposed environment, as anybody can see, the host key are stored on the image itself, so all instances would use the same keys.

The Dockerfile updates all packages and installs the required ones, in this case the openssh server and the java environment.

The buildah image came with a builtin user called build, but I had to create the .ssh directory. I used install command to ensure proper permissions and owner were set.

$ docker build -t juanjovlc2/sshbuildah:dev .
Sending build context to Docker daemon  2.048kB
Step 1/5 : FROM quay.io/buildah/stable:latest
 ---> 203eb35eefff
Step 2/5 : RUN dnf update -y     && dnf install -y openssh-server java-1.8.0-openjdk-devel java-1.8.0-openjdk    && dnf clean all     && install -d /home/build/.ssh --mode 700 --owner build --group build    && install -d /home/build/workspace --mode 700 --owner build --group build    && /usr/libexec/openssh/sshd-keygen ecdsa     && /usr/libexec/openssh/sshd-keygen rsa
 ---> Using cache
 ---> 0c023031c140
Step 3/5 : EXPOSE 22
 ---> Using cache
 ---> b263d2df6d40
Step 4/5 : ENTRYPOINT ["/usr/sbin/sshd"]
 ---> Using cache
 ---> b95cb64ffbe8
Step 5/5 : CMD ["-D", "-e"]
 ---> Running in 42a50f246031
Removing intermediate container 42a50f246031
 ---> 013826d3cb2a
Successfully built 013826d3cb2a
Successfully tagged juanjovlc2/sshbuildah:dev

After building and uploading it, it was ready to be used, so I added another service to the docker-compose file:

  buildah:
    container_name: buildah
    image: juanjovlc2/sshbuildah:dev
    volumes:
      - "$PWD/buildah/build:/home/build"

I populated the directory with a .ssh/authorized_keys file with the proper owner’s uid: 1000 and gid: 1000, and the 600 file mode.

Configuring the node as jenkins agent

Once the new container was running after issuing a docker-compose up -d command, I went to the Manage Jenkins page on the UI and selected the Manage nodes and clouds option. Then New Node from the left menu.

I entered a name and marked it as Permanent and hit enter. In the next screen there were some important fields under the Launch method section.

configuration form

The launch method should be set to Launch agents via SSH and then new fields for host and credentials are shown. Clicking on the Add button another dialog appeared and allowed me to enter the credentials.

credentials dialog

As the image was using ssh keys authentication, I selected “SSH Username with private key” and entered the username and pasted the private key which pairs with the public key added to buildah container as authorized key file.

After saving all dialogs, I received a connection error, good thing there is a “Logs” button which allowed me see what was the problem. It was regarding the known_hosts file was missing. I created it and the error changed to “Host not in known_hosts file”, so I entered the jenkins container and tried to connect in order to generate the known_hosts file:

juanjo@Anthrax jenkins $ docker exec -it jenkins bash
jenkins@6450f72c147e:/$ ssh buildah
The authenticity of host 'buildah (172.21.0.3)' can't be established.
ECDSA key fingerprint is SHA256:tl/B9ERUksMT8NL21jizHLq5fUKJbPkslt6I6wLlwBA.
Are you sure you want to continue connecting (yes/no/[fingerprint])? yes
Warning: Permanently added 'buildah,172.21.0.3' (ECDSA) to the list of known hosts.
jenkins@buildah's password:

Once the host was added, the agent could be started.

Later I realized there were several options on the Host Key Verification Strategy, and one of them was Non Verifying Verification Strategy, I suspect that would saved me a bit of headache.

Conclusions

This work is not completed, I haven’t build an image using buildah yet, but it has introduced me into jenkins and also allowed me to practice a bit with Dockerfiles, building images, pushing the to registries and a bit of problem solving.

More to come…