Skip to main content

Dynamically add Jenkins slaves using Kubernetes

CI is a very interesting area where Kubernetes can be applied. It's a common use case to have a demand of containers to be able to run tests of code changes, or to create some build artifacts.
In this post I will be explaining how to use Kubernetes to spin up Jenkins slaves and connect them to a Jenkins master, allowing to run tests on demand, and have a very easy and fast way to scale depending on our needs.

Pre-requisites needed on Jenkins master

Installing Jenkins plugin

This approach relies on Jenkins Swarm Plugin. So you need to ensure that you have installed this module on your master:
https://wiki.jenkins-ci.org/display/JENKINS/Swarm+Plugin

For Jenkins Swarm Plugin to work, you need to enable CSRF. You need to visit http://url.to.master/configureSecurity/ and check the Prevent Cross Site Request Forgery exploits option.


Configure security

Also consider that Jenkins uses a random port to enable connection to slaves using JNLP. If you are under a protected environment, you may desire to choose a fixed port and enable security rules for it. To achieve that, you need to visit the configureSecurity url again, and set up the port for Jenkins slaves:

After that, add the rules you need on your firewall, to allow connection between Jenkins master and slaves using that port.

Configure credentials

Jenkins Swarm plugin will use jenkins credentials, that needs to be passed to the slave, to connect to the master. So be sure to create some credentials for the Jenkins master domain, based on user/pass. You will need the https://wiki.jenkins-ci.org/display/JENKINS/Credentials+Plugin to achieve it.

How to create a Jenkins slave using Kubernetes

Create the image

First item that we need, is a docker image that is capable to act as a Jenkins Swarm slave, and capable to run some tests on it.
For this purpose, i've created a Docker image that is available at https://hub.docker.com/r/yrobla/jenkins-slave-swarm-infra/ *. This image is based on general Docker java images, using version 8. In top of that, we download the Jenkins Swarm client to be able to connect this node to the Jenkins master, and we also add some extra packages that are needed normally to peform python tests.

* Based on https://hub.docker.com/r/csanchez/jenkins-swarm-slave/

Integrate with Kubernetes

To use this image in Kubernetes to be able to scale on demand, a replication controller that uses this image needs to be created:

https://github.com/kubestack/kubestack/blob/master/jenkins/replication.json

You can create this replication controller using Kubernetes API:
export JENKINS_USER=jenkins_user
export JENKINS_PASS=xxxx
export JENKINS_LABEL=label

kubectl create -f ./replication.json 

* Please note that label is an optional parameter, very useful when you need to limit the jobs that run in this type of container.

Once the replication controller has been created, you will see a new jenkins slave connected to your jenkins master.

Scale it!

Now you can scale the number of your jenkins slaves on demand, using Kubernetes API:

kubectl scale --replicas=5 rc jenkins-slave
 

This command automatically creates and attaches 5 jenkins slaves to your master:

Using this technology gives you a very powerful and easy way to perform your tests or build your artifacts, with the only limit of the minions you are able to provide in your Kubernetes cluster.

Labels:

Comments

Post a Comment

Popular posts from this blog

Enable UEFI PXE boot in Supermicro SYS-E200

When provisioning my Supermicro SYS-E200-8D machines (X10 motherboard), i had the need to enable UEFI boot mode, and provision through PXE. This may seem straightforward, but there is a set of BIOS settings that need to be changed in order to enable it. First thing is to enable EFI on LAN , and enable Network Stack. To do that, enter into BIOS > Advanced > PCIe/PCI/PnP configuration and check that your settings match the following: See that PCI-E have EFI firmware loaded. Same for Onboard LAN OPROM and Onboard Video OPROM. And UEFI Network stack is enabled , as well as IPv4 PXE/IPv6 PXE support. Next thing is to modify boot settings. The usual boot order for PXE is to first add hard disk and second PXE network . The PXE tools (for example Ironic) will set a temporary boot order for PXE (one time) to enable the boot from network, but then the reboot will be done from hard disk. So be sure that your boot order matches the following: See that the first order is hard d...
1 comment
Read more

Test API endpoint with netcat

Do you need a simple way to validate that an API endpoint is responsive, but you don't want to use curl? There is a simple way to validate the endpoint with nc, producing an output that can be redirected to a logfile and parsed later: URL=$1 PORT=$2 while true; do     RESULT=$(nc -vz $URL $PORT 2>&1)     DATE=$(date)     echo $DATE $RESULT     sleep 1 done You can all this script with the API URL as first parameter, and API port as the second. netcat will be accessing to that endpoint and will report the results, detecting when the API is down. We also can output the date to have a reference when failures are detected. The produced output will be something like: vie jun 26 08:19:28 UTC 2020 Ncat: Version 7.70 ( https://nmap.org/ncat ) Ncat: Connected to 192.168.111.3:6443. Ncat: 0 bytes sent, 0 bytes received in 0.01 seconds. vie jun 26 08:19:29 UTC 2020 Ncat: Version 7.70 ( https://nmap.org/ncat ) Ncat: Connec...
1 comment
Read more

Create and restore external backups of virtual machines with libvirt

A common need for deployments in production, is to have the possibility of taking backups of your working virtual machines, and export them to some external storage. Although libvirt offers the possibility of taking snapshots and restore them, those snapshots are intended to be managed locally, and are lost when you destroy your virtual machines. There may be the need to just trash all your environment, and re-create the virtual machines from an external backup, so this article offers a procedure to achieve it. First step, create an external snapshot So the first step will be taking an snapshot from your running vm. The best way to take an isolated backup is using blockcopy virsh command. So, how to proceed? 1. First you need to extract all the disks that your vm has. This can be achieved with domblklist command:   DISK_NAME=$(virsh domblklist {{domain}} --details | grep 'disk' | awk '{print $3}') This will extract the name of the device that the vm is using ...
4 comments
Read more