Skip to main content

Dynamically add Jenkins slaves using Kubernetes

CI is a very interesting area where Kubernetes can be applied. It's a common use case to have a demand of containers to be able to run tests of code changes, or to create some build artifacts.
In this post I will be explaining how to use Kubernetes to spin up Jenkins slaves and connect them to a Jenkins master, allowing to run tests on demand, and have a very easy and fast way to scale depending on our needs.

Pre-requisites needed on Jenkins master

Installing Jenkins plugin

This approach relies on Jenkins Swarm Plugin. So you need to ensure that you have installed this module on your master:
https://wiki.jenkins-ci.org/display/JENKINS/Swarm+Plugin

For Jenkins Swarm Plugin to work, you need to enable CSRF. You need to visit http://url.to.master/configureSecurity/ and check the Prevent Cross Site Request Forgery exploits option.


Configure security

Also consider that Jenkins uses a random port to enable connection to slaves using JNLP. If you are under a protected environment, you may desire to choose a fixed port and enable security rules for it. To achieve that, you need to visit the configureSecurity url again, and set up the port for Jenkins slaves:

After that, add the rules you need on your firewall, to allow connection between Jenkins master and slaves using that port.

Configure credentials

Jenkins Swarm plugin will use jenkins credentials, that needs to be passed to the slave, to connect to the master. So be sure to create some credentials for the Jenkins master domain, based on user/pass. You will need the https://wiki.jenkins-ci.org/display/JENKINS/Credentials+Plugin to achieve it.

How to create a Jenkins slave using Kubernetes

Create the image

First item that we need, is a docker image that is capable to act as a Jenkins Swarm slave, and capable to run some tests on it.
For this purpose, i've created a Docker image that is available at https://hub.docker.com/r/yrobla/jenkins-slave-swarm-infra/ *. This image is based on general Docker java images, using version 8. In top of that, we download the Jenkins Swarm client to be able to connect this node to the Jenkins master, and we also add some extra packages that are needed normally to peform python tests.

* Based on https://hub.docker.com/r/csanchez/jenkins-swarm-slave/

Integrate with Kubernetes

To use this image in Kubernetes to be able to scale on demand, a replication controller that uses this image needs to be created:

https://github.com/kubestack/kubestack/blob/master/jenkins/replication.json

You can create this replication controller using Kubernetes API:
export JENKINS_USER=jenkins_user
export JENKINS_PASS=xxxx
export JENKINS_LABEL=label

kubectl create -f ./replication.json 

* Please note that label is an optional parameter, very useful when you need to limit the jobs that run in this type of container.

Once the replication controller has been created, you will see a new jenkins slave connected to your jenkins master.

Scale it!

Now you can scale the number of your jenkins slaves on demand, using Kubernetes API:

kubectl scale --replicas=5 rc jenkins-slave
 

This command automatically creates and attaches 5 jenkins slaves to your master:

Using this technology gives you a very powerful and easy way to perform your tests or build your artifacts, with the only limit of the minions you are able to provide in your Kubernetes cluster.

Comments

Post a Comment

Popular posts from this blog

Setup an NFS client provisioner in Kubernetes

Setup an NFS client provisioner in Kubernetes One of the most common needs when deploying Kubernetes is the ability to use shared storage. While there are several options available, one of the most commons and easier to setup is to use an NFS server.
This post will explain how to setup a dynamic NFS client provisioner on Kubernetes, relying on an existing NFS server on your systems.
Step 1. Setup an NFS server (sample for CentOS) First thing you will need, of course, is to have an NFS server. This can be easily achieved with some easy steps:

Install nfs package: yum install -y nfs-utils Enable and start nfs service and rpcbind:
systemctl enable rpcbind
systemctl enable nfs-server
systemctl start rpcbind
systemctl start nfs-server
Create the directory that will be shared by NFS, and change the permissions:
mkdir /var/nfsshare
chmod -R 755 /var/nfsshare
chown nfsnobody:nfsnobody /var/nfsshare
 Share the NFS directory over the network, creating the /etc/exports file:
vi /etc/exports
/var/nfsshare …

Create and restore external backups of virtual machines with libvirt

A common need for deployments in production, is to have the possibility of taking backups of your working virtual machines, and export them to some external storage.
Although libvirt offers the possibility of taking snapshots and restore them, those snapshots are intended to be managed locally, and are lost when you destroy your virtual machines.
There may be the need to just trash all your environment, and re-create the virtual machines from an external backup, so this article offers a procedure to achieve it.
First step, create an external snapshot So the first step will be taking an snapshot from your running vm. The best way to take an isolated backup is using blockcopy virsh command. So, how to proceed?

1. First you need to extract all the disks that your vm has. This can be achieved with domblklist command:
DISK_NAME=$(virsh domblklist {{domain}} --details | grep 'disk' | awk '{print $3}')

This will extract the name of the device that the vm is using (vda, hda, et…

Automating local mirrors creation in RHEL

Sometimes there is a need to consume RHEL mirrors locally, not using the Red Hat content delivery network. It may be needed to speed up some deployment, or due to network constraints.

I create an ansible playbook, rhel-local-mirrors (https://github.com/redhat-nfvpe/rhel-local-mirrors), that can help with that.
What does rhel-local-mirrors do? It is basically a tool that connects to the Red Hat CDN, and syncs the repositories locally, allowing to populate the desired mirrors, that can be accessed by other systems via HTTP.

The playbook is performing several tasks, that can be run together or independently:
register a system on the Red Hat Networkprepare the system to host mirrorscreate the specified mirrorsschedule automatic updates of the mirrors How to use it?It is an Ansible playbook, so start by installing it, in any prefered format. Then continue by cloning the playbook:
git clone https://github.com/redhat-nfvpe/rhel-local-mirrors.gitThis playbook expects a group of servers called