Skip to main content

Build and use security hardened images with TripleO

Starting to apply since Pike

Concept of security hardened images

Normally the images used for overcloud deployment in TripleO are not security hardened. It means, the images lack all the extra security measures needed to accomplish with ANSSI requirements. These extra measures are needed to deploy TripleO in environments where security is an important feature.
The following recommendations are given to accomplish with security guidelines:
  • ensure that /tmp is mounted on a separate volume or partition, and that it is mounted with rw,nosuid,nodev,noexec,relatime flags
  • ensure that /var, /var/log and /var/log/audit are mounted on separates volumes or partitions, and that are mounted with rw,relatime flags.
  • ensure that /home is mounted on a separate partition or volume, and that it is mounted with rw,nodev,relatime flags.
  • include extra kernel boot flag to enable auditing: add audit=1 to GRUB_CMDLINE_LINUX setting
  • disable kernel support for USB via bootloader configuration: add nousb to GRUB_CMDLINE_LINUX setting
  • remove unsecure boot flags: remove crashkernel=auto from GRUB_CMDLINE_LINUX setting
  • blacklist insecure modules, preventing those to be loaded: usb-storage, cramfs, freevxfs, jffs2, hfs, hfsplus, squashfs, udf, vfat
  • remove unsecure packages from the image, as they are installed by default: kdump (installed by kexec-tools) and telnet
  • add new package needed for security: screen
To achieve these settings, a new image can be generated and built by TripleO, using what we call "Security hardened image". The produced image will be stored on a new qcow2 file, and can be used to provision all the overcloud servers for TripleO.

How to build the security hardened image

Traditionally, TripleO ships with pre-defined images already built. For the RHEL image, it is available on rhosp-director-images package. However, it is also possible to build your own image, to adapt to your own needs. That is documented on http://tripleo.org/install/basic_deployment/basic_deployment_cli.html

How to build for CentOS 

Additional steps need to be performed in order to build the CentOS image:
  • Install the current-tripleo delorean repository and deps repository:
    sudo curl -L -o /etc/yum.repos.d/delorean.repo https://trunk.rdoproject.org/centos7-master/current-passed-ci/delorean.repo
    sudo curl -L -o /etc/yum.repos.d/delorean-current.repo https://trunk.rdoproject.org/centos7/current/delorean.repo
    sudo sed -i 's/\[delorean\]/\[delorean-current\]/' /etc/yum.repos.d/delorean-current.repo sudo /bin/bash -c "cat <<EOF>>/etc/yum.repos.d/delorean-current.repo includepkgs=diskimage-builder,instack,instack-undercloud,os-apply-config,os-collect-config,os-net-config,os-refresh-config,python-tripleoclient,openstack-tripleo-common*,openstack-tripleo-heat-templates,openstack-tripleo-image-elements,openstack-tripleo,openstack-tripleo-puppet-elements,openstack-puppet-modules,openstack-tripleo-ui,puppet-* EOF"
    sudo curl -L -o /etc/yum.repos.d/delorean-deps.repo https://trunk.rdoproject.org/centos7/delorean-deps.repo
  • If Ceph needed:
    sudo yum -y install --enablerepo=extras centos-release-ceph-jewel
    sudo sed -i -e 's%gpgcheck=.*%gpgcheck=0%' /etc/yum.repos.d/CentOS-Ceph-Jewel.repo
  • Export the environment variables:
    export DIB_YUM_REPO_CONF="/etc/yum.repos.d/delorean*"
    
  • If Ceph needed:
    export DIB_YUM_REPO_CONF="$DIB_YUM_REPO_CONF /etc/yum.repos.d/CentOS-Ceph-Jewel.repo"
    

How to build for RHEL

Additional steps need to be performed in order to build the RHEL image:
  • Get and export the local image that is going to be used:
    export DIB_LOCAL_IMAGE=rhel-guest-image-7.4.x86_64.qcow2
  • Register into the system:
    export REG_METHOD=portal export REG_USER="[your username]" export REG_PASSWORD="[your password]" # Find this with `sudo subscription-manager list --available` export REG_POOL_ID="[pool id]" export REG_REPOS="rhel-7-server-rpms rhel-7-server-extras-rpms rhel-ha-for-rhel-7-server-rpms \ rhel-7-server-optional-rpms rhel-7-server-openstack-12.0-rpms \
    [rhel-7-server-rhceph-2-mon-rpms rhel-7-server-rhceph-2-osd-rpms \
    rhel-7-server-rhceph-2-tools-rpms]"

Common steps

Basically you need to rely on openstack overcloud image build command, to build the image for your needs. In this case,  the security hardened images for the overcloud are contained in a config file called overcloud-hardened-images.yaml. After following all the process documented on OpenStack, you need to execute the command:
openstack overcloud image build --image-name overcloud-hardened-full --config-file /usr/share/openstack-tripleo-common/image-yaml/overcloud-hardened-images.yaml --config-file /usr/share/openstack-tripleo-common/image-yaml/overcloud-hardened-images-[centos7|rhel7].yaml

This will produce the overcloud-hardened-full.qcow2 image, that will contains all the features needed for security. Next you can upload the image to glance and start using it from TripleO.

How to customize the security hardened image

One of the main disadvantages of the security hardened images, is that the partitions for all the filesystems are pre-defined and hardcoded. The pre-defined sizes are:
  • /              -> 6g
  • /tmp           -> 1g
  • /var           -> 7g
  • /var/log       ->5g
  • /var/log/audit -> 900m
  • /home          -> 100m
 Composing an image of 20G . Although this may be ok for most of the deployments, it may be needed to alter this partitioning size, depending on the hardware or environmetn requirements. This is possible with two steps:
  • modify partitioning schema, exporting DIB_BLOCK_DEVICE_CONFIG
  • modify image global size, updating DIB_IMAGE_SIZE var

1. Modify partitioning schema

To modify the partitioning schema, either to alter the partitioning size or create/remove existing partitions, you need to execute: 

export DIB_BLOCK_DEVICE_CONFIG='<yaml_schema_with_partitions>'


Before executing the openstack overcloud image build command. The current YAML used to produce the security hardened image is the following, so you can reuse and update the sizes of the partitions as needed:

export DIB_BLOCK_DEVICE_CONFIG='''
- local_loop:
    name: image0
- partitioning:
    base: image0
    label: mbr
    partitions:
      - name: root
        flags: [ boot,primary ]
        size: 6G
        mkfs:
            type: xfs
            label: "img-rootfs"
            mount:
                mount_point: /
                fstab:
                    options: "rw,relatime"
                    fck-passno: 1
      - name: tmp
        size: 1G
        mkfs:
            type: xfs
            mount:
                mount_point: /tmp
                fstab:
                    options: "rw,nosuid,nodev,noexec,relatime"
      - name: var
        size: 7G
        mkfs:
            type: xfs
            mount:
                mount_point: /var
                fstab:
                    options: "rw,relatime"
      - name: log
        size: 5G
        mkfs:
            type: xfs
            mount:
                mount_point: /var/log
                fstab:
                    options: "rw,relatime"
      - name: audit
        size: 900M
        mkfs:
            type: xfs
            mount:
                mount_point: /var/log/audit
                fstab:
                    options: "rw,relatime"
      - name: home
        size: 100M
        mkfs:
            type: xfs
            mount:
                mount_point: /home
                fstab:
                    options: "rw,nodev,relatime"

'''

For a reference about the YAML schema, please visit https://docs.openstack.org/developer/diskimage-builder/user_guide/building_an_image.html

2. Update image size

Once you modify the partitioning schema, you may need to update the size of the generated image, because the global sum of partition sizes may exceed the one by default (20G). To modify the image size, you may need to update the config files generated to produce the image.
To achieve this , you need to make a copy of the /usr/share/openstack-tripleo-common/image-yaml/overcloud-hardened-images.yaml:

cp /usr/share/openstack-tripleo-common/image-yaml/overcloud-hardened-images.yaml /home/stack/overcloud-hardened-images-custom.yaml

Then you may need to edit the DIB_IMAGE_SIZE setting contained there, to give the right value to it:

...

environment:
DIB_PYTHON_VERSION: '2'
DIB_MODPROBE_BLACKLIST: 'usb-storage cramfs freevxfs jffs2 hfs hfsplus squashfs udf vfat bluetooth'
DIB_BOOTLOADER_DEFAULT_CMDLINE: 'nofb nomodeset vga=normal console=tty0 console=ttyS0,115200 audit=1 nousb'
DIB_IMAGE_SIZE: '20'
COMPRESS_IMAGE: '1'


After creating that new file, you can execute the image build command, pointing to that new generated file:

openstack overcloud image build --image-name overcloud-hardened-full --config-file /home/stack/overcloud-hardened-images-custom.yaml --config-file /usr/share/openstack-tripleo-common/image-yaml/overcloud-hardened-images-[centos7|rhel7].yaml

It is important to define the right partition sizes for the image, as it will be very hard to resize after deployment (it will need interactions with parted and xfs_growfs). No automatic growth of the filesystem is performed, so the partitions will be fixed independently of disk space, leaving the remaining space of the disk (if existing) not used.

How to upload the security hardened image

The generated image will be a whole disk one. It means, it will not have initrd and vmlinuz files, just the qcow2 one. So once, you have the security hardened image generated, overwrite the original overcloud-full.qcow2 image you had, with the newly generated once.
Then, to upload it, you  need to pass a special flag to the command: 

openstack overcloud image upload --whole-disk --image-path /home/stack/images --update-existing

This will overwrite the original overcloud-full.qcow2 image you had, with the new security hardened image you just generated.

Starting at that point, you can continue with the TripleO deployment as usual, and the deployed overcloud nodes will boot with the partitioned images, that will accomplish with the ANSSI guidelines for security hardening.

Comments

Popular posts from this blog

Describing TOAD - TripleO automated deployer

An overview of the TOAD framework and the advantagesWhat is TOAD? Fully automated deployment using Ansible (single command spin up)Main goal: to automate OSP deployments for continuous integration (CI) and development purposesTOAD is a CI framework using off-the-shelf components that many partners are familiar with:JenkinsJenkins Job Builder (JJB): http://docs.openstack.org/infra/jenkins-job-builder/TripleO Quickstart (oooq): https://www.rdoproject.org/tripleo/Optional ELK Stack (ElasticSearch, Logstash, Kibana) Its core component is TripleO Quickstart, used for TripleO upstream testingFully customizable with oooq settings; can be extendedDeploy environments with one click; trash after finished Of course it’s open source! :)https://github.com/redhat-nfvpe/toadhttps://github.com/redhat-nfvpe/jenkins-jobshttps://github.com/redhat-nfvpe/job-configs What components make up TOAD?Requirements to install TOADTwo different use cases: virtualized and baremetalOnly one Jenkins+Nginx VM needed t…

Start using whole disk images with TripleO

What are the differences between flat partition image and whole disk image? In order to understand this article, you first need to know what a flat partition image and a whole disk image are, and the differences between each other.
flat partition image: disk image that just contains all the desired content in a filesystem, but does not carry any information about partitions on it, and it does not include a bootloader. In order to boot from a whole disk image, the kernel and ramdisk images need to be passed independently when booting, relying on an external system to mount.whole disk image: image that contains all the information about partitions, bootloaders... as well as all the desired content. It can boot independently, without the need of external kernels or systems to mount it. Right now, OpenStack Ironic  supports both kind of images, but OpenStack TripleO was only supporting flat partition images.

TripleO added support for whole disk images Since python-tripleoclient 5.6.0 ver…

Automated OSP deployments with Tripleo Quickstart

In this article I'm going to show a method for automating OSP (RedHat OpenStack platform) deployments. These automated deployments can be very useful for CI, or simply to experiment and test with the system.
Components involvedTOAD: set of playbooks to deploy Jenkins, jenkins-job-builder and an optional ELK stack. This will install a ready to use system with all the preconfigured jobs (including OSP10 deployments and image building).TOAD jenkins-jobs: A set of job templates and macros, using jenkins-job-builder syntax, that get converted into Jenkins jobs for building the OSP base images and for deploying the system.TOAD job-configs: A set of job configurations to be used along with the jenkins-jobs repo. It provides a set of basic configs to build OpenStack using RDO or OSP.TripleO quickstart: set of ansible playbooks, used for building images and for RDO/OSP deployments. System requirementsOne VM to run a Jenkins master + nginx provided by TOAD. If you want to deploy additiona…