Start using whole disk images with TripleO

What are the differences between flat partition image and whole disk image?

In order to understand this article, you first need to know what a flat partition image and a whole disk image are, and the differences between each other.

• flat partition image: disk image that just contains all the desired content in a filesystem, but does not carry any information about partitions on it, and it does not include a bootloader. In order to boot from a whole disk image, the kernel and ramdisk images need to be passed independently when booting, relying on an external system to mount.
• whole disk image: image that contains all the information about partitions, bootloaders... as well as all the desired content. It can boot independently, without the need of external kernels or systems to mount it.

Right now, OpenStack Ironic  supports both kind of images, but OpenStack TripleO was only supporting flat partition images.

TripleO added support for whole disk images

Since python-tripleoclient 5.6.0 version, TripleO supports the upload and usage of whole disk images. It will land officially in Ocata release.
Right now, the overcloud image upload command is performing the following steps:

1. Upload overcloud-full.qcow2 image
2. Upload vmlinuz and initrd related files
3. Add a property on glance to overcloud-full image, to associate that with target vmlinuz and initrd files.

This approach is the desired one for flat partition images, because we need independent vmlinuz and initrd files to boot. But in the case of whole disk images, we just need the qcow2 file itself.

So a new flag has been added to the overcloud image upload command,  called --whole-disk, that will just upload the qcow2 image, skipping all the additional steps. The following command can be used to upload whole disk images:

openstack overcloud image upload --whole-disk

How to generate whole disk images

Now that TripleO can use whole disk images, there needs to be a way of building them. The standard TripleO overcloud image, is just a flat partition image. And although there are efforts in diskimage-builder project to produce whole images with the right partitions and volumes, this is Work in Progress and will not land in Ocata.
A simple alternative can be using guestfish to convert the flat partition overcloud image to a whole disk one. With a simple scripts that takes the initial image, and adds partitions on demand, and installs bootloader, you can produce a whole disk image that will be ready to use in TripleO.

A sample script will look like this:

#!/usr/bin/env python
import guestfs
import os

# remove old generated drive
try:
    os.unlink("/tmp/overcloud-full-partitioned.qcow2")
except:
    pass

g = guestfs.GuestFS(python_return_dict=True)

# import old and new images
print("Creating new repartitioned image")
g.add_drive_opts("/tmp/overcloud-full.qcow2", format="qcow2", readonly=1)
g.disk_create("/tmp/overcloud-full-partitioned.qcow2", "qcow2", 10.2 * 1024 * 1024 * 1024) #10.1G
g.add_drive_opts("/tmp/overcloud-full-partitioned.qcow2", format="qcow2", readonly=0)
g.launch()

# create the partitions for new image
print("Creating the initial partitions")
g.part_init("/dev/sdb", "mbr")
g.part_add("/dev/sdb", "primary", 2048, 616448)
g.part_add("/dev/sdb", "primary", 616449, -1)

g.pvcreate("/dev/sdb2")
g.vgcreate("vg", ['/dev/sdb2', ])
g.lvcreate("var", "vg", 5 * 1024)
g.lvcreate("tmp", "vg", 500)
g.lvcreate("swap", "vg", 250)
g.lvcreate("home", "vg", 100)
g.lvcreate("root", "vg", 4 * 1024)
g.part_set_bootable("/dev/sdb", 1, True)

# add filesystems to volumes
print("Adding filesystems")
ids = {}
keys = [ 'var', 'tmp', 'swap', 'home', 'root' ]
volumes = ['/dev/vg/var', '/dev/vg/tmp', '/dev/vg/swap', '/dev/vg/home', '/dev/vg/root']
swap_volume = volumes[2]

count = 0
for volume in volumes:
    if count!=2:
        g.mkfs('ext4', volume)
        ids[keys[count]] = g.vfs_uuid(volume)
    count +=1

# create filesystem on boot and swap
g.mkfs('ext4', '/dev/sdb1')
g.mkswap_opts(volumes[2])
ids['swap'] = g.vfs_uuid(volumes[2])

# mount drives and copy content
print("Start copying content")
g.mkmountpoint('/old')
g.mkmountpoint('/root')
g.mkmountpoint('/boot')
g.mkmountpoint('/home')
g.mkmountpoint('/var')
g.mount('/dev/sda', '/old')

g.mount('/dev/sdb1', '/boot')
g.mount(volumes[4], '/root')
g.mount(volumes[3], '/home')
g.mount(volumes[0], '/var')

# copy content to root
results = g.ls('/old/')
for result in results:
    if result not in ('boot', 'home', 'tmp', 'var'):
        print("Copying %s to root" % result)
        g.cp_a('/old/%s' % result, '/root/')

# copy extra content
folders_to_copy = ['boot', 'home', 'var']
for folder in folders_to_copy:
    results = g.ls('/old/%s/' % folder)
    for result in results:
        print("Copying %s to %s" % (result, folder))
        g.cp_a('/old/%s/%s' % (folder, result),
               '/%s/' % folder)

# create /etc/fstab file
print("Generating fstab content")
fstab_content = """
UUID={boot_id} /boot ext4 defaults 0 2
UUID={root_id} / ext4 defaults 0 1
UUID={swap_id} none swap sw 0 0
UUID={tmp_id} /tmp ext4 defaults 0 2
UUID={home_id} /home ext4 defaults 0 2
UUID={var_id} /var ext4 defaults 0 2
""".format(
    boot_id=g.vfs_uuid('/dev/sdb1'),
    root_id=ids['root'],
    swap_id=ids['swap'],
    tmp_id=ids['tmp'],
    home_id=ids['home'],
    var_id=ids['var'])

g.write('/root/etc/fstab', fstab_content)


# unmount filesystems
g.umount('/root')
g.umount('/boot')
g.umount('/old')
g.umount('/var')

# mount in the right directories to install bootloader
print("Installing bootloader")
g.mount(volumes[4], '/')
g.mkdir('/boot')
g.mkdir('/var')
g.mount('/dev/sdb1', '/boot')
g.mount(volumes[0], '/var')




# do a selinux relabel
g.selinux_relabel('/etc/selinux/targeted/contexts/files/file_contexts', '/', force=True)
g.selinux_relabel('/etc/selinux/targeted/contexts/files/file_contexts', '/var', force=True)


g.sh('grub2-install --target=i386-pc /dev/sdb')
g.sh('grub2-mkconfig -o /boot/grub2/grub.cfg')

# create dracut.conf file
dracut_content = """
add_dracutmodules+="lvm crypt"
"""
g.write('/etc/dracut.conf', dracut_content)

# update initramfs to include lvm and crypt
kernels = g.ls('/lib/modules')
for kernel in kernels:
    print("Updating dracut to include modules in kernel %s" % kernel)
    g.sh('dracut -f /boot/initramfs-%s.img %s --force' % (kernel, kernel))
g.umount('/boot')
g.umount('/var')
g.umount('/')

# close images
print("Finishing image")
g.shutdown()
g.close()
This sample script will create a whole disk image with the following steps:

1. Open the old overcloud-full image (flat partition) to use it as a base to generate new one
2. Create a new image (the whole one), with the desired size (for example 10gb)
3. Create partitions and volumes on the new image. Note that you can create the partitions and volumes you desire, with the sizes that match your environment. In the example we create an isolated partition for boot, and we go with volumes for the other content in the filesystem.
4. Create the initial filesystems on the partitions and volumes. You can add ext4, xfs, swap partitions... depending on your needs
5. Mount filesystems and start copying content to the right partitions from the origin to the target image. See that the target partition is mounted as /root, not as / , because it will give naming conflicts when moving old content to new if not.
6. Generate and copy the /etc/fstab  to the target image. You need to capture the UUID of the generated filesystems properly, and reflect on the generated /etc/fstab file
7. Unmount all the filesystems, either in the old and the new image
8. Mount the partitions only in the target image now. Do it with the right naming (root under /, and use the right /boot and /var partitions as well)
9. Install the bootloader properly. The best way to do it is running a shell on the chroot that generates the image, calling grub2-install and grub2-mkconfig . This will install grub2 bootloader in the whole disk image
10. Unmount all the filesystems and close the image
11. After that you will have a overcloud-full-partitioned image, that will be ready to use on TripleO deployments

Thanks to Pino Toscano and Richard Jones for really good advices on the guestfs process.