Skip to main content

Automating local mirrors creation in RHEL



Sometimes there is a need to consume RHEL mirrors locally, not using the Red Hat content delivery network. It may be needed to speed up some deployment, or due to network constraints.

I create an ansible playbook, rhel-local-mirrors (https://github.com/redhat-nfvpe/rhel-local-mirrors), that can help with that.

What does rhel-local-mirrors do?

It is basically a tool that connects to the Red Hat CDN, and syncs the repositories locally, allowing to populate the desired mirrors, that can be accessed by other systems via HTTP.

The playbook is performing several tasks, that can be run together or independently:
  • register a system on the Red Hat Network
  • prepare the system to host mirrors
  • create the specified mirrors
  • schedule automatic updates of the mirrors

How to use it?

  • It is an Ansible playbook, so start by installing it, in any prefered format.
  • Then continue by cloning the playbook:
    git clone https://github.com/redhat-nfvpe/rhel-local-mirrors.git
  • This playbook expects a group of servers called rhel_mirrors to be created. So start creating that entry in /etc/ansible/hosts:
[rhel_mirrors]
127.0.0.1 ansible_connection=local
  •  This playbook ships with a default config, but it expects some values to be populated. It expects a file in ~/.ansible/vars/rhel_local_mirrors_vars.yml to be created, that will contain all  the settings that need to be customized. Following settings are mandatory:
    • rhn_subscription_username
    • rhn_subscription_password
    • rhn_subscription_pool_id
Once you have these files created, you simply can execute the playbook with:
ansible-playbook  -vvvv site.yml -i /etc/ansible/hosts 

With these settings populated, the playbook will just create a mirror called rhel.repo, that will contain rhel-7-server-rpms, rhel-7-server-extras-rpms, rhel-7-server-rh-common-rpms, rhel-ha-for-rhel-7-server-rpms repos. It will be placed on /var/www/mirrors/rhel_repo folder, and can be consumed externally by all systems that have HTTP access to your host.

Steps and customization

Register a system in the local mirror

In order to access the content from the Red Hat CDN, the instance where you run this playbook needs to be registered on the system. You need to define the following vars in ansible, that need to match with the data from your RHEL account:
  • rhn_subscription_username
  • rhn_subscription_password
  • rhn_subscription_pool_id
If you manage subscriptions on your own, you can disable this step by setting the subscribe_rhn var to False.

This task will collect all the repositories that you defined for your mirrors and will enable those on your system.

To limit the playbook just to execute this register step, you can run it with: --tags rhel_register.

Prepare the system to host mirrors

This step will install all the packages needed to perform the mirror creation ( createrepo and yum-utils packages), and will install and configure the nginx service that will help to populate the mirrors and serve them via HTTP. It will also open the HTTP port in your system, if you have iptables or firewalld active.
To limit the playbook just to execute this register step, you can run it with: --tags prepare_system.

Mirror creation

This step will perform the actual mirror creation, based on your settings. You can define several mirrors and the playbook will automate the creation for all of those independently.
To define the mirrors, you need to use the mirrors var, that expects a list of dictionaries with the name, folder and items keys:

mirrors:
  - name: osp8.repo
    folder: osp8_repo
    items:
        - rhel-7-server-rpms
        - rhel-7-server-extras-rpms
        - rhel-7-server-rh-common-rpms
        - rhel-ha-for-rhel-7-server-rpms
        - rhel-7-server-openstack-8-rpms
        - rhel-7-server-openstack-8-director-rpms
        - rhel-7-server-rhceph-2-osd-rpms
        - rhel-7-server-rhceph-2-mon-rpms
        - rhel-7-server-rhceph-2-tools-rpms
  - name: osp9.repo
    folder: osp9_repo
    items:
        - rhel-7-server-rpms
        - rhel-7-server-extras-rpms
        - rhel-7-server-rh-common-rpms
        - rhel-ha-for-rhel-7-server-rpms
        - rhel-7-server-openstack-9-rpms
        - rhel-7-server-openstack-9-director-rpms
        - rhel-7-server-rhceph-2-osd-rpms
        - rhel-7-server-rhceph-2-mon-rpms
        - rhel-7-server-rhceph-2-tools-rpms
  - name: osp10.repo
    folder: osp10_repo
    items:
        - rhel-7-server-rpms
        - rhel-7-server-extras-rpms
        - rhel-7-server-rh-common-rpms
        - rhel-ha-for-rhel-7-server-rpms
        - rhel-7-server-openstack-10-rpms
        - rhel-7-server-rhceph-2-osd-rpms
        - rhel-7-server-rhceph-2-mon-rpms
        - rhel-7-server-rhceph-2-tools-rpms


After running this task, you will end with a /var/www/mirrors/general_mirror folder (that will contain all the repositories defined for all mirrors), and a set of individual folders (/var/www/mirrors/<<mirror.folder>>), that will contain just the defined repos for each mirror, along with a <<mirror.name>> file, that will define the contents of the repository, and that you can reuse to consume these mirrors from outer systems:

|--- /var/www/mirrors
|--- |--- general_mirror
|--- |--- osp8_repo
|--- |--- |--- osp8.repo
|--- |--- osp9_repo
|--- |--- |--- osp9.repo
|--- |--- osp10_repo
|--- |--- |--- osp10.repo

A sample <<mirror.name>>.repo file will have this content:

[rhel-7-server-extras-rpms]
name=rhel-7-server-extras-rpms
baseurl=http://10.10.0.13/pub/osp10_repo/rhel-7-server-extras-rpms/
enabled=1
gpgcheck=1
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-redhat-release


[rhel-7-server-openstack-10-rpms]
name=rhel-7-server-openstack-10-rpms
baseurl=http://10.10.0.13/pub/osp10_repo/rhel-7-server-openstack-10-rpms/
enabled=1
gpgcheck=1
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-redhat-release


[rhel-7-server-openstack-8-director-rpms]
name=rhel-7-server-openstack-8-director-rpms
baseurl=http://10.10.0.13/pub/osp10_repo/rhel-7-server-openstack-8-director-rpms/
enabled=0
gpgcheck=1
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-redhat-release


[rhel-7-server-openstack-8-rpms]
name=rhel-7-server-openstack-8-rpms
baseurl=http://10.10.0.13/pub/osp10_repo/rhel-7-server-openstack-8-rpms/
enabled=0
gpgcheck=1
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-redhat-release


[rhel-7-server-openstack-9-director-rpms]
name=rhel-7-server-openstack-9-director-rpms
baseurl=http://10.10.0.13/pub/osp10_repo/rhel-7-server-openstack-9-director-rpms/
enabled=0
gpgcheck=1
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-redhat-release


[rhel-7-server-openstack-9-rpms]
name=rhel-7-server-openstack-9-rpms
baseurl=http://10.10.0.13/pub/osp10_repo/rhel-7-server-openstack-9-rpms/
enabled=0
gpgcheck=1
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-redhat-release


[rhel-7-server-rh-common-rpms]
name=rhel-7-server-rh-common-rpms
baseurl=http://10.10.0.13/pub/osp10_repo/rhel-7-server-rh-common-rpms/
enabled=1
gpgcheck=1
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-redhat-release


[rhel-7-server-rhceph-2-mon-rpms]
name=rhel-7-server-rhceph-2-mon-rpms
baseurl=http://10.10.0.13/pub/osp10_repo/rhel-7-server-rhceph-2-mon-rpms/
enabled=1
gpgcheck=1
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-redhat-release


[rhel-7-server-rhceph-2-osd-rpms]
name=rhel-7-server-rhceph-2-osd-rpms
baseurl=http://10.10.0.13/pub/osp10_repo/rhel-7-server-rhceph-2-osd-rpms/
enabled=1
gpgcheck=1
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-redhat-release


[rhel-7-server-rhceph-2-tools-rpms]
name=rhel-7-server-rhceph-2-tools-rpms
baseurl=http://10.10.0.13/pub/osp10_repo/rhel-7-server-rhceph-2-tools-rpms/
enabled=1
gpgcheck=1
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-redhat-release


[rhel-7-server-rpms]
name=rhel-7-server-rpms
baseurl=http://10.10.0.13/pub/osp10_repo/rhel-7-server-rpms/
enabled=1
gpgcheck=1
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-redhat-release


[rhel-ha-for-rhel-7-server-rpms]
name=rhel-ha-for-rhel-7-server-rpms
baseurl=http://10.10.0.13/pub/osp10_repo/rhel-ha-for-rhel-7-server-rpms/
enabled=1
gpgcheck=1
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-redhat-release


To limit the playbook just to execute this mirror creation step, you can run it with: --tags create_mirror.

Repository synchronization 

When  using local repositories, it is important to have them on sync. This playbook offers the possibility of setting up a cronjob that will trigger the sync automatically, on the desired period. To enable it, you need to set the repo_autosync var to True. It then will use the following vars:
  • crontab_day
  • crontab_hour
  • crontab_minute
  • crontab_month
  • crontab_weekday
It will install a cronjob that will trigger reposync for all the defined repositories, with the settings specified in the crontab_* vars. You can always remove the cron, running the playbook with repo_autosync var set to False.

To limit the playbook just to execute this repo synchronization step, you can run it with: --tags prepare_cron.

Contributing

Several features can be added to this playbook, such as improving the registration system, or adding more features to mirror creation.
If you want to contribute, please send your pull requests to https://github.com/redhat-nfvpe/rhel-local-mirrors

Comments

Popular posts from this blog

Describing TOAD - TripleO automated deployer

An overview of the TOAD framework and the advantagesWhat is TOAD? Fully automated deployment using Ansible (single command spin up)Main goal: to automate OSP deployments for continuous integration (CI) and development purposesTOAD is a CI framework using off-the-shelf components that many partners are familiar with:JenkinsJenkins Job Builder (JJB): http://docs.openstack.org/infra/jenkins-job-builder/TripleO Quickstart (oooq): https://www.rdoproject.org/tripleo/Optional ELK Stack (ElasticSearch, Logstash, Kibana) Its core component is TripleO Quickstart, used for TripleO upstream testingFully customizable with oooq settings; can be extendedDeploy environments with one click; trash after finished Of course it’s open source! :)https://github.com/redhat-nfvpe/toadhttps://github.com/redhat-nfvpe/jenkins-jobshttps://github.com/redhat-nfvpe/job-configs What components make up TOAD?Requirements to install TOADTwo different use cases: virtualized and baremetalOnly one Jenkins+Nginx VM needed t…

Start using whole disk images with TripleO

What are the differences between flat partition image and whole disk image? In order to understand this article, you first need to know what a flat partition image and a whole disk image are, and the differences between each other.
flat partition image: disk image that just contains all the desired content in a filesystem, but does not carry any information about partitions on it, and it does not include a bootloader. In order to boot from a whole disk image, the kernel and ramdisk images need to be passed independently when booting, relying on an external system to mount.whole disk image: image that contains all the information about partitions, bootloaders... as well as all the desired content. It can boot independently, without the need of external kernels or systems to mount it. Right now, OpenStack Ironic  supports both kind of images, but OpenStack TripleO was only supporting flat partition images.

TripleO added support for whole disk images Since python-tripleoclient 5.6.0 ver…

Automated OSP deployments with Tripleo Quickstart

In this article I'm going to show a method for automating OSP (RedHat OpenStack platform) deployments. These automated deployments can be very useful for CI, or simply to experiment and test with the system.
Components involvedTOAD: set of playbooks to deploy Jenkins, jenkins-job-builder and an optional ELK stack. This will install a ready to use system with all the preconfigured jobs (including OSP10 deployments and image building).TOAD jenkins-jobs: A set of job templates and macros, using jenkins-job-builder syntax, that get converted into Jenkins jobs for building the OSP base images and for deploying the system.TOAD job-configs: A set of job configurations to be used along with the jenkins-jobs repo. It provides a set of basic configs to build OpenStack using RDO or OSP.TripleO quickstart: set of ansible playbooks, used for building images and for RDO/OSP deployments. System requirementsOne VM to run a Jenkins master + nginx provided by TOAD. If you want to deploy additiona…