In this article we will show you how to install OpenShift in mutliple nodes using a quick install command, atomic-openshift-installer, which is powered by ansible.

Host preparation

I use under virtual machines for OpenShift nodes to deploy

Type CPU Mem HDD hostname OS
Master 1 2 GB 20 GB master.example.com RHEL 7
node 1 2 GB 20 GB node1.example.com RHEL 7
node 1 2 GB 20 GB node2.example.com RHEL 7

Host Registration

Note: If you are using other OS but not RHEL, Please go to # Install necessary packages to install the packages. If you can not install some of them, try to add some repos or get the rpm file.

Register each host with RHSM(Red Hat Subscription Manager) to access the required packages.

  1. Register with RHSM for each host:

    1

    # subscription-manager register --username=<user_name> --password=<password>

  2. List the available OpenShift subscriptions:

    1

    # subscription-manager list --available --matches '*OpenShift*'

  3. Find pool ID for an OpenShift Container Platform subscription and attach it.

    1

    # subscription-manager attach --pool=<pool_id>

  4. Disable all repositories and enable only the repositories required by OpenShift Container Platform 3.5

    1
2
3
4
5
6
7

    # subscription-manager repos --disable="*"
# yum-config-manager --disable \*
# subscription-manager repos \
   --enable="rhel-7-server-rpms" \
   --enable="rhel-7-server-extras-rpms" \
   --enable="rhel-7-server-ose-3.5-rpms" \
   --enable="rhel-7-fast-datapath-rpms"

Install necessary packages

Install the following packages.

1
2
3
4

   # yum -y install wget git net-tools bind-utils iptables-services bridge-utils bash-completion kexec sos psacct
   # yum update
   # yum -y install atomic-openshift-utils atomic-openshift-excluder atomic-openshift-docker-excluder
   # atomic-openshift-excluder unexclude

Install and configure docker

Install docker

1

   # yum -y install docker

Add parameter to docker configuration file

Edit /etc/sysconfig/docker file and add --insecure-registry 172.30.0.0/16 to the OPTIONS parameter.

1

   OPTIONS='--selinux-enabled --insecure-registry 172.30.0.0/16'

Configure Docker Storage

Here we use an additional block device for docker storage. In /etc/sysconfig/docker-storage-setup , set DEVS to the path of the disk device. Set VG to the volume group name you wish to create.

1
2
3
4

   # cat <<EOF > /etc/sysconfig/docker-storage-setup
   DEVS=/dev/vdc
   VG=docker-vg
   EOF

Then run docker-storage-setup and check to make sure the docker-vg was created.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39

   # docker-storage-setup                                                                                                                                                                                                                                [5/1868]
   0
   Checking that no-one is using this disk right now ...
   OK
   
   Disk /dev/vdc: 31207 cylinders, 16 heads, 63 sectors/track
   sfdisk:  /dev/vdc: unrecognized partition table type
   
   Old situation:
   sfdisk: No partitions found
   
   New situation:
   Units: sectors of 512 bytes, counting from 0
   
      Device Boot    Start       End   #sectors  Id  System
   /dev/vdc1          2048  31457279   31455232  8e  Linux LVM
   /dev/vdc2             0         -          0   0  Empty
   /dev/vdc3             0         -          0   0  Empty
   /dev/vdc4             0         -          0   0  Empty
   Warning: partition 1 does not start at a cylinder boundary
   Warning: partition 1 does not end at a cylinder boundary
   Warning: no primary partition is marked bootable (active)
   This does not matter for LILO, but the DOS MBR will not boot this disk.
   Successfully wrote the new partition table
   
   Re-reading the partition table ...
   
   If you created or changed a DOS partition, /dev/foo7, say, then use dd(1)
   to zero the first 512 bytes:  dd if=/dev/zero of=/dev/foo7 bs=512 count=1
   (See fdisk(8).)
     Physical volume "/dev/vdc1" successfully created
     Volume group "docker-vg" successfully created
     Rounding up size to full physical extent 16.00 MiB
     Logical volume "docker-poolmeta" created.
     Logical volume "docker-pool" created.
     WARNING: Converting logical volume docker-vg/docker-pool and docker-vg/docker-poolmeta to pool's data and metadata volumes.
     THIS WILL DESTROY CONTENT OF LOGICAL VOLUME (filesystem etc.)
     Converted docker-vg/docker-pool to thin pool.
     Logical volume "docker-pool" changed.

Enable and start docker service.

1
2
3

   # systemctl enable docker
   # systemctl start docker
   # systemctl is-active docker

Ensure Host Access

On each hosts, generate an SSH key WITHOUT a password

1

   # ssh-keygen

Copy the id_rsa.pub to each host:

1
2
3
4
5

   # for host in master.example.com \
       node1.example.com \
       node2.example.com; \
       do ssh-copy-id -i ~/.ssh/id_rsa.pub $host; \
       done

Quick Installation

Running an Interactive Installation

Start the interactive installation by running under command, and follow the on-screen instructions to install a new OpenShift Continer Platform cluster.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97

   $ atomic-openshift-installer install
   *** Installation Summary ***
   
   Hosts:
   - master.example.com
     - OpenShift master
     - OpenShift node (Unscheduled)
     - Etcd
     - Storage
   - node1.example.com
     - OpenShift node (Dedicated)
   - node2.example.com
     - OpenShift node (Dedicated)
   
   Total OpenShift masters: 1
   Total OpenShift nodes: 3
   
   NOTE: Add a total of 3 or more masters to perform an HA installation.
   
   Gathering information from hosts...
   All hosts in config are uninstalled. Proceeding with installation...
   
   Wrote atomic-openshift-installer config: /root/.config/openshift/installer.cfg.yml
   Wrote Ansible inventory: /root/.config/openshift/hosts
   
   Ready to run installation process.
   
   Play 1/28 (Create initial host groups for localhost)
   ..
   Play 2/28 (Create initial host groups for all hosts)
   .
   Play 3/28 (Populate config host groups)
   ................
   Play 4/28 (Ensure that all non-node hosts are accessible)
   .
   Play 5/28 (Initialize host facts)
   ................
   Play 6/28 (Gather and set facts for node hosts)
   ...............
   Play 7/28 (Verify compatible yum/subscription-manager combination)
   ..
   Play 8/28 (Determine openshift_version to configure on first master)
   ............................................................................................
   Play 9/28 (Set openshift_version for all hosts)
   ............................................................................................
   Play 10/28 (Set oo_option facts)
   ........
   Play 11/28 (Disable excluders)
   ..........................
   Play 12/28 (Configure etcd)
   ................................................................................................................................................Pausing for 10 seconds
   (ctrl+C then 'C' = continue early, ctrl+C then 'A' = abort)
   ....................................................................
   Play 13/28 (Configure nfs)
   ...............................................
   Play 14/28 (Gather and set facts for master hosts)
   .......................
   Play 15/28 (Determine if session secrets must be generated)
   ..............
   Play 16/28 (Generate master session secrets)
   ..............
   Play 17/28 (Configure masters)
   .............................................................................................................................................................................................................................................................................................................................................................................................................................................
   Play 18/28 (Additional master configuration)
   .......................................................................................................................................................................................................................
   Play 19/28 (Gather and set facts for node hosts)
   ...............
   Play 20/28 (Evaluate node groups)
   ..
   Play 21/28 (Configure nodes)
   .............................................................................................................................................................................................................................................................................................................................................
   Play 22/28 (Additional node config)
   .....................................................................................................................
   Play 23/28 (Create persistent volumes)
   ............................................................................................................................................................
   Play 24/28 (Create Hosted Resources)
   .......................................................................................................................................................................................................Pausing for 30 seconds
   (ctrl+C then 'C' = continue early, ctrl+C then 'A' = abort)
   ................................................
   Play 25/28 (Re-enable excluder if it was previously enabled)
   ...............
   localhost                  : ok=11   changed=0    unreachable=0    failed=0   
   master.example.com         : ok=606  changed=151  unreachable=0    failed=0   
   node1.example.com          : ok=202  changed=39   unreachable=0    failed=0   
   node2.example.com          : ok=202  changed=39   unreachable=0    failed=0   
   
   
   Installation Complete: Note: Play count is only an estimate, some plays may have been skipped or dynamically added
   
   
   The installation was successful!
   
   If this is your first time installing please take a look at the Administrator
   Guide for advanced options related to routing, storage, authentication, and
   more:
   
   http://docs.openshift.com/enterprise/latest/admin_guide/overview.html

Running an unattended Installation

Unattended installation allow you to run the installation with a pre-defined configuration file. The default installation configure file path is ~/.config/openshift/installer.cfg.yml. Define the configuration file and run the install command with the -u option.

1

   $ atomic-openshift-installer -u install

Here is a simple example of the install.cfg.yml file. For further information, please follow Defining an Installation Configuration File

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46

   ansible_callback_facts_yaml: /root/.config/openshift/.ansible/callback_facts.yaml
   ansible_inventory_path: /root/.config/openshift/hosts
   ansible_log_path: /tmp/ansible.log
   deployment:
     ansible_ssh_user: root
     hosts:
     - connect_to: master.example.com
       hostname: master
       ip: 10.64.221.200
       public_hostname: master
       public_ip: 10.64.221.200
       roles:
       - master
       - etcd
       - node
       - storage
     - connect_to: node1.example.com
       hostname: node1
       ip: 10.64.221.47
       node_labels: '{''region'': ''infra''}'
       public_hostname: node1
       public_ip: 10.64.221.47
       roles:
       - node
     - connect_to: node2.example.com
       hostname: node2
       ip: 10.64.221.192
       node_labels: '{''region'': ''infra''}'
       public_hostname: node2
       public_ip: 10.64.221.192
       roles:
       - node
     master_routingconfig_subdomain: ''
     openshift_master_cluster_hostname: None
     openshift_master_cluster_public_hostname: None
     proxy_exclude_hosts: ''
     proxy_http: ''
     proxy_https: ''
     roles:
       etcd: {}
       master: {}
       node: {}
       storage: {}
   variant: openshift-enterprise
   variant_version: '3.5'
   version: v2

Also you can specify a different path of the configuration file with the -c option.

1

   $ atomic-openshift-installer -u -c </path/to/file> install

Verifying the installation

After the installation is completed. 1. Verify the master and nodes are started in Ready status. On the master host, run the following as root

1
2
3
4
5
6

   # oc get nodes
   
   NAME                        STATUS                     AGE
   master.example.com          Ready,SchedulingDisabled   165d
   node1.example.com           Ready                      165d
   node2.example.com           Ready                      165d

  1. The web console use the master host name with a default port number 8443. In this test environment, you can find the web console at https://master.openshift.com:8443/console

  2. Now that the install has been verified, run the following command on each master and node host to add the atomic-openshift packages back to the list of yum excludes on the host:

    1

    # atomic-openshift-excluder exclude

Uninstallation

You can uninstall OpenShift Container Platform from all hosts using the follow commands

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26

   $ atomic-openshift-installer uninstall
   OpenShift will be uninstalled from the following hosts:
   
     * master.example.com
     * node1.example.com
     * node2.example.com
   
   Do you want to proceed? [y/N]: y
   
   Play 1/9 (OSEv3:children)
   ....
   Play 2/9 (nodes)
   ..
   Play 3/9 (masters)
   ..
   Play 4/9 (etcd)
   ..
   Play 5/9 (nodes)
   ...............................
   Play 6/9 (masters)
   ............
   Play 7/9 (etcd)
   ............
   master.example.com         : ok=60   changed=17   unreachable=0    failed=0
   node1.example.com          : ok=35   changed=8    unreachable=0    failed=0
   node2.example.com          : ok=35   changed=8    unreachable=0    failed=0

If you are using a configuration file, specify the file path for the uninstallation:

1

   $ atomic-openshift-installer -c </path/to/file> uninstall