49 Prepare downstream cluster image for connected scenarios #

Edge Image Builder (Chapter 12, Edge Image Builder) is used to prepare a modified SLEMicro base image which is provisioned on downstream cluster hosts.

Much of the configuration via Edge Image Builder is possible, but in this guide, we cover the minimal configurations necessary to set up the downstream cluster.

49.1 Prerequisites for connected scenarios #

A container runtime such as Podman or Rancher Desktop is required to run Edge Image Builder.
The base image will be built using the following guide Chapter 64, Building Updated SUSE Linux Micro Images with Kiwi with the profile Base (or Base-RT for the Real-Time kernel). The process is the same for both architectures (x86-64 and aarch64).

Note

It is required to use a build host with the same architecture of the images being built. In other words, to build an aarch64 image, it is required to use an aarch64 build host, and vice-versa for x86-64 (cross-builds are not supported at this time).

49.2 Image configuration for connected scenarios #

When running Edge Image Builder, a directory is mounted from the host, so it is necessary to create a directory structure to store the configuration files used to define the target image.

downstream-cluster-config.yaml is the image definition file, see Chapter 5, Standalone clusters with Edge Image Builder for more details.
The base image folder will contain the output raw image generated following the guide Chapter 64, Building Updated SUSE Linux Micro Images with Kiwi with the profile Base (or Base-RT for the Real-Time kernel) must be copied/moved under the base-images folder.
The custom/files directory contains the performance-settings.sh and sriov-auto-filler.sh files to be copied to the image during the image creation process.
The custom/scripts directory contains scripts to be run on first-boot:
1. 01-fix-growfs.sh script is required to resize the OS root partition on deployment
2. 02-performance.sh script is optional and can be used to configure the system for performance tuning.
3. 03-sriov.sh script is optional and can be used to configure the system for SR-IOV.
The network folder is optional, see Section 49.2.6, “Additional script for Advanced Network Configuration” for more details.
The os-files folder contains the necessary configuration files to load, at each node boot, the sctp and vfio-pci kernel modules with the required options.

├── downstream-cluster-config.yaml
├── base-images
│   └── SL-Micro.x86_64-6.2-Base-GM.raw
├── custom
│   ├── files
│   │   ├── performance-settings.sh
│   │   └── sriov-auto-filler.sh
│   └── scripts
│       ├── 01-fix-growfs.sh
│       ├── 02-performance.sh
│       └── 03-sriov.sh
├── network
│   └── configure-network.sh
└── os-files
    └── etc
        ├── modprobe.d
        │   └── vfio-pci-options.conf
        └── modules-load.d
            ├── sctp.conf
            └── vfio-pci.conf

49.2.1 Downstream cluster image definition file #

The downstream-cluster-config.yaml file is the main configuration file for the downstream cluster image. The following is a minimal example for deployment via Metal³:

apiVersion: 1.3
image:
  imageType: raw
  arch: x86_64
  baseImage: SL-Micro.x86_64-6.2-Base-GM.raw
  outputImageName: eibimage-output-telco.raw
operatingSystem:
  kernelArgs:
    - ignition.platform.id=openstack
  systemd:
    disable:
      - rebootmgr
      - transactional-update.timer
      - transactional-update-cleanup.timer
      - fstrim
      - time-sync.target
  users:
    - username: root
      encryptedPassword: $ROOT_PASSWORD
      sshKeys:
      - $USERKEY1
  packages:
    packageList:
      - jq
    sccRegistrationCode: $SCC_REGISTRATION_CODE

Where $SCC_REGISTRATION_CODE is the registration code copied from SUSE Customer Center, and the package list contains jq which is required.

$ROOT_PASSWORD is the encrypted password for the root user, which can be useful for test/debugging. It can be generated with the openssl passwd -6 PASSWORD command

For the production environments, it is recommended to use the SSH keys that can be added to the users block replacing the $USERKEY1 with the real SSH keys.

Note

arch: x86_64 is the architecture of the image. For arm64 architecture, use arch: aarch64.

Note ignition.platform.id=openstack is mandatory, without this argument SLEMicro configuration via ignition will fail in the Metal³ automated flow.

For SUSE Linux Micro versions earlier than 6.2, net.ifnames=1 must be explicitly set as a kernel argument to enable Predictable Network Interface Naming. This is enforced by default from 6.2 onwards and explicit configuration is not required. This aligns with the predictableNicNames configuration in the Management Cluster’s Metal³ Helm chart, which is required for Directed Network Provisioning to function correctly. Consistent interface naming is also critical when SR-IOV is utilized.

49.2.2 Growfs script #

Currently, a custom script (custom/scripts/01-fix-growfs.sh) is required to grow the file system to match the disk size on first-boot after provisioning. The 01-fix-growfs.sh script contains the following information:

#!/bin/bash
growfs() {
  mnt="$1"
  dev="$(findmnt --fstab --target ${mnt} --evaluate --real --output SOURCE --noheadings)"
  # /dev/sda3 -> /dev/sda, /dev/nvme0n1p3 -> /dev/nvme0n1
  parent_dev="/dev/$(lsblk --nodeps -rno PKNAME "${dev}")"
  # Last number in the device name: /dev/nvme0n1p42 -> 42
  partnum="$(echo "${dev}" | sed 's/^.*[^0-9]\([0-9]\+\)$/\1/')"
  ret=0
  growpart "$parent_dev" "$partnum" || ret=$?
  [ $ret -eq 0 ] || [ $ret -eq 1 ] || exit 1
  /usr/lib/systemd/systemd-growfs "$mnt"
}
growfs /

49.2.3 Performance script #

The following optional script (custom/scripts/02-performance.sh) can be used to configure the system for performance tuning:

#!/bin/bash

# create the folder to extract the artifacts there
mkdir -p /opt/performance-settings

# copy the artifacts
cp performance-settings.sh /opt/performance-settings/

The content of custom/files/performance-settings.sh is a script that can be used to configure the system for performance tuning and can be downloaded from the following link.

49.2.4 SR-IOV script #

The following optional script (custom/scripts/03-sriov.sh) can be used to configure the system for SR-IOV:

#!/bin/bash

# create the folder to extract the artifacts there
mkdir -p /opt/sriov
# copy the artifacts
cp sriov-auto-filler.sh /opt/sriov/sriov-auto-filler.sh

The content of custom/files/sriov-auto-filler.sh is a script that can be used to configure the system for SR-IOV and can be downloaded from the following link.

Note

Add your own custom scripts to be executed during the provisioning process using the same approach. For more information, see Chapter 5, Standalone clusters with Edge Image Builder.

49.2.5 Additional configuration for Telco workloads #

To enable Telco features like dpdk, sr-iov or FEC, additional packages may be required as shown in the following example.

apiVersion: 1.3
image:
  imageType: raw
  arch: x86_64
  baseImage: SL-Micro.x86_64-6.2-Base-GM.raw
  outputImageName: eibimage-output-telco.raw
operatingSystem:
  kernelArgs:
    - ignition.platform.id=openstack
  systemd:
    disable:
      - rebootmgr
      - transactional-update.timer
      - transactional-update-cleanup.timer
      - fstrim
      - time-sync.target
  users:
    - username: root
      encryptedPassword: $ROOT_PASSWORD
      sshKeys:
      - $user1Key1
  packages:
    packageList:
      - jq            # Non Telco specific - category: testing/utils
      - dpdk          # TelCo Core-Network UserPlane (5G UPF) and RAN UserPlane (gNB: DU and CU-UP) - category: runtime
      - dpdk-tools    # TelCo Core-Network UserPlane (5G UPF) and RAN UserPlane (gNB: DU and CU-UP) - category: testing/utils
      - libdpdk-25    # TelCo Core-Network UserPlane (5G UPF) and RAN UserPlane (gNB: DU and CU-UP) - category: runtime
      - pf-bb-config  # TelCo RAN (gNB: DU) - category: config.
      - open-iscsi    # Non Telco specific (required by SUSE Storage/Longhorn) - category: runtime
      - tuned         # Non Telco specific (linux tuning) - category: config.
      - cpupower      # Non Telco specific (linux tuning - CPU power related settings) - category: config.
      - rt-tests      # Non Telco specific (RT scheduler-latency testing) - category: testing/utils
      - linuxptp      # TelCo RAN; time sync (gNB: DU) - category: runtime
      - synce4l       # TelCo RAN; time sync (gNN: DU) - category: runtime
      - pciutils      # Non Telco specific (provides "lspci" command) - category: testing/utils
      - numactl       # Non Telco specific (provides "numactl" command) - category: testing/utils
    sccRegistrationCode: $SCC_REGISTRATION_CODE

Where $SCC_REGISTRATION_CODE is the registration code copied from SUSE Customer Center, and the package list contains the minimum packages to be used for the Telco profiles.

Note

arch: x86_64 is the architecture of the image. For arm64 architecture, use arch: aarch64.

49.2.6 Additional script for Advanced Network Configuration #

If you need to configure static IPs or more advanced networking scenarios as described in Chapter 53, Advanced Network Configuration, the following additional configuration is required.

In the network folder, create the following configure-network.sh file - this consumes configuration drive data on first-boot, and configures the host networking using the NM Configurator tool.

#!/bin/bash

set -eux

# Attempt to statically configure a NIC in the case where we find a network_data.json
# In a configuration drive

CONFIG_DRIVE=$(blkid --label config-2 || true)
if [ -z "${CONFIG_DRIVE}" ]; then
  echo "No config-2 device found, skipping network configuration"
  exit 0
fi

mount -o ro $CONFIG_DRIVE /mnt

META_DATA_FILE="/mnt/openstack/latest/meta_data.json"
if [ ! -f "${META_DATA_FILE}" ]; then
  umount /mnt
  echo "No meta_data.json found, skipping hostname configuration"
  exit 0
fi

DESIRED_HOSTNAME=$(cat /mnt/openstack/latest/meta_data.json | tr ',{}' '\n' | grep '"metal3-name"' | sed 's/.*\"metal3-name\": \"\(.*\)\"/\1/')
echo "${DESIRED_HOSTNAME}" > /etc/hostname

NETWORK_DATA_FILE="/mnt/openstack/latest/network_data.json"

if [ ! -f "${NETWORK_DATA_FILE}" ]; then
  umount /mnt
  echo "No network_data.json found, skipping network configuration"
  exit 0
fi

mkdir -p /tmp/nmc/{desired,generated}
cp ${NETWORK_DATA_FILE} /tmp/nmc/desired/_all.yaml
umount /mnt

./nmc generate --config-dir /tmp/nmc/desired --output-dir /tmp/nmc/generated
./nmc apply --config-dir /tmp/nmc/generated

49.2.7 Telco required kernel modules #

To ensure that the kernel modules required for telecom workloads are loaded on each node boot, the following configuration files must be added to the os-files EIB folder:

49.2.7.1 sctp.conf #

To be placed in the os-files/etc/modules-load.d/ path, instructing the systemd-modules-load.service to load the sctp kernel module at node boot.

$ cat os-files/etc/modules-load.d/sctp.conf
# Request "systemd-modules-load.service" to load sctp module at boot time
sctp

49.2.7.2 vfio-pci.conf #

To be placed in the os-files/etc/modules-load.d/ path, instructing the systemd-modules-load.service to load the vfio-pci kernel module at node boot.

$ cat os-files/etc/modules-load.d/vfio-pci.conf
# Request "systemd-modules-load.service" to load vfio-pci module at boot time
# NOTE: The (SRIOV & DPDK related) required load-module options settings are defined in related "/etc/modprobe.d/vfio-pci-options.conf" file
vfio_pci

49.2.7.3 vfio-pci-options.conf #

To be placed in the os-files/etc/modprobe.d/ path, setting the required vfio-pci kernel module parameters to be applied when the kernel module is loaded.

$ cat os-files/etc/modprobe.d/vfio-pci-options.conf
# Enable SR-IOV and disable idle D3 state for vfio-pci driver
options vfio_pci enable_sriov=1 disable_idle_d3=1

49.3 Image creation #

Once the directory structure is prepared following the previous sections, run the following command to build the image:

podman run --rm --privileged -it -v $PWD:/eib \
 registry.suse.com/edge/3.6/edge-image-builder:1.3.3.1 \
 build --definition-file downstream-cluster-config.yaml

This creates the output ISO image file named eibimage-output-telco.raw, based on the definition described above.

The output image must then be made available via a webserver, either the media-server container enabled via the Management Cluster Documentation (Note) or some other locally accessible server. In the examples below, we refer to this server as imagecache.local:8080