Onload Custom Resource (CR) - Onload Custom Resource (CR) - UG1656

Instruct the Onload Operator to deploy the components necessary for accelerating workload pods by deploying an Onload kind of Custom Resource (CR).

If your cluster is internet-connected OpenShift® and you want to use in-cluster builds with the current version of OpenOnload, run:

kubectl apply -k https://github.com/Xilinx-CNS/kubernetes-onload/config/samples/onload/overlays/in-cluster-build-ocp?ref=v3.0

This takes a base Onload CR template from config/samples/onload/base/onload_v1alpha1_onload.yaml and adds the appropriate image versions ( config/samples/onload/overlays/in-cluster-build-ocp/kustomization.yaml ) and in-cluster build configuration ( config/samples/onload/overlays/in-cluster-build-ocp/patch-onload.yaml ). To customize this recommended overlay further, see comments in these files and the variant steps below.

The above overlay configures KMM to modprobe onload and modprobe sfc. Both are required, but the latter can occur outside the Onload Operator. See Out-of-tree sfc Kernel Module for options.

For further explanation of available properties for an Onload CR, refer to either inline comments in these templates or the built-in explain command, for example kubectl explain onload.spec.

The schema for the above templates is defined by an Onload Custom Resource Definition (CRD) in api/v1alpha1/onload_types.go which is distributed as part of the config/crd/bases/onload.amd.com_onloads.yaml generated YAML bundle for the Onload Operator.

The following example loads Onload 8.1.0, and illustrates fields that might be used in a complete CR:

apiVersion: onload.amd.com/v1alpha1
kind: Onload
metadata:
  labels:
    app.kubernetes.io/name: onload
    app.kubernetes.io/instance: onload-sample
    app.kubernetes.io/part-of: onload-operator
    app.kubernetes.io/managed-by: kustomize
    app.kubernetes.io/created-by: onload-operator
  name: onload-sample
spec:
  serviceAccountName: onload-operator-sa
  selector:
    node-role.kubernetes.io/worker: ""
  onload:
    # Example image locations using openshift local image registry.
    kernelMappings:
      - regexp: '^.*\.x86_64$'
        kernelModuleImage: image-registry.openshift-image-registry.svc:5000/onload-clusterlocal/onload-module:v8.1.0-${KERNEL_FULL_VERSION}
        sfc: {}
    userImage: image-registry.openshift-image-registry.svc:5000/onload-clusterlocal/onload-user:v8.1.0
    version: 8.1.0
    imagePullPolicy: Always
  devicePlugin:
    imagePullPolicy: Always

where:

apiVersion: onload.amd.com/v1alpha1, kind: Onload

This specifies to the Kubernetes control plane that the resource creation request must go to the Onload Operator controller.

Note: The URL-like string is not a URL but is instead a Kubernetes-formatted identifier.

metadata.name

The name of the Onload CR.

Important: Due to Kubernetes limitations on label lengths, the combined length of the name and namespace of the Onload CR must be less than 32 characters.

spec.serviceAccountName: onload-operator-sa

This is the privileged Service Account (SA) on whose behalf the Onload Operator creates the Onload sub-resources (sometimes called "operands").

spec.selector

This configures where (in which nodes) to deploy the Onload operands. AMD suggests that you install the Node Feature Discovery (NFD) operator to label the nodes with the strings containing PCI device details. In this case, you can express "Deploy Onload in nodes with the Solarflare cards."

spec.onload.kernelMappings

Allows a set of heterogeneous base OSes in the nodes to support the node upgrade:

kernelModuleImage

This specifies a pre-built Docker image with the Onload kernel modules (literally *.ko files).

sfc

An empty object that, when present, instructs the Onload Operator to deploy the SFC kernel modules from the above Docker image.

spec.onload.userImage

This specifies another pre-built Docker image with the Onload userspace libraries. It must match the kernel part, including release/debug, branch, and git-hash.

spec.onload.version

This is a string the Onload Operator uses to detect the Onload upgrade request. You can edit the created CRs, for example with oc edit onload onload-sample and change fields. You can change the Onload Docker images and version to kick off the rolling Onload upgrade (node by node).