Skip to main content

Override Registry Configuration

You can override the image registry configuration for Palette to reference a different image registry. This feature is useful when you want to use a custom image registry to store and manage the images Palette uses.

Prerequisites​

Before overriding the image registry configuration for Palette, ensure you have the following:

  • A deployed and healthy Palette cluster.

  • Access to the kubeconfig file for the Palette cluster. You need the kubeconfig file to access the Palette cluster and apply the image registry configuration.

    tip

    If you deployed Palette through the Palette CLI, then you can download the kubeconfig file from the Palette cluster details page in the system console. Navigate to the Enterprise Cluster Migration page. Click on the Admin Kubeconfig link to download the kubeconfig file. If you need help with configuring kubectl to access the Palette cluster, refer to the Access Cluster with CLI guide. If you deployed Palette onto an existing Kubernetes cluster, reach out to your cluster administrator for the kubeconfig file.

  • Access to a terminal session that has network access to the Palette cluster.

  • The kubectl command-line tool installed on your local machine. Refer to the kubectl installation guide to learn how to install kubectl.

  • If you are in an airgap environment, ensure Helm is installed on your local machine.

  • If you are in an airgap environment, you must have the image-swap Helm chart available there. You can get access to the image-swap Helm chart by contacting the support team at support@spectrocloud.com

Override Image Registry Configuration​

Select the appropriate tab below based on the environment in which your VertX cluster is deployed.

  1. Open a terminal session.

  2. Configure kubectl to use the kubeconfig file for the Palette cluster. Refer to the Access Cluster with CLI for guidance on configuring kubectl.

  3. Navigate to the folder where you have the image-swap Helm chart available. You may have to extract the Helm chart if it is in a compressed format to access the values.yaml file.

  4. Open the file values.yaml in a text editor and append the URL to your OCI registry that also includes the namespace or project that is hosting the Spectro Cloud images. The URL should be in the format <registry-url>/<namespace>. In the example configuration below, the value my-oci-registry.com/spectro-images is prefixed to each URL. Save the file after you have appended the URL.

    image:
      cainjectorImage: "my-oci-registry.com/spectro-images/gcr.io/spectro-images-public/release-fips/jetstack/cert-manager-cainjector:spectro-v1.11.0-20230427"
      controllerImage: "my-oci-registry.com/spectro-images/gcr.io/spectro-images-public/release-fips/jetstack/cert-manager-controller:spectro-v1.11.0-20230427"
      webhookImage: "my-oci-registry.com/spectro-images/gcr.io/spectro-images-public/release-fips/jetstack/cert-manager-webhook:spectro-v1.11.0-20230808"
      amceResolverImage: "my-oci-registry.com/spectro-images/gcr.io/spectro-images-public/release-fips/jetstack/cert-manager-acmesolver:spectro-v1.11.0-20230427"

  5. Next, update theociImageRegistry section with your OCI registry values. Refer to the table below for a description of each parameter.

    ociImageRegistry:
      endpoint: "REPLACE_WITH_CUSTOM_REGISTRY_DOMAIN"
      name: "REPLACE_WITH_CUSTOM_REGISTRY_NAME"
      password: "REPLACE_WITH_CUSTOM_REGISTRY_PASSWORD"
      username: "REPLACE_WITH_CUSTOM_REGISTRY_USERNAME"
      baseContentPath: "REPLACE_WITH_CUSTOM_REGISTRY_BASE_PATH"
      insecureSkipVerify: false
      caCert: "REPLACE_WITH_CUSTOM_REGISTRY_CA_CERT"
      mirrorRegistries: "REPLACE_WITH_CUSTOM_REGISTRY_MIRROR_REGISTRIES"
    ParameterDescriptionRequired
    endpointThe URL of the custom image registry.Yes
    nameThe name of the custom image registry.Yes
    passwordThe password to authenticate with the custom image registry. If the custom image registry does not require authentication, you can leave this field empty.No
    usernameThe username to authenticate with the custom image registry. If the custom image registry does not require authentication, you can leave this field empty.No
    baseContentPathThe base path of the custom image registry.Yes
    insecureSkipVerifySet to true if the custom image registry uses an insecure connection or a self-signed certificate. Set to false if the custom image registry uses a secure connection.Yes
    caCertThe Certificate Authority of the custom image registry in the PEM format. Required if the custom image registry uses a self-signed certificate.No
    mirrorRegistriesimage swap format for pulling images. For example: docker.io::harbor.example.org/airgap-images/docker.io,gcr.io::harbor.example.org/airgap-images/gcr.io,ghcr.io::harbor.example.org/airgap-images/ghcr.io,k8s.gcr.io::harbor.example.org/airgap-images/gcr.io,registry.k8s.io::harbor.example.org/airgap-images/k8s.io,quay.io::harbor.example.org/airgap-images/quay.io,us-east1-docker.pkg.dev::harbor.example.org/airgap-imagesYes
    Click here for a complete example configuration.
    config:
      imageSwapImages:
        imageSwapInitImage: "harbor.example.org/airgap-images/gcr.io/spectro-images-public/release/thewebroot/imageswap-init:v1.5.2-spectro-4.1.1"
        imageSwapImage: "harbor.example.org/airgap-images/gcr.io/spectro-images-public/release/thewebroot/imageswap:v1.5.2-spectro-4.1.1"

      imageSwapConfig:
        isEKSCluster: true #If the Cluster you are trying to install is EKS cluster set value to true else set to false

      ociImageRegistry:
        endpoint: "harbor.example.org" #<Contact Spectro Cloud Sales for More info>
        name: "Internal Registry" #<Contact Spectro Cloud Sales for More info>
        password: "" #<Contact Spectro Cloud Sales for More info>
        username: "" #<Contact Spectro Cloud Sales for More info>
        baseContentPath: "airgap-images" #<Contact Spectro Cloud Sales for More info>
        insecureSkipVerify: false
        caCert: ""
        mirrorRegistries: "docker.io::harbor.example.org/airgap-images/docker.io,gcr.io::harbor.example.org/airgap-images/gcr.io,ghcr.io::harbor.example.org/airgap-images/ghcr.io,k8s.gcr.io::harbor.example.org/airgap-images/gcr.io,registry.k8s.io::harbor.example.org/airgap-images/k8s.io,quay.io::harbor.example.org/airgap-images/quay.io,us-east1-docker.pkg.dev::harbor.example.org/airgap-images"

  6. Once you have configured the ociImageRegistry section, issue the following command from the folder where you have the image-swap Helm chart available to deploy the image-swap Helm chart. You may have to modify the command below based on the location of values.yaml file and the compressed Helm chart file.

    helm upgrade --values values.yaml image-swap image-swap-*.tgz --install
    Release "image-swap" does not exist. Installing it now.
    NAME: image-swap
    LAST DEPLOYED: Mon March 20 17:04:23 2024
    NAMESPACE: default
    STATUS: deployed
    REVISION: 1
    TEST SUITE: None

  7. Create an empty YAML file with the name registry-secret.yaml. Use the following command to create the file.

    touch registry-secret.yaml

  8. Open the registry-secret.yaml file and copy the placeholder configuration below.

     ---
     apiVersion: v1
     stringData:
       DOMAIN: "REPLACE_WITH_CUSTOM_REGISTRY_DOMAIN"
       BASE_PATH: "REPLACE_WITH_CUSTOM_REGISTRY_BASE_PATH"
       USERNAME: "REPLACE_WITH_CUSTOM_REGISTRY_USERNAME"
       PASSWORD: "REPLACE_WITH_CUSTOM_REGISTRY_PASSWORD"
       INSECURE: "false"
       CA_CERT: "REPLACE_WITH_CUSTOM_REGISTRY_CA_CERT"
       MIRROR_REGISTRIES: REPLACE_WITH_CUSTOM_REGISTRY_MIRROR_REGISTRIES
     kind: Secret
     metadata:
       name: registry-info
       namespace: jet-system
     type: Opaque
     ---
    warning

    Do not change the kind, metadata and type fields in the YAML file. The values provided in the placeholder configuration are required to override the image registry configuration for the Palette cluster.

  9. Replace the placeholder values with the actual values for your custom image registry. Use the same values that you used in the ociImageRegistry section of the values.yaml file for the image-swap Helm chart. Refer to the table below for a description of each parameter.

    ParameterDescriptionRequired
    DOMAINThe domain of the custom image registry.Yes
    BASE_PATHThe base path to the custom image registry.Yes
    USERNAMEThe username to authenticate with the custom image registry. If the custom image registry does not require authentication, you can leave this field empty.No
    PASSWORDThe password to authenticate with the custom image registry. If the custom image registry does not require authentication, you can leave this field empty.No
    INSECURESet to true if the custom image registry uses an insecure connection or a self-signed certificate. Set to false if the custom image registry uses a secure connection.Yes
    CA_CERTThe Certificate Authority of the custom image registry in the PEM format. Required if the custom image registry uses a self-signed certificate.No
    MIRROR_REGISTRIESA comma-separated list of mirror registries in the image swap format to use for pulling images. For example: docker.io::harbor.example.org/airgap-images/docker.io,gcr.io::harbor.example.org/airgap-images/gcr.io,ghcr.io::harbor.example.org/airgap-images/ghcr.io,k8s.gcr.io::harbor.example.org/airgap-images/gcr.io,registry.k8s.io::harbor.example.org/airgap-images/k8s.io,quay.io::harbor.example.org/airgap-images/quay.io,us-east1-docker.pkg.dev::harbor.example.org/airgap-imagesYes
    Click here for a complete example configuration.
    ---
    apiVersion: v1
    stringData:
      DOMAIN: "harbor.example.org"
      BASE_PATH: "airgap-images"
      USERNAME: ""
      PASSWORD: ""
      INSECURE: "false"
      CA_CERT: ""
      MIRROR_REGISTRIES: docker.io::harbor.example.org/airgap-images/docker.io,gcr.io::harbor.example.org/airgap-images/gcr.io,ghcr.io::harbor.example.org/airgap-images/ghcr.io,k8s.gcr.io::harbor.example.org/airgap-images/gcr.io,registry.k8s.io::harbor.example.org/airgap-images/k8s.io,quay.io::harbor.example.org/airgap-images/quay.io,us-east1-docker.pkg.dev::harbor.example.org/airgap-images
    kind: Secret
    metadata:
      name: registry-info
      namespace: jet-system
    type: Opaque
    ---

  10. Once you have created the YAML file and configured the parameter values, issue the following command to create the Kubernetes secret containing the image registry configuration.

    kubectl create --filename registry-secret.yaml

Validate​

  1. Open a terminal session with a network access to the VeteX cluster.

  2. Configure kubectl to use the kubeconfig file for the Palette cluster. Refer to the Access Cluster with CLI for guidance on configuring kubectl.

  3. Issue the following command to verify that the secret containing the image registry configuration is created.

    kubectl get secret registry-info --namespace=jet-system  \
    --output jsonpath='{.data.MIRROR_REGISTRIES}' | base64 --decode

    The command returns the mirror registries that you configured in the MIRROR_REGISTRIES parameter.

    docker.io::harbor.example.org/airgap-images/docker.io,gcr.io::harbor.example.org/airgap-images/gcr.io,ghcr.io::harbor.example.org/airgap-images/ghcr.io,k8s.gcr.io::harbor.example.org/airgap-images/gcr.io,registry.k8s.io::harbor.example.org/airgap-images/k8s.io,quay.io::harbor.example.org/airgap-images/quay.io,us-east1-docker.pkg.dev::harbor.example.org/airgap-images

  4. Deploy a cluster through Palette. Palette will propagate the image registry configuration to the workload cluster, and the cluster will use the custom image registry to pull images if specified in the mirror registry configuration.

  5. SSH into one of the workload cluster nodes. You can verify the image registry configuration on the workload cluster by checking the containerd configuration file. Use the following command to check the containerd configuration file.

    cat /etc/containerd/config.toml

    Each mirror registry specified in the MIRROR_REGISTRIES parameter is added to the plugins."io.containerd.grpc.v1.cri".registry.mirrors. section. Based on the example configuration we provided in step four, the configuration file should contain the following details.

    ## template: jinja

    # Use config version 2 to enable new configuration fields.
    # Config file is parsed as version 1 by default.
    version = 2

    imports = ["/etc/containerd/conf.d/*.toml"]

    [plugins]
      [plugins."io.containerd.grpc.v1.cri"]
      [plugins."io.containerd.grpc.v1.cri".containerd.runtimes.runc]
        runtime_type = "io.containerd.runc.v2"
      [plugins."io.containerd.grpc.v1.cri".containerd.runtimes.runc.options]
        SystemdCgroup = true


      # contains spectro changes
      [plugins."io.containerd.grpc.v1.cri".registry]
        [plugins."io.containerd.grpc.v1.cri".registry.mirrors]
          [plugins."io.containerd.grpc.v1.cri".registry.mirrors."docker.io"]
            endpoint = ["harbor.example.org/airgap-images/docker.io"]
          [plugins."io.containerd.grpc.v1.cri".registry.mirrors."gcr.io"]
            endpoint = ["harbor.example.org/airgap-images/gcr.io"]
          [plugins."io.containerd.grpc.v1.cri".registry.mirrors."ghcr.io"]
            endpoint = ["harbor.example.org/airgap-images/ghcr.io"]
          [plugins."io.containerd.grpc.v1.cri".registry.mirrors."k8s.gcr.io"]
            endpoint = ["harbor.example.org/airgap-images/gcr.io"]
          [plugins."io.containerd.grpc.v1.cri".registry.mirrors."registry.k8s.io"]
            endpoint = ["harbor.example.org/airgap-images/k8s.io"]
          [plugins."io.containerd.grpc.v1.cri".registry.mirrors."quay.io"]
            endpoint = ["harbor.example.org/airgap-images/quay.io"]
          [plugins."io.containerd.grpc.v1.cri".registry.mirrors."us-east1-docker.pkg.dev"]
            endpoint = ["harbor.example.org/airgap-images"]