...

Consider the following example for loading the images.
In order to preserve the images digests in the container registry, we recommend copying the downloaded images into the container registry using a recent version (1.13.3+) of the skopeo utility (available as a package for most distributions: Installing Skopeo).
Note: The example uses basic authentication. If the authentication is by a token, replace --dest-creds with --dest-registry-token along with the authentication token in the commands below.

1. Set variables with the source, destination, versions, credentials, etc. according to your environment:

   Code Block
   CONTAINER_REGISTRY_PATH="my-container-registry.example.com/dpod-cloud-agent" DPOD_CLOUD_AGENT_VERSION="1.0.2021.0" DPOD_CLOUD_AGENT_OPERATOR_VERSION="1.01.0" IMAGES_DIR="/tmp" USER_ID="user" USER_PASSWORD="password"

2. Load the images to the container registry:

   Code Block

   skopeo copy --all --preserve-digests --dest-creds=${USER_ID}:${USER_PASSWORD} docker-archive:${IMAGES_DIR}/dpod-ca-operator-catalog-${DPOD_CLOUD_AGENT_VERSION}.tgz \ docker://${CONTAINER_REGISTRY_PATH}/dpod-cloud-agent-operator-catalog:${DPOD_CLOUD_AGENT_OPERATOR_VERSION}-amd64 skopeo copy --all --preserve-digests --dest-creds=${USER_ID}:${USER_PASSWORD} docker-archive:${IMAGES_DIR}/dpod-ca-operator-bundle-${DPOD_CLOUD_AGENT_VERSION}.tgz \ docker://${CONTAINER_REGISTRY_PATH}/dpod-cloud-agent-operator-bundle:${DPOD_CLOUD_AGENT_OPERATOR_VERSION}-amd64 skopeo copy --all --preserve-digests --dest-creds=${USER_ID}:${USER_PASSWORD} docker-archive:${IMAGES_DIR}/dpod-ca-operator-${DPOD_CLOUD_AGENT_VERSION}.tgz \ docker://${CONTAINER_REGISTRY_PATH}/dpod-cloud-agent-operator:${DPOD_CLOUD_AGENT_OPERATOR_VERSION}-amd64 skopeo copy --all --preserve-digests --dest-creds=${USER_ID}:${USER_PASSWORD} docker-archive:${IMAGES_DIR}/dpod-ca-api-proxy-${DPOD_CLOUD_AGENT_VERSION}.tgz \ docker://${CONTAINER_REGISTRY_PATH}/dpod-cloud-agent-api-proxy:${DPOD_CLOUD_AGENT_VERSION}-amd64 skopeo copy --all --preserve-digests --dest-creds=${USER_ID}:${USER_PASSWORD} docker-archive:${IMAGES_DIR}/dpod-ca-http-ingester-${DPOD_CLOUD_AGENT_VERSION}.tgz \ docker://${CONTAINER_REGISTRY_PATH}/dpod-cloud-agent-http-ingester:${DPOD_CLOUD_AGENT_VERSION}-amd64 skopeo copy --all --preserve-digests --dest-creds=${USER_ID}:${USER_PASSWORD} docker-archive:${IMAGES_DIR}/dpod-ca-manager-${DPOD_CLOUD_AGENT_VERSION}.tgz \ docker://${CONTAINER_REGISTRY_PATH}/dpod-cloud-agent-manager:${DPOD_CLOUD_AGENT_VERSION}-amd64 skopeo copy --all --preserve-digests --dest-creds=${USER_ID}:${USER_PASSWORD} docker-archive:${IMAGES_DIR}/dpod-ca-messaging-broker-${DPOD_CLOUD_AGENT_VERSION}.tgz \ docker://${CONTAINER_REGISTRY_PATH}/dpod-cloud-agent-messaging-broker:${DPOD_CLOUD_AGENT_VERSION}-amd64 skopeo copy --all --preserve-digests --dest-creds=${USER_ID}:${USER_PASSWORD} docker-archive:${IMAGES_DIR}/dpod-ca-syslog-ingester-${DPOD_CLOUD_AGENT_VERSION}.tgz \ docker://${CONTAINER_REGISTRY_PATH}/dpod-cloud-agent-syslog-ingester:${DPOD_CLOUD_AGENT_VERSION}-amd64

Configuring Mirroring

...

The registry mirroring configuration may change based on the Kubernetes's container runtime.
The mirroring configuration should be added for each one of the worker nodes.
There are some methods to automate this configuration. For examples see following TODO : document

In order to identify the k8s cluster container runtime use the following command

kubectl get nodes -o wide

Example output (see CONTAINER-RUNTIME):

# For Docker runtime
NAME         STATUS   VERSION    CONTAINER-RUNTIME
node-1       Ready    v1.16.15   docker://19.3.1
node-2       Ready    v1.16.15   docker://19.3.1

# For containerd runtime
NAME         STATUS   VERSION   CONTAINER-RUNTIME
node-1       Ready    v1.19.6   containerd://1.4.1
node-2       Ready    v1.19.6   containerd://1.4.1

# For CRI-IO runtime
NAME         STATUS   VERSION   CONTAINER-RUNTIME
node-1       Ready    v1.25.11   cri-o://1.25.4
node-2       Ready    v1.25.11   cri-o://1.25.4

Containerd

The containerd Kubernetes's container runtime is used by many k8s providers like Amazon, Google, Microsoft and more. for extended list see following document

The containerd k8s cluster node configuration is located under /etc/containerd directory.
The CRI Registry Configuration is described here.

For the containerd CRI configuration changes take effect a containerd service restart is required systemctl restart containerd .

This is an example for the containerd CRI configuration. Change the configuration according to your environment:

Add the following attribute config_path = "/etc/containerd/certs.d" to /etc/containerd/config.toml (if not already exist):

[plugins."io.containerd.grpc.v1.cri".registry]
  config_path = "/etc/containerd/certs.d"

Create the following directory structure and update the hosts.toml file in each directory:

$ tree /etc/containerd/certs.d
/etc/containerd/certs.d
└── icr.io
    └── hosts.toml
    cp.icr.io
    └── hosts.toml

$ cat /etc/containerd/certs.d/icr.io/hosts.toml
server = "https://icr.io"
[host."https://my-container-registry.example.com/v2/dpod-cloud-agent"]
   capabilities = ["pull", "resolve"]
   skip_verify = false 
   override_path = true
   
$ cat /etc/containerd/certs.d/cp.icr.io/hosts.toml
server = "https://cp.icr.io"
[host."https://my-container-registry.example.com/v2/dpod-cloud-agent"]
   capabilities = ["pull", "resolve"]
   skip_verify = false 
   override_path = true

[plugins.cri.registry.mirrors]
  [plugins.cri.registry.mirrors.icr.io"]
    endpoint = ["https://my-container-registry.example.com/v2/dpod-cloud-agent"]
[plugins.cri.registry.mirrors.cp.icr.io"]
    endpoint = ["https://my-container-registry.example.com/v2/dpod-cloud-agent"]    

Authenticating to the Container Registry

This is an example of registry authentication via CRI (additional example is describe in here)

[plugins."io.containerd.grpc.v1.cri".registry.configs."gcr.io".auth]
  username = ""
  password = ""
  auth = ""
  identitytoken = ""

Docker , CRI-O

The configuration for these container runtime is located in /etc/containers/registries.conf

...

.
For the CRI configuration changes take effect a reboot of each worker node is required systemctl reboot .
This is an example of the configuration, change the registry.mirror entries according to your environment

...

:

[[registry]]
  prefix = ""
  location = "cp.icr.io/cp/dpod"
  mirror-by-digest-only = true
  [[registry.mirror]]
    location = "my-container-registry.example.com/dpod-cloud-agent"

[[registry]]
  prefix = ""
  location = "icr.io/cpopen"
  mirror-by-digest-only = true
  [[registry.mirror]]
    location = "my-container-registry.example.com/dpod-cloud-agent"

Authenticating to the Container Registry

Add the following authentication configuration (if needed) for each one of the worker nodes in config.json.

...

Change the URL and the auth value according to your environment.

{
    "auths": {
        ...
        "https://my-container-registry.example.com/dpod-cloud-agent": {
            "auth": "..."
        },
        ...
    }
}

...