Cloud Agent Deployment on K8S Cloud Providers Examples

IBM DataPower Operations Dashboard v1.0.21.x

A newer version of this product documentation is available.

You are viewing an older version. View latest at https://ibm.biz/dpod-docs.

Cloud Agent Deployment on K8S Cloud Providers Examples

The following scenarios for deploying DPOD Cloud Agent on k8s cloud providers are provided as an examples/samples and may change between different cloud providers, k8s versions and k8s cluster custom configuration.

Set variables according to your environment:

DPOD_CLOUD_AGENT_VERSION="1.0.21.0" DPOD_CLOUD_AGENT_OPERATOR_VERSION="1.1.0"

Configure Amazon EKS for Cloud Agent Deployment

Configure cluster nodes for registry mirror (containerd)

  • Create EC2 Launch Template for the cluster nodes.

  • In the Launch Template Advanced details -> User data add the following configuration:
    Change the configuration according to your environment (at least the host attribute).

    MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="//" --// Content-Type: text/x-shellscript; charset="us-ascii" #!/bin/bash mkdir -p /etc/containerd/certs.d/icr.io printf 'server = "https://icr.io"\n[host."https://icr.io/dpod"]\n capabilities = ["pull", "resolve"]\n skip_verify = false\n override_path = true\n' | tee /etc/containerd/certs.d/icr.io/hosts.toml --//--
  • Make sure to create the cluster with Node Group that uses the new Launch Template

Configure Google GKE for Cloud Agent Deployment

Configure cluster nodes for registry mirror (containerd)

A DaemonSet can be used to configure the GKE cluster nodes containerd runtime.

K8s DaemonSet ensures that all (or some) Nodes run a copy of a Pod, this Pod can execute logic which in this case can be used to configure the registry mirror as described here.

For further information about K8S DaemonSet can be found in K8s documentation.

A generic example of “startup-script” DeamonSet can be found here.

GKE nodes are automatically configured using the deprecated registry mirror format so the logic implemented in the DaemonSet should continuing this method.
The user may change the DamonSet logic to convert the configuration to the new format

Deploy DPOD Cloud Agent on k8s cluster using OLM

Starting k8s 1.25 Pod Security Admission became stable feature and adopted by most of the cloud providers which requires some configurations in order to deploy OLM and use it to deploy operators and applications. For further information see K8s documentation.

  • Change the DPOD Cloud Agent NS labels to conform with Pod Security Admission

    apiVersion: v1 kind: Namespace metadata: labels: kubernetes.io/metadata.name: example-dpod-cloudagent-ns pod-security.kubernetes.io/enforce: restricted pod-security.kubernetes.io/enforce-version: latest pod-security.kubernetes.io/audit: restricted pod-security.kubernetes.io/audit-version: latest pod-security.kubernetes.io/warn: restricted pod-security.kubernetes.io/warn-version: latest name: example-dpod-cloudagent-ns
  • Create the DPOD Cloud Agent CatalogSource in OLM NS

    • Based on OLM documentation in order to create subscription the CatalogSource should be created in the target NS (where the subscription for DPOD Cloud Agent is created) or in OLM NS (ClusterScope).

    • Use the following definition to create CatalogSource which conform with Pod Security Admission configured for the OLM NS:

      apiVersion: operators.coreos.com/v1alpha1 kind: CatalogSource metadata: name: ibm-dpod-cloud-agent-catalog namespace: olm spec: displayName: IBM DataPower Operations Dashboard Cloud Agent image: icr.io/cpopen/dpod-cloud-agent-operator-catalog:${DPOD_CLOUD_AGENT_OPERATOR_VERSION}-amd64 publisher: IBM sourceType: grpc grpcPodConfig: securityContextConfig: restricted
  • Create OperatorGroup for the new namespace for ClusterScope

    apiVersion: operators.coreos.com/v1 kind: OperatorGroup metadata: name: dpod-cloudagent-group namespace: example-dpod-cloudagent-ns
  • Install the DPOD Cloud Agent operator

    apiVersion: operators.coreos.com/v1alpha1 kind: Subscription metadata: name: ibm-dpod-cloud-agent-operator namespace: example-dpod-cloudagent-ns spec: channel: stable-v1.0 installPlanApproval: Automatic name: dpod-cloud-agent-operator source: ibm-dpod-cloud-agent-catalog sourceNamespace: olm startingCSV: dpod-cloud-agent-operator.v${DPOD_CLOUD_AGENT_OPERATOR_VERSION}
  • Create CR to deploy the DPOD Cloud Agent

Create LoadBalancer services for DPOD Cloud Agent on Amazon EKS

Deploy Manager LB service

apiVersion: v1 kind: Service metadata: name: dpod-cloud-agent-prod-mng-lb-svc namespace: example-dpod-cloudagent-ns labels: app.kubernetes.io/component: dpod-cloud-agent-manager app.kubernetes.io/instance: dpod-cloud-agent-prod app.kubernetes.io/managed-by: dpod-cloud-agent-operator app.kubernetes.io/name: dpod-cloud-agent-manager app.kubernetes.io/part-of: dpod-cloud-agent app.kubernetes.io/version: ${DPOD_CLOUD_AGENT_OPERATOR_VERSION} annotations: service.beta.kubernetes.io/aws-load-balancer-type: external service.beta.kubernetes.io/aws-load-balancer-scheme: internet-facing service.beta.kubernetes.io/aws-load-balancer-nlb-target-type: instance spec: type: LoadBalancer ports: - port: 443 targetPort: 8443 name: https-manager selector: app.kubernetes.io/component: dpod-cloud-agent-manager app.kubernetes.io/instance: dpod-cloud-agent-prod app.kubernetes.io/name: dpod-cloud-agent-manager app.kubernetes.io/part-of: dpod-cloud-agent sessionAffinity: None

Deploy LoadBalancer service based on the number of messaging brokers replicas.

  • Deploy Messaging bootstrap LB service

    apiVersion: v1 kind: Service metadata: name: dpod-cloud-agent-prod-msg-bse-lb-svc namespace: example-dpod-cloudagent-ns labels: app.kubernetes.io/component: dpod-cloud-agent-manager app.kubernetes.io/instance: dpod-cloud-agent-prod app.kubernetes.io/managed-by: dpod-cloud-agent-operator app.kubernetes.io/name: dpod-cloud-agent-manager app.kubernetes.io/part-of: dpod-cloud-agent app.kubernetes.io/version: ${DPOD_CLOUD_AGENT_OPERATOR_VERSION} annotations: service.beta.kubernetes.io/aws-load-balancer-type: external service.beta.kubernetes.io/aws-load-balancer-scheme: internet-facing service.beta.kubernetes.io/aws-load-balancer-nlb-target-type: instance spec: type: LoadBalancer ports: - port: 30100 targetPort: 29092 name: kafka-bootstrap selector: app.kubernetes.io/component: dpod-cloud-agent-messaging app.kubernetes.io/instance: dpod-cloud-agent-prod app.kubernetes.io/name: dpod-cloud-agent-messaging app.kubernetes.io/part-of: dpod-cloud-agent sessionAffinity: None
  • Deploy broker-0 LB service

apiVersion: v1 kind: Service metadata: name: dpod-cloud-agent-prod-msg-dir-lb-svc-0 namespace: example-dpod-cloudagent-ns labels: app.kubernetes.io/component: dpod-cloud-agent-manager app.kubernetes.io/instance: dpod-cloud-agent-prod app.kubernetes.io/managed-by: dpod-cloud-agent-operator app.kubernetes.io/name: dpod-cloud-agent-manager app.kubernetes.io/part-of: dpod-cloud-agent app.kubernetes.io/version: ${DPOD_CLOUD_AGENT_OPERATOR_VERSION} annotations: service.beta.kubernetes.io/aws-load-balancer-type: external service.beta.kubernetes.io/aws-load-balancer-scheme: internet-facing service.beta.kubernetes.io/aws-load-balancer-nlb-target-type: instance spec: type: LoadBalancer ports: - port: 30101 targetPort: 29092 name: kafka-broker-0 selector: app.kubernetes.io/component: dpod-cloud-agent-messaging app.kubernetes.io/instance: dpod-cloud-agent-prod app.kubernetes.io/name: dpod-cloud-agent-messaging app.kubernetes.io/part-of: dpod-cloud-agent statefulset.kubernetes.io/pod-name: dpod-cloud-agent-prod-msg-0 sessionAffinity: None
  • Deploy broker-1 LB service

    apiVersion: v1 kind: Service metadata: name: dpod-cloud-agent-prod-msg-dir-lb-svc-1 namespace: example-dpod-cloudagent-ns labels: app.kubernetes.io/component: dpod-cloud-agent-manager app.kubernetes.io/instance: dpod-cloud-agent-prod app.kubernetes.io/managed-by: dpod-cloud-agent-operator app.kubernetes.io/name: dpod-cloud-agent-manager app.kubernetes.io/part-of: dpod-cloud-agent app.kubernetes.io/version: ${DPOD_CLOUD_AGENT_OPERATOR_VERSION} annotations: service.beta.kubernetes.io/aws-load-balancer-type: external service.beta.kubernetes.io/aws-load-balancer-scheme: internet-facing service.beta.kubernetes.io/aws-load-balancer-nlb-target-type: instance spec: type: LoadBalancer ports: - port: 30102 targetPort: 29092 name: kafka-broker-1 selector: app.kubernetes.io/component: dpod-cloud-agent-messaging app.kubernetes.io/instance: dpod-cloud-agent-prod app.kubernetes.io/name: dpod-cloud-agent-messaging app.kubernetes.io/part-of: dpod-cloud-agent statefulset.kubernetes.io/pod-name: dpod-cloud-agent-prod-msg-1 sessionAffinity: None
  • Deploy broker-2 LB service

apiVersion: v1 kind: Service metadata: name: dpod-cloud-agent-prod-msg-dir-lb-svc-2 namespace: example-dpod-cloudagent-ns labels: app.kubernetes.io/component: dpod-cloud-agent-manager app.kubernetes.io/instance: dpod-cloud-agent-prod app.kubernetes.io/managed-by: dpod-cloud-agent-operator app.kubernetes.io/name: dpod-cloud-agent-manager app.kubernetes.io/part-of: dpod-cloud-agent app.kubernetes.io/version: ${DPOD_CLOUD_AGENT_OPERATOR_VERSION} annotations: service.beta.kubernetes.io/aws-load-balancer-type: external service.beta.kubernetes.io/aws-load-balancer-scheme: internet-facing service.beta.kubernetes.io/aws-load-balancer-nlb-target-type: instance spec: type: LoadBalancer ports: - port: 30103 targetPort: 29092 name: kafka-broker-2 selector: app.kubernetes.io/component: dpod-cloud-agent-messaging app.kubernetes.io/instance: dpod-cloud-agent-prod app.kubernetes.io/name: dpod-cloud-agent-messaging app.kubernetes.io/part-of: dpod-cloud-agent statefulset.kubernetes.io/pod-name: dpod-cloud-agent-prod-msg-2 sessionAffinity: None

Create LoadBalancer services for DPOD Cloud Agent on Google GKE

Deploy Manager LB service

apiVersion: v1 kind: Service metadata: name: dpod-cloud-agent-prod-mng-lb-svc namespace: example-dpod-cloudagent-ns labels: app.kubernetes.io/component: dpod-cloud-agent-manager app.kubernetes.io/instance: dpod-cloud-agent-prod app.kubernetes.io/managed-by: dpod-cloud-agent-operator app.kubernetes.io/name: dpod-cloud-agent-manager app.kubernetes.io/part-of: dpod-cloud-agent app.kubernetes.io/version: ${DPOD_CLOUD_AGENT_OPERATOR_VERSION} annotations: cloud.google.com/l4-rbs: "enabled" spec: type: LoadBalancer ports: - port: 443 targetPort: 8443 name: https-manager selector: app.kubernetes.io/component: dpod-cloud-agent-manager app.kubernetes.io/instance: dpod-cloud-agent-prod app.kubernetes.io/name: dpod-cloud-agent-manager app.kubernetes.io/part-of: dpod-cloud-agent sessionAffinity: None

Deploy LoadBalancer service based on the number of messaging brokers replicas.

  • Deploy messaging bootstrap LB service

    apiVersion: v1 kind: Service metadata: name: dpod-cloud-agent-prod-msg-bse-lb-svc namespace: example-dpod-cloudagent-ns labels: app.kubernetes.io/component: dpod-cloud-agent-manager app.kubernetes.io/instance: dpod-cloud-agent-prod app.kubernetes.io/managed-by: dpod-cloud-agent-operator app.kubernetes.io/name: dpod-cloud-agent-manager app.kubernetes.io/part-of: dpod-cloud-agent app.kubernetes.io/version: ${DPOD_CLOUD_AGENT_OPERATOR_VERSION} annotations: cloud.google.com/l4-rbs: "enabled" spec: type: LoadBalancer ports: - port: 30100 targetPort: 29092 name: kafka-bootstrap selector: app.kubernetes.io/component: dpod-cloud-agent-messaging app.kubernetes.io/instance: dpod-cloud-agent-prod app.kubernetes.io/name: dpod-cloud-agent-messaging app.kubernetes.io/part-of: dpod-cloud-agent sessionAffinity: None
  • Deploy broker-0 LB service

apiVersion: v1 kind: Service metadata: name: dpod-cloud-agent-prod-msg-dir-lb-svc-0 namespace: example-dpod-cloudagent-ns labels: app.kubernetes.io/component: dpod-cloud-agent-manager app.kubernetes.io/instance: dpod-cloud-agent-prod app.kubernetes.io/managed-by: dpod-cloud-agent-operator app.kubernetes.io/name: dpod-cloud-agent-manager app.kubernetes.io/part-of: dpod-cloud-agent app.kubernetes.io/version: ${DPOD_CLOUD_AGENT_OPERATOR_VERSION} annotations: cloud.google.com/l4-rbs: "enabled" spec: type: LoadBalancer ports: - port: 30101 targetPort: 29092 name: kafka-broker-0 selector: app.kubernetes.io/component: dpod-cloud-agent-messaging app.kubernetes.io/instance: dpod-cloud-agent-prod app.kubernetes.io/name: dpod-cloud-agent-messaging app.kubernetes.io/part-of: dpod-cloud-agent statefulset.kubernetes.io/pod-name: dpod-cloud-agent-prod-msg-0 sessionAffinity: None
  • Deploy broker-1 LB service

    apiVersion: v1 kind: Service metadata: name: dpod-cloud-agent-prod-msg-dir-lb-svc-1 namespace: example-dpod-cloudagent-ns labels: app.kubernetes.io/component: dpod-cloud-agent-manager app.kubernetes.io/instance: dpod-cloud-agent-prod app.kubernetes.io/managed-by: dpod-cloud-agent-operator app.kubernetes.io/name: dpod-cloud-agent-manager app.kubernetes.io/part-of: dpod-cloud-agent app.kubernetes.io/version: ${DPOD_CLOUD_AGENT_OPERATOR_VERSION} annotations: cloud.google.com/l4-rbs: "enabled" spec: type: LoadBalancer ports: - port: 30102 targetPort: 29092 name: kafka-broker-1 selector: app.kubernetes.io/component: dpod-cloud-agent-messaging app.kubernetes.io/instance: dpod-cloud-agent-prod app.kubernetes.io/name: dpod-cloud-agent-messaging app.kubernetes.io/part-of: dpod-cloud-agent statefulset.kubernetes.io/pod-name: dpod-cloud-agent-prod-msg-1 sessionAffinity: None
  • Deploy broker-2 LB service

apiVersion: v1 kind: Service metadata: name: dpod-cloud-agent-prod-msg-dir-lb-svc-2 namespace: example-dpod-cloudagent-ns labels: app.kubernetes.io/component: dpod-cloud-agent-manager app.kubernetes.io/instance: dpod-cloud-agent-prod app.kubernetes.io/managed-by: dpod-cloud-agent-operator app.kubernetes.io/name: dpod-cloud-agent-manager app.kubernetes.io/part-of: dpod-cloud-agent app.kubernetes.io/version: ${DPOD_CLOUD_AGENT_OPERATOR_VERSION} annotations: cloud.google.com/l4-rbs: "enabled" spec: type: LoadBalancer ports: - port: 30103 targetPort: 29092 name: kafka-broker-2 selector: app.kubernetes.io/component: dpod-cloud-agent-messaging app.kubernetes.io/instance: dpod-cloud-agent-prod app.kubernetes.io/name: dpod-cloud-agent-messaging app.kubernetes.io/part-of: dpod-cloud-agent statefulset.kubernetes.io/pod-name: dpod-cloud-agent-prod-msg-2 sessionAffinity: None

Update the DNS records

  • Get the LB external address

    kubectl get svc -n example-dpod-cloudagent-ns |grep lb NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) dpod-cloud-agent-prod-mng-lb-svc LoadBalancer 10.100.12.184 34.41.36.111 443:30997/TCP dpod-cloud-agent-prod-msg-bse-lb-svc LoadBalancer 10.100.13.1 34.66.80.161 30100:31232/TCP dpod-cloud-agent-prod-msg-dir-lb-svc-0 LoadBalancer 10.100.12.9 35.223.187.112 30101:31830/TCP dpod-cloud-agent-prod-msg-dir-lb-svc-1 LoadBalancer 10.100.12.10 35.223.182.103 30101:31830/TCP dpod-cloud-agent-prod-msg-dir-lb-svc-2 LoadBalancer 10.100.12.11 35.223.116.42 30101:31830/TCP
  •  

 

Copyright © 2015 MonTier Software (2015) Ltd.