Create a New Cluster

Several small procedures work together to create a new Azure Kubernetes cluster. Be sure to read each one carefully, as some are optional. The basic process is:

Satisfy the prerequisites.
Name your cluster.
Create new Azure Kubernetes cluster objects.
Inspect and edit the cluster objects

Editing the objects allows you to use the optional procedures for:
- Specifying an HTTP proxy
- Configuring the cluster to use existing network infrastructure
Create the actual Azure Kubernetes cluster

Be sure also that you review the known limitations section

Prerequisites

Before you begin, make sure you have created a Bootstrap cluster.

Name your cluster

Give your cluster a unique name suitable for your environment.
Set the environment variable:
```
export CLUSTER_NAME=azure-example
```

Naming Tips and Tricks

To create a cluster name that is unique, use the following command:
```
export CLUSTER_NAME=azure-example-cluster-$(LC_CTYPE=C tr -dc 'a-z0-9' </dev/urandom | fold -w 5 | head -n1)
echo $CLUSTER_NAME
```
The output appears similar to:
```
azure-example-cluster-pf4a3
```
This command creates a unique name every time you run it, so use it carefully.

Create new Azure Kubernetes cluster objects

This procedure uses the --dry-run and --output-yaml flags together to create basic Azure Kubernetes cluster objects in a YAML file. This approach allows you to examine the YAML objects before creating the actual Azure Kubernetes cluster itself.

When creating the basic Azure Kubernetes cluster objects, you need to first consider whether you need to use an HTTP proxy. If you do, you need to do some additional configuration when creating the cluster objects. Consult the optional “Configure the control plane and workers to use an HTTP Proxy” section for more details.

Generate the basic Kubernetes cluster objects:

dkp create cluster azure --cluster-name=${CLUSTER_NAME} \
--dry-run \
--output=yaml \
> ${CLUSTER_NAME}.yaml

The output of this command is a ${CLUSTER_NAME}.yaml file that you can examine or modify. If you use this method to create a basic cluster without HTTP proxies, skip to the heading, “Inspect or edit the cluster objects.”

(Optional) Configure the control plane and workers to use an HTTP Proxy

To configure the Control Plane and Worker nodes to use an HTTP proxy:

Copy the commands in the following code block to an editor and apply the list of edits that follows to customize them, then execute them from the command line:

export CONTROL_PLANE_HTTP_PROXY=http://example.org:8080
export CONTROL_PLANE_HTTPS_PROXY=http://example.org:8080
export CONTROL_PLANE_NO_PROXY="example.org,example.com,example.net,localhost,127.0.0.1,10.96.0.0/12,192.168.0.0/16,kubernetes,kubernetes.default,kubernetes.default.svc,kubernetes.default.svc.cluster,kubernetes.default.svc.cluster.local,.svc,.svc.cluster,.svc.cluster.local,169.254.169.254,.cloudapp.azure.com"

export WORKER_HTTP_PROXY=http://example.org:8080
export WORKER_HTTPS_PROXY=http://example.org:8080
export WORKER_NO_PROXY="example.org,example.com,example.net,localhost,127.0.0.1,10.96.0.0/12,192.168.0.0/16,kubernetes,kubernetes.default,kubernetes.default.svc,kubernetes.default.svc.cluster,kubernetes.default.svc.cluster.local,.svc,.svc.cluster,.svc.cluster.local,169.254.169.254,.cloudapp.azure.com"

Replace example.org,example.com,example.net with your internal addresses
localhost and 127.0.0.1 addresses should not use the proxy
10.96.0.0/12 is the default Kubernetes service subnet
192.168.0.0/16 is the default Kubernetes pod subnet
kubernetes,kubernetes.default,kubernetes.default.svc,kubernetes.default.svc.cluster,kubernetes.default.svc.cluster.local is the internal Kubernetes kube-apiserver service
.svc,.svc.cluster,.svc.cluster.local is the internal Kubernetes services
169.254.169.254 is the Azure metadata server
.cloudapp.azure.com allows the worker nodes to communicate directly to the kube-apiserver load balancer

Copy and run the following command to create a Kubernetes cluster with HTTP proxy configured. (This step assumes you did not already create a cluster in the previous procedure.)

dkp create cluster azure --cluster-name=${CLUSTER_NAME} \
--control-plane-http-proxy="${CONTROL_PLANE_HTTP_PROXY}" \
--control-plane-https-proxy="${CONTROL_PLANE_HTTPS_PROXY}" \
--control-plane-no-proxy="${CONTROL_PLANE_NO_PROXY}" \
--worker-http-proxy="${WORKER_HTTP_PROXY}" \
--worker-https-proxy="${WORKER_HTTPS_PROXY}" \
--worker-no-proxy="${WORKER_NO_PROXY}" \
--dry-run \
--output=yaml \
> ${CLUSTER_NAME}.yaml

The output of this command is a ${CLUSTER_NAME}.yaml file that you can examine or modify further.

Inspect or edit the cluster objects

Inspect or edit the cluster objects:

NOTE: Familiarize yourself with Cluster API before editing the cluster objects as edits may prevent the cluster from deploying successfully.

The objects are Custom Resources defined by Cluster API components, and they belong in three different categories:
1. Cluster
  
  A Cluster object has references to the infrastructure-specific and control plane objects. Because this is an Azure cluster, there is an AzureCluster object that describes the infrastructure-specific cluster properties. Here, this means the Azure region, the VPC ID, subnet IDs, and security group rules required by the Pod network implementation.
2. Control Plane
  
  A KubeadmControlPlane object describes the control plane, which is the group of machines that run the Kubernetes control plane components, which include the etcd distributed database, the API server, the core controllers, and the scheduler. The object describes the configuration for these components. The object also has a reference to an infrastructure-specific object that describes the properties of all control plane machines. Here, it references an AzureMachineTemplate object, which describes the instance type, the type of disk used, and the size of the disk, among other properties.
3. Node Pool
  
  A Node Pool is a collection of machines with identical properties. For example, a cluster might have one Node Pool with large memory capacity, another Node Pool with GPU support. Each Node Pool is described by three objects: The MachinePool references an object that describes the configuration of Kubernetes components (for example, the kubelet) deployed on each node pool machine, and an infrastructure-specific object that describes the properties of all node pool machines. Here, it references a KubeadmConfigTemplate, and an AzureMachineTemplate object, which describes the instance type, the type of disk used, the size of the disk, among other properties.
For in-depth documentation about the objects, read Concepts in the Cluster API Book.

(Optional) Configure existing network infrastructure in the cluster

As part of inspecting and editing your cluster objects, you can also configure it to use existing network infrastructure. If you do not need to use an existing network infrastructure, you can skip this step.

Review the following AzureCluster excerpt, noting the entries under networkSpec for the apiServerLB, nodeOutboundLB, subnets, and vnet values:

---
apiVersion: infrastructure.cluster.x-k8s.io/v1alpha4
kind: AzureCluster
metadata:
  creationTimestamp: "2021-12-17T20:25:12Z"
  generation: 1
  name: azure-example
  namespace: default
  uid: 64851501-f658-4b40-84a7-6a9f7c871629
spec:
  additionalTags:
    konvoy.d2iq.io_cluster-name: azure-example
    konvoy.d2iq.io_version: v2.1.0
  azureEnvironment: AzurePublicCloud
  bastionSpec: {}
  controlPlaneEndpoint:
    host: ""
    port: 0
  location: westus
  networkSpec:
    apiServerLB:
      frontendIPs:
      - name: azure-example-public-lb-frontEnd
        publicIP:
          name: pip-azure-example-apiserver
      idleTimeoutInMinutes: 4
      name: azure-example-public-lb
      sku: Standard
      type: Public
    nodeOutboundLB:
      frontendIPs:
      - name: azure-example-frontEnd
        publicIP:
          name: pip-azure-example-node-outbound
      frontendIPsCount: 1
      idleTimeoutInMinutes: 4
      name: azure-example
      sku: Standard
      type: Public
    subnets:
    - cidrBlocks:
      - 10.0.0.0/16
      name: azure-example-controlplane-subnet
      natGateway:
        ip:
          name: ""
      role: control-plane
      routeTable: {}
      securityGroup:
        name: azure-example-controlplane-nsg
    - cidrBlocks:
      - 10.1.0.0/16
      name: azure-example-node-subnet
      natGateway:
        ip:
          name: ""
      role: node
      routeTable:
        name: azure-example-node-routetable
      securityGroup:
        name: azure-example-node-nsg
    vnet:
      cidrBlocks:
      - 10.0.0.0/8
      name: azure-example-vnet
      resourceGroup: azure-example
  resourceGroup: azure-example

After you make and verify changes in these areas, save the file and go to the “create the actual cluster” procedure.

Create the actual Azure Kubernetes cluster

Use this procedure to create the Azure Kubernetes cluster when you finish your inspection and edits.

Create the cluster from the generated, and any modified, YAML objects using the command:

kubectl apply -f ${CLUSTER_NAME}.yaml

The output appears similar to:

cluster.cluster.x-k8s.io/azure-example created
azurecluster.infrastructure.cluster.x-k8s.io/azure-example created
kubeadmcontrolplane.controlplane.cluster.x-k8s.io/azure-example-control-plane created
azuremachinetemplate.infrastructure.cluster.x-k8s.io/azure-example-control-plane created
clusterresourceset.addons.cluster.x-k8s.io/calico-installation-azure-example created
configmap/calico-cni-azure-example created
machinedeployment.cluster.x-k8s.io/azure-example-md-0 created
azuremachinetemplate.infrastructure.cluster.x-k8s.io/azure-example-md-0 created
kubeadmconfigtemplate.bootstrap.cluster.x-k8s.io/azure-example-md-0 created

Wait for the cluster’s control-plane Status to be Ready:
```
kubectl wait --for=condition=ControlPlaneReady "clusters/${CLUSTER_NAME}" --timeout=20m
```
When the control plane status is Ready, the output is similar to:
```
cluster.cluster.x-k8s.io/azure-example condition met
```
After DKP creates the objects on the API server, the Cluster API controllers reconcile them. They create infrastructure and machines, and as they progress, they update the Status of each object.

Run the DKP Konvoy command to describe the current status of the cluster:

dkp describe cluster -c ${CLUSTER_NAME}

The output is similar to:

NAME                                                              READY  SEVERITY  REASON  SINCE  MESSAGE
/azure-example                                                    True                     6m37s
├─ClusterInfrastructure - AzureCluster/azure-example              True                     13m
├─ControlPlane - KubeadmControlPlane/azure-example-control-plane  True                     6m37s
│ └─3 Machines...                                                 True                     10m    See azure-example-control-plane-bmc9b, azure-example-control-plane-msftd, ...
└─Workers
└─MachineDeployment/azure-example-md-0                            True                     7m58s
└─4 Machines...                                                   True                     8m10s  See azure-example-md-0-84bd8b5f5b-b8cnq, azure-example-md-0-84bd8b5f5b-j8ldg, ...

As they progress, the controllers also create Events. List the Events using the command:

kubectl get events | grep ${CLUSTER_NAME}

For brevity, the example command also uses grep. You can use separate commands to get Events for specific objects, for example, kubectl get events --field-selector involvedObject.kind="AzureCluster" and kubectl get events --field-selector involvedObject.kind="AzureMachine".