Networking

This section describes different networking components that come together to form a Konvoy networking stack. It assumes familiarity with Kubernetes networking.

Service

A Service is an API resource which defines a logical set of pods and a policy by which to access them. Services are an abstracted manner to expose applications as network services.

Kubernetes gives pods their own IP addresses and a single DNS name for a set of pods. Services are used as entrypoints to load-balance the traffic across the pods. A selector determines the set of Pods targeted by a Service.

For example, if you have a set of pods that each listen on TCP port 9191 and carry a label app=MyKonvoyApp, as configured in the following:

apiVersion: v1
kind: Service
metadata:
  name: my-konvoy-service
  namespace: default
spec:
  selector:
    app: MyKonvoyApp
  ports:
    - protocol: TCP
      port: 80
      targetPort: 9191

This specification creates a new Service object named "my-konvoy-service", that targets TCP port 9191 on any pod with the app=MyKonvoyApp label.

Kubernetes assigns this Service an IP address. In particular, the kube-proxy implements a form of virtual IP for Services of type other than ExternalName.

NOTE: The name of a Service object must be a valid DNS label name.

EndpointSlices

EndpointSlices are an API resource that appeared as a scalable and more manageable solution to network endpoints within a Kubernetes cluster. EndpointSlices allow for distributing network endpoints across multiple resources with a limit of 100 endpoints per EndpointSlice.

An EndpointSlice contains references to a set of endpoints. The control plane takes care of creating EndpointSlices for any Service that has a selector specified. These EndpointSlices include references to all the pods that match the Service selector.

Like Services, the name of a EndpointSlice object must be a valid DNS subdomain name.

In the following, here’s a sample EndpointSlice resource for the example Kubernetes Service:

apiVersion: discovery.k8s.io/v1beta1
kind: EndpointSlice
metadata:
  name: konvoy-endpoint-slice
  namespace: default
  labels:
    kubernetes.io/service-name: my-konvoy-service
addressType: IPv4
ports:
  - name: http
    protocol: TCP
    port: 80
endpoints:
  - addresses:
      - "192.168.126.168"
    conditions:
      ready: true
    hostname: ip-10-0-135-39.us-west-2.compute.internal
    topology:
      kubernetes.io/hostname: ip-10-0-135-39.us-west-2.compute.internal
      topology.kubernetes.io/zone: us-west2-b

DNS for Services and Pods

Every new Service object in Kubernetes gets assigned a DNS name. The Kubernetes DNS component schedules a DNS name for the pods and services created on the cluster. Then the Kubelets are configured so containers can resolve these DNS names.

From our previous examples, assume there is a Service named my-konvoy-service in the Kubernetes namespace default. A Pod running in namespace default can look up this service by simply doing a DNS query for my-konvoy-service. A Pod running in namespace kommander can look up this service by doing a DNS query for my-konvoy-service.default.

In general, a pod has the following DNS resolution:

pod-ip-address.namespace-name.pod-name.cluster-domain.example.

Similarly, a service has the following DNS resolution:

service-name.namespace-name.svc.cluster-domain.example.

You can find additional information about all the possible record types and layout here.

Ingress

Ingress is an API resource that manages external access to the services in a cluster through HTTP or HTTPS. It offers name-based virtual hosting, SSL termination and load balancing when exposing HTTP/HTTPS routes from outside to services in the cluster.

The traffic policies are controlled by rules as part of the Ingress definition. Each rule defines the following details:

• An optional host to which apply the rules.

• A list of paths or routes which has an associated backend defined with a Service name, a port name and number.

• A backend is a combo of a Service and port names, or a custom resource backend defined as a CRD. Consequently HTTP/HTTPS requests to the Ingress that matches the host and path of the rule are sent to the listed backend.

An example of an Ingress specification is:

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: konvoy-ingress
  namespace: default
  annotations:
    nginx.ingress.kubernetes.io/rewrite-target: /
spec:
  rules:
  - http:
      paths:
      - path: /path
        pathType: Prefix
        backend:
          service:
            name: my-konvoy-service
            port:
              number: 80

In Konvoy, we expose services to the outside world using Ingress objects.

Ingress Controllers

In contrast with other controllers in the Kubernetes control plane, the Ingress controllers are not started with a cluster. Users need to choose the desired Ingress controller.

An Ingress controller has to be deployed in a cluster for the Ingress definitions to work.

Kubernetes as a project currently supports and maintains GCE and nginx controllers.

These are four of the most known ingress controller:

• HAProxy Ingress is a highly customizable community-driven ingress controller for HAProxy.

• NGINX offers support and maintenance for the NGINX Ingress Controller for Kubernetes.

• Traefik is a fully featured ingress controller (Let’s Encrypt, secrets, http2, websocket), and has commercial support.

• Ambassador API Gateway Experimental is an Envoy based ingress controller with community and commercial support.

In Konvoy, we deploy by default Traefik as a well-suited Ingress controller.

Network Policies

NetworkPolicy is an API resource that controls the traffic flow at port level 3 or 4, or at the IP address level. It enables defining constraints on how a pod communicates with various network services such as endpoints and services.

A Pod can be restricted to talk to other network services through a selection of the following identifiers:

• Namespaces that have to access. There can be pods that are not allowed to talk to other namespaces.

• Other allowed IP blocks regardless of the node or IP address assigned to the targeted Pod.

• Other allowed Pods.

An example of a NetworkPolicy specification is:

apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: network-konvoy-policy
  namespace: default
spec:
  podSelector:
    matchLabels:
      role: db
  policyTypes:
  - Ingress
  - Egress
  ingress:
  - from:
    - ipBlock:
        cidr: 172.17.0.0/16
        except:
        - 172.17.1.0/24
    - namespaceSelector:
        matchLabels:
          app: MyKonvoyApp
    - podSelector:
        matchLabels:
          app: MyKonvoyApp
    ports:
    - protocol: TCP
      port: 6379
  egress:
  - to:
    - ipBlock:
        cidr: 10.0.0.0/24
    ports:
    - protocol: TCP
      port: 5978

As shown in the example, when defining a pod or namespace based NetworkPolicy, you use a selector to specify what traffic is allowed to and from the Pod(s).

Adding entries to Pod /etc/hosts with HostAliases

The Pod API resource definition has a HostAliases field that allows adding entries to the Pod’s container /etc/hosts file. This field overrides the hostname resolution when DNS and other options are not applicable.

For example, to resolve foo.node.local, bar.node.local to 127.0.0.1 and foo.node.remote, bar.node.remote to 10.1.2.3, configure the HostAliases values as follows:

apiVersion: v1
kind: Pod
metadata:
  name: hostaliases-konvoy-pod
spec:
  restartPolicy: Never
  hostAliases:
  - ip: "127.0.0.1"
    hostnames:
    - "foo.node.local"
    - "bar.node.local"
  - ip: "10.1.2.3"
    hostnames:
    - "foo.node.remote"
    - "bar.node.remote"
  containers:
  - name: cat-hosts
    image: busybox
    command:
    - cat
    args:
    - "/etc/hosts"