Konvoy supports environments where access to the Internet is restricted, and must be made through an HTTP/HTTPS proxy.

In these environments, configure Konvoy to use the HTTP/HTTPS proxy. In turn, Konvoy configures all core Kubernetes components to use the HTTP/HTTPS proxy.

NOTE: Konvoy follows a common convention for using an HTTP proxy server. The convention is based on three environment variables, and is supported by many, though not all, applications.

HTTP_PROXY: the HTTP proxy server address
HTTPS_PROXY: the HTTPS proxy server address
NO_PROXY: a list of IPs and domain names that are not subject to proxy settings

Konvoy downloads addon configurations to the bootstrap host, i.e., the host where the konvoy binary runs. If the addon configurations are not available on the bootstrap host, Konvoy accesses the Internet from the host.

Konvoy downloads software packages to cluster nodes. Kubernetes downloads container images to cluster nodes. If the packages or container images are not available on a cluster node, Konvoy or Kubernetes, respectively, access the Internet from that node.

IMPORTANT: Konvoy requires the http scheme to be used for both the HTTP and HTTPS proxy server. This is for two reasons: First, Konvoy uses Ansible to download software packages, and Ansible does not support the https scheme. Second, Konvoy does not support verification of the HTTP proxy server certificate. Note that, even when using the `http` scheme, the proxy server cannot read the body or headers of an HTTPS request.

Before you start

In the examples below:

The curl command-line tool is available on the host.
The proxy server address is http://proxy.company.com:3128.
The proxy server address uses the http scheme.
The proxy server can reach www.google.com using HTTP or HTTPS.

Verify that the bootstrap host can access the Internet through the proxy server

On the bootstrap host, run:

curl --proxy http://proxy.company.com:3128 --head http://www.google.com
curl --proxy http://proxy.company.com:3128 --head https://www.google.com

If the proxy is working for HTTP and HTTPS, respectively, the curl command returns a 200 OK HTTP response.

Verify that cluster nodes can access the Internet through the proxy server

On each cluster node, run:

curl --proxy http://proxy.company.com:3128 --head http://www.google.com
curl --proxy http://proxy.company.com:3128 --head https://www.google.com

If the proxy is working for HTTP and HTTPS, respectively, the curl command returns a 200 OK HTTP response.

Configure Konvoy

Set the HTTP_PROXY and HTTPS_PROXY environment variables to the address of the HTTP and HTTPS proxy server, respectively. Set the NO_PROXY environment variable to the addresses that should be accessed directly, not through the proxy.

IMPORTANT: Both the HTTP and HTTPS proxy server address must use the http scheme.

This example shows how to configure the HTTP proxy server, then run konvoy up:

export HTTP_PROXY=http://proxy.company.com:3128
export HTTPS_PROXY=http://proxy.company.com:3128
konvoy up

Configure Kubernetes core components and addons

Edit the cluster configuration file, cluster.yaml. Set httpProxy and httpsProxy to the HTTP and HTTPS proxy server address, respectively. Set noProxy to the addresses that should be accessed directly, not through the proxy.

IMPORTANT: Both the HTTP and HTTPS proxy server address must use the http scheme.

NOTE: In order to ensure core components work correctly, Konvoy always adds these addresses to noProxy:

Loopback addresses (127.0.0.1 and localhost)
Kubernetes API Server addresses
Kubernetes Pod IPs (e.g. 192.168.0.0/16)
Kubernetes Service addresses (e.g., 10.0.0.0/18, kubernetes, kubernetes.default, kubernetes.default.svc, kubernetes.default.svc.cluster, kubernetes.default.svc.cluster.local, .svc, .svc.cluster, .svc.cluster.local)

In this example, the proxy server is http://proxy.company.com:3128, and additional addresses that should be accessed directly are IPs in the 172.0.0.0/16 CIDR, and hosts in the .intra.net domain.

kind: ClusterConfiguration
apiVersion: konvoy.mesosphere.io/v1beta2
spec:
  kubernetes:
    networking:
      httpProxy: "http://proxy.company.com:3128"
      httpsProxy: "http://proxy.company.com:3128"
      noProxy: [ "172.0.0.0/16", ".intra.net" ]

In a cluster with the default configuration, the kommander and gatekeeper addons are installed. Gatekeeper will automatically configure the right proxy settings to kommander. If on the other hand, gatekeeper is not enabled then you will have to manually configure kommander.

To manually configure the addon to use the proxy settings, modify its values field as follows:

kind: ClusterConfiguration
apiVersion: konvoy.mesosphere.io/v1beta2
spec:
  addons:
    addonsList:
    - name: kommander
      enabled: true
      values: |
        kommander-federation:
          utilityApiserver:
            env:
              HTTP_PROXY: "http://proxy.company.com:3128"
              HTTPS_PROXY: "http://proxy.company.com:3128"
              NO_PROXY: "10.0.0.0/18,localhost,127.0.0.1,169.254.169.254,169.254.0.0/24,kubernetes.default,kubernetes.default.svc,kubernetes.default.svc.cluster,kubernetes.default.svc.cluster.local,.svc,.svc.cluster,.svc.cluster.local"
    ...

IMPORTANT: The NO_PROXY variable contains the Kubernetes Services CIDR. This example uses the default CIDR, 10.0.0.0/18. If your cluster's CIDR is different, update the value in NO_PROXY.

After you change the cluster configuration, apply the changes by running konvoy up.

Configure your applications

In a default installation with gatekeeper enabled, you can have proxy environment variables applied to all your pods automatically by adding the following label to your namespace:

"gatekeeper.d2iq.com/mutate": "pod-proxy"

No further manual changes are required.

IMPORTANT: If Gatekeeper is not installed, and you need to use a http proxy, then you must manually configure your addons as described further in this section.

Manually configure your application

Many, but not all, applications follow the convention of HTTP_PROXY, HTTPS_PROXY, and NO_PROXY environment variables.

In this example, the environment variables are set for a container in a Pod:

apiVersion: v1
kind: Pod
spec:
  containers:
  - name: example-container
    env:
    - name: HTTP_PROXY
      value: "http://proxy.company.com:3128"
    - name: HTTPS_PROXY
      value: "http://proxy.company.com:3128"
    - name: NO_PROXY
      value: "10.0.0.0/18,localhost,127.0.0.1,169.254.169.254,169.254.0.0/24,kubernetes.default,kubernetes.default.svc,kubernetes.default.svc.cluster,kubernetes.default.svc.cluster.local,.svc,.svc.cluster,.svc.cluster.local"

See Define Environment Variables for a Container for more details.

Configure HTTP Proxy