Ingress

Kubernetes is a powerful orchestration tool that helps manage containarized applications across a cluster of machines. This has obvious advantages over maintaining the entire cluster of machines ourselves. One of the key features is networking, which includes services and ingress resources. Ingress, in particular, plays a crucial role in routing external traffic to the appropriate services within kubernetes cluster. This blog would delve into a few different types of Ingresses available in Kubernetes and their functionalities.

What exactly is an Ingress in Kubernetes ?

An Ingress is a collection of rules that are applied on a load balancer. These rules dictate how the external traffic should reach the services which are hosted within Kubernetes cluster. It provides a way to expose myriad of protocols (HTTP, HTTPs, Websockets, GRPC) routes. These routes are genrally implemented by Ingress controllers.

What is an Ingress Controller ?

Ingress Controller like other controllers within Kubernetes acts to fullfill the configuration written within Ingress resource, usually with a load-balancer. It acts as a layer of abstraction over Load Balancer to add additional capabilities such as Traffic Routing, Security, Policy Enforcement etc..

There are currently multiple implementations to choose from for ingress controllers :

• AKS Application Gateway Ingress Controller is an ingress controller that configures the Azure Application Gateway.
• Ambassador API Gateway is an Envoy-based ingress controller.
• Apache APISIX ingress controller is an Apache APISIX-based ingress controller.
• Avi Kubernetes Operator provides L4-L7 load-balancing using VMware NSX Advanced Load Balancer.
• BFE Ingress Controller is a BFE-based ingress controller.
• Cilium Ingress Controller is an ingress controller powered by Cilium.
• The Citrix ingress controller works with Citrix Application Delivery Controller.
• Contour is an Envoy based ingress controller.
• EnRoute is an Envoy based API gateway that can run as an ingress controller.
• Easegress IngressController is an Easegress based API gateway that can run as an ingress controller.
• F5 BIG-IP Container Ingress Services for Kubernetes lets you use an Ingress to configure F5 BIG-IP virtual servers.
• Gloo is an open-source ingress controller based on Envoy, which offers API gateway functionality.
• HAProxy Ingress is an ingress controller for HAProxy.
• The HAProxy Ingress Controller for Kubernetes is also an ingress controller for HAProxy.
• Istio Ingress is an Istio based ingress controller.
• The Kong Ingress Controller for Kubernetes is an ingress controller driving Kong Gateway.
• Kusk Gateway is an OpenAPI-driven ingress controller based on Envoy.
• The NGINX Ingress Controller for Kubernetes works with the NGINX webserver (as a proxy).
• The ngrok Kubernetes Ingress Controller is an open source controller for adding secure public access to your K8s services using the ngrok platform.
• The Pomerium Ingress Controller is based on Pomerium, which offers context-aware access policy.
• Skipper HTTP router and reverse proxy for service composition, including use cases like Kubernetes Ingress, designed as a library to build your custom proxy.
• The Traefik Kubernetes Ingress provider is an ingress controller for the Traefik proxy.
• Tyk Operator extends Ingress with Custom Resources to bring API Management capabilities to Ingress. Tyk Operator works with the - Open Source Tyk Gateway & Tyk Cloud control plane.
• Voyager is an ingress controller for HAProxy.
• Wallarm Ingress Controller is an Ingress Controller that provides WAAP (WAF) and API Security capabilities.

Connecting Ingress and Ingress Controllers

You may deploy any number of ingress controllers using an object named Ingress Class within a cluster. Note the spec.controller below in the IngressClass definition this is where you would specify the required Ingress Controller that the IngressClass is linked to. A sample IngressClass would look as below :

apiVersion: networking.k8s.io/v1
kind: IngressClass
metadata:
  name: external-lb
spec:
  controller: example.com/ingress-controller
  parameters:
    apiGroup: k8s.example.com
    kind: IngressParameters
    name: external-lb

Any IngressClass scope is by default cluster-wide but can be altered using parameters .spec.parameters.scope this could be set to Cluster or Namespace. What that means is this IngressClass can be used across cluster or within a specified namespace alone. Also an IngressClass can be set as default IngressClass so that all Ingresses would use it by default.

Example for this is :

apiVersion: networking.k8s.io/v1
kind: IngressClass
metadata:
  name: external-lb-1
  annotations:
    ingressclass.kubernetes.io/is-default-class: "true"
spec:
  controller: example.com/ingress-controller
  parameters:
    # The parameters for this IngressClass are specified in a
    # ClusterIngressParameter (API group k8s.example.net) named
    # "external-config-1". This definition tells Kubernetes to
    # look for a cluster-scoped parameter resource.
    scope: Cluster
    apiGroup: k8s.example.net
    kind: ClusterIngressParameter
    name: external-config-1

Again a sample Ingress using the above IngressClass would be (as the above is a default IngressClass we need not define it in Ingress) :

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: ingress-wildcard-host
spec:
  rules:
  - host: "foo.bar.com"
    http:
      paths:
      - pathType: Prefix
        path: "/bar"
        backend:
          service:
            name: service1
            port:
              number: 80
  - host: "*.foo.com"
    http:
      paths:
      - pathType: Prefix
        path: "/foo"
        backend:
          service:
            name: service2
            port:
              number: 80

Although above example is a good start you might find yourself in need of a different IngressClasses for different Ingress often Implemented by different implementations. (Istio and nginx or nginx and HA proxy). In this situation each Ingress should specify a class, a reference to an IngressClass resource that contains the details of how the rules in Ingress should be applied on the load-balancer. Note the .metadata.name of your IngressClass resource. When you create an Ingress resource you would need that name to specify the ingressClassName field on your Ingress object (refer to IngressSpec v1 reference).

How to specify IngressClass within an Ingress Resource ?

There have been atleast a few ways to mention the IngressClass within Ingress.

Deprecated kubernetes.io/ingress.class

prior to Kubernetes 1.18 Ingress classes were specified with an annotation kubernetes.io/ingress.class on the Ingress. This annotation was never formally defined but, was widely supported by Ingress controllers.

Latest Method

Moving on from the Kubernetes 1.18 version there is a new field named ingressClassName. This is a replacement of the older annotation method but is not a direct equivalent.

An example of using this latest method is :

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: ingress-with-class
  namespace: default
spec:
  ingressClassName: my-ingress-class
  rules:
  - host: example.com
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: my-service
            port:
              number: 80
  tls:
  - hosts:
    - example.com
    secretName: example-tls

Above you’ll notice under .spec.ingressClassName is the value of the specific IngressClass within the definition of Ingress. This helps to identify which IngressClass and hence the IngressController the above Ingress should refer to.

Using IngressClass in the Ingress resources allows you to manage multiple Ingress Controllers efficiently. It provides flexibility in directing the traffic and ensures that specific Ingress Controllers handle the appropriate requests. This setup is especially advantageous in complex environments where different teams bring their own Ingress Controllers that they would prefer to use rather than a default Ingress Controller for all.

Common source of confusion

Usually the popular options for Ingress Controller are one of the below This list may differ but usually has the mix of choice from nginx, ha-proxy or envoy as the underlying rever-proxy server to translate the configurations. :

• Kuberenetes Ingress
• Nginxinc Ingress
• Istio Ingress
• HA Proxy Ingress

The top two usually become a source of confusion as both uses nginx as the reverse-proxy but one is k8s community built and the other is also opensource but sponsored and controlled by Nginx Inc. To add to the confusion there’s a commercial product from Nginx Inc. called NginxPlus which has advanced capabilities to support cross cluster routing and much more.

For a differences between the three please check out this gist.