Skip to content

Istio

Learn how and why deployKF uses Istio. Learn how to integrate your existing Istio with deployKF and Kubeflow.


What is Istio?

Istio is a service mesh for Kubernetes which is based around the Envoy proxy. A service mesh is a dedicated infrastructure layer for managing service-to-service network communication.

Istio changes Kubernetes Pod definitions (dynamically at runtime) so they have a sidecar container (Envoy proxy) which is configured to intercept all network traffic to and from the Pod. Together, these sidecars form a mesh network that can implement advanced networking features like Traffic Management, Security, and Observability with minimal changes to the application code.

How is Istio configured?

The Istio mesh is configured declaratively using Kubernetes Custom Resources (CRDs), so you don't need to configure the Envoy proxies directly. Some of the most important Istio CRDs are: Gateway, VirtualService, DestinationRule, ServiceEntry, PeerAuthentication, AuthorizationPolicy and EnvoyFilter.

How can external traffic access the mesh?

Mutual TLS

The Envoy sidecar uses Mutual TLS (mTLS) to verify if network traffic destined for the Pod is coming from within the mesh. Based on the PeerAuthentication policy for the Pod, the Envoy sidecar will either allow or deny external traffic (not from within the mesh).

Istio PeerAuthentication Policies

If external traffic is allowed to reach the Pod (like when PeerAuthentication has an mtls.mode of PERMISSIVE), it will be accessing the Pod directly, effectively bypassing the mesh. This means that AuthorizationPolicy and EnvoyFilter policies would not be applied to that traffic.

Gateways

To expose services in the mesh to external traffic (e.g. from the internet), Istio provides the concept of a "Gateway", which sits at the edge of the mesh and can route external traffic to virtual services defined in the mesh.

The idea of "Gateways" is a common source of confusion for new Istio users because it refers to two different things.

Gateway Deployments Gateway Resources
A Kubernetes Deployment of special Envoy proxy Pods which act as an "entry point" to the mesh. The Gateway is the CRD which configures the Envoy proxies of a "Gateway Deployment", and can be selected by VirtualServices to define routes to Pods in the mesh.

Here are examples of the two different types of "Gateway" in Istio:

Example - Gateway Deployment

The following Deployment will be automatically mutated by Istio to include an Envoy proxy sidecar container:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: my-gateway-deployment
  namespace: gateway-namespace
spec:
  selector:
    matchLabels:
      istio: my-gateway-deployment
  template:
    metadata:
      annotations:
        ## this tells istio to inject using the "gateway" template,
        ## rather than the "sidecar" template (which is the default)
        inject.istio.io/templates: "gateway"
      labels:
        ## this ensures that the istio-proxy container is injected
        sidecar.istio.io/inject: "true"

        ## the pod label (same as any normal kubernetes deployment)
        ## but which is also used by Gateway resources to select these pods
        istio: my-gateway-deployment
    spec:
      ## allow binding to all ports (such as 80 and 443)
      securityContext:
        sysctls:
          - name: net.ipv4.ip_unprivileged_port_start
            value: "0"
      containers:
        - name: istio-proxy

          ## the image is automatically replaced by istio
          image: auto

          ## drop all privileges, allowing running as non-root
          securityContext:
            capabilities:
              drop:
                - ALL
            runAsUser: 1337
            runAsGroup: 1337

The following Service exposes the "Gateway Deployment" from above with a LoadBalancer type so it can be accessed from outside the cluster:

apiVersion: v1
kind: Service
metadata:
  name: my-gateway-service
  namespace: gateway-namespace
spec:
  type: LoadBalancer
  selector:
    istio: my-gateway-deployment
  ports:
    - port: 80
      name: http
    - port: 443
      name: https
Example - Gateway Resource

The following Gateway resource configures the "Gateway Deployment" from above:

apiVersion: networking.istio.io/v1beta1
kind: Gateway
metadata:
  name: my-gateway
  namespace: gateway-namespace
spec:
  ## a selector for Pod labels, which selects the "Gateway Deployment"
  selector:
    istio: my-gateway-deployment
  servers:
    - port:
        number: 80
        name: http
        protocol: HTTP
      hosts:
        - "*"
    - port:
        number: 443
        name: https
        protocol: HTTPS
      hosts:
        - "*"
      tls:
        mode: SIMPLE
        ## the name of the Kubernetes Secret with a TLS certificate
        credentialName: my-gateway-certificate

The following VirtualService selects the Gateway defined above, and routes traffic to an application Service named my-service:

apiVersion: networking.istio.io/v1beta1
kind: VirtualService
metadata:
  name: my-virtual-service
  ## virtual services can be in any namespace, not only the same as the gateway
  namespace: my-service-namespace
spec:
  hosts:
    ## you could replace this with a specific hostname if you want
    ## to do Host-based routing, so multiple services can share the same gateway/port
    - "*"
  gateways:
    ## this selects the gateway defined above
    - some-namespace/my-gateway
  http:
    - route:
        ## this assumes there is a `Service` named `my-service` in `my-service-namespace`
        - destination:
            host: my-service.my-service-namespace.svc.cluster.local
            port:
              number: 80

TLS Termination

In this example, the backend service listens for HTTP traffic (port 80). However, because the gateway is doing TLS termination, end-clients can access the service over HTTPS (port 443).


How does deployKF use Istio?

deployKF uses Istio as a service mesh to provide advanced networking features.

Here are some of the ways deployKF itself uses Istio:

Feature Implementation
External Traffic Routing Services are exposed to non-mesh traffic via Gateways and VirtualServices.

(Learn More: Expose Gateway)
Authentication & Authorization External traffic is authenticated using EnvoyFilters, then authorized with AuthorizationPolicies.

(Learn More: User Authentication)
Internal Service Communication Internal services talk through the mesh, and service-to-service access controls are enforced with AuthorizationPolicies to restrict which services can talk to each other.

Additionally, some tools in deployKF make direct use of Istio:

Tool How it uses Istio
Kubeflow Notebooks Manages VirtualServices for each Notebook Pod to make it accessible on the main Istio gateway.

Can I use my existing Istio?

Yes.

By default, deployKF will install Istio and create an ingress gateway deployment. However, you may use your existing Istio installation and/or gateway deployment instead.

As the gateway deployment is separate from Istio itself, there are 3 common combinations of who manages what:

Configuration Istio Installation Gateway Deployment Gateway Resources
Default deployKF deployKF Always by deployKF
Custom Istio, Managed Gateway You deployKF Always by deployKF
Fully Custom You You Always by deployKF

Use an existing istio installation

If you already have an Istio installation, you may use it instead of the deployKF-managed one by following these steps. See the version matrix for which versions of Istio are supported by deployKF.

Gateway Version Alignment

If you are NOT also bringing your own gateway deployment, you MUST ensure that the deployKF-managed gateway matches your Istio version. The deploykf_core.deploykf_istio_gateway.charts.istioGateway.version value sets the version of the embedded gateway deployment.

For example, the following deployKF values will deploy a gateway for Istio 1.19.6:

deploykf_core:
  deploykf_istio_gateway:
    charts:
      istioGateway:
        name: gateway
        version: 1.19.6
        repository: https://istio-release.storage.googleapis.com/charts
Step 1 - Disable embedded Istio

Disable the embedded Istio installation by setting the deploykf_dependencies.istio.enabled value to false:

deploykf_dependencies:
  istio:
    enabled: false
Step 2 - Configure your Istio

deployKF requires some non-default mesh configs which you MUST set in your Istio installation:

Mesh Config Value Purpose
defaultConfig.holdApplicationUntilProxyStarts true Ensures the Istio sidecar is fully initialized before application containers start. Prevents race-conditions where application containers start before the sidecar is ready.
defaultConfig.proxyMetadata { "ISTIO_META_DNS_AUTO_ALLOCATE": "true", "ISTIO_META_DNS_CAPTURE": "true" } Enable DNS Proxying, which deployKF requires.

Default Namespace Injection

Ensure you do NOT have sidecarInjectorWebhook.enableNamespacesByDefault set to true.
(In the Istio Helm Chart this defaults to false, but you should check to be sure).

For example, if you are using the Istio Helm Chart to install Istio, you may set these Helm values:

meshConfig:
  defaultConfig:
    holdApplicationUntilProxyStarts: true
    proxyMetadata:
      ISTIO_META_DNS_AUTO_ALLOCATE: "true"
      ISTIO_META_DNS_CAPTURE: "true"

sidecarInjectorWebhook:
  enableNamespacesByDefault: false

Use an existing gateway deployment

If you have an existing Istio gateway deployment, you can use it instead of the deployKF-managed one. You may do this even when using the deployKF-managed Istio installation.

Step 1 - Disable embedded Gateway Deployment

Disable the embedded gateway deployment by setting the deploykf_core.deploykf_istio_gateway.charts.istioGateway.enabled value to false:

deploykf_core:
  deploykf_istio_gateway:

    ## disable the embedded gateway deployment
    charts:
      istioGateway:
        enabled: false
Step 2 - Configure deployKF

You must set the following deployKF values to match your existing gateway deployment:

For example, you might set the following values:

deploykf_core:
  deploykf_istio_gateway:

    ## the namespace where your gateway deployment is running
    namespace: my-gateway-namespace

    gateway:
      ## the label selector for your gateway deployment pods
      selectorLabels:
        app: my-gateway-deployment
        istio: my-gateway-deployment

      ## the ports on your gateway deployment which deployKF should use
      ##  - the "internal" ports which deployKF will use on your gateway deployment
      ##  - they must be different from any existing services on this gateway deployment
      ##  - they may be different to the user-facing ports, as users might connect
      ##    to the gateway through a Service or Ingress (see `gatewayService.ports`)
      ports:
        http: 80
        https: 443

    gatewayDeployment:
      serviceAccount:
        ## the name of the SERVICE ACCOUNT used by the gateway deployment pods
        ##  - deployKF needs to know this so it can trust traffic from the gateway
        ##    check the `serviceAccountName` field in your gateway Pods
        name: my-gateway-service-account

    #gatewayService:
    #
    #  ## the ports which clients are actually using to connect to the gateway
    #  ##  - the "public" ports which clients are expected to connect to
    #  ##  - they can be different to the "internal" ports defined in `gateway.ports`
    #  ##  - these values affect the ports presented in user-facing HTTP links
    #  ##  - if unset, they default to the corresponding value of `gateway.ports`
    #  ports:
    #    http: 80
    #    https: 443
Step 3 - Expose your Gateway Deployment

If you havent already, you will need to create a Service (and possibly Ingress) that selects your gateway deployment to expose it to external traffic.

Service Health Checks

Many LoadBalancer Service implementations require a "health check" to pass before allowing traffic to flow (e.g. AWS NLB/ALB), and will send health-check requests to one or more ports on the Service.

Istio gateway Pods will always return a 200 OK response on port 15021, under the /healthz/ready HTTP path for this purpose. Therefore, you can expose the 15021 port on the Service, and configure the health-check path to /healthz/ready.

TLS Termination and SNI

If you put the Gateway behind a proxy which terminates TLS (like AWS ALB), you will probably need to disable SNI Matching. This is because most proxies don't forward the original request's Server Name Indication (SNI) to the backend service after TLS termination.

To disable SNI Matching, set deploykf_core.deploykf_istio_gateway.gateway.tls.matchSNI to false:

deploykf_core:
  deploykf_istio_gateway:

    gateway:
      tls:
        matchSNI: false

Read more about this in the Expose Gateway and configure HTTPS guide.

Can I have other services on the deployKF Gateway?

Yes. You may expose your non-deployKF Gateway and VirtualService resources on the same Gateway Deployment as deployKF, as long as the ports/hostnames are not incompatible with deployKF's configuration.

Limitations in deployKF 0.1.3 and earlier

In deployKF 0.1.3 and earlier, you MUST use a DEDICATED gateway deployment for deployKF. That is, you can't expose non-deployKF services on the same gateway deployment as deployKF.

This limitation was removed in deployKF 0.1.4.

Using non-standard ports

If you already have Istio VirtualServices on your gateway deployment using ports 80 and 443, you will need to use non-standard ports (like 18080 and 18443) for deployKF.

For example, you might set the following values to use non-standard ports:

deploykf_core:
  deploykf_istio_gateway:

    ## the ports on your gateway deployment which deployKF should use
    gateway:
      ports:
        http: 18080
        https: 18443

    ## the ports which clients are actually using to connect to the gateway
    gatewayService:
      ports:
        http: 80
        https: 443

You will probably also want to use an Ingress which listens on standard ports, and routes traffic to the correct gateway port based on the hostname. This will prevent users from seeing the non-standard ports in their URLs like https://deploykf.example.com:18443.

For example, your Ingress could route traffic like this:

  • other-service.example.com → gateway port 443
  • deploykf.example.com → gateway port 18443
  • *.deploykf.example.com → gateway port 18443

Use custom gateway resources

You are NOT able to use your own Gateway and VirtualService resources for deployKF.

While you may attach deployKF to an existing Gateway Deployment (Pods + Service), ALL virtual Gateway and VirtualService resources are managed by deployKF, this is a result of how deployKF implements features like authentication.

For reference, here are some gateway resources that deployKF creates:

Resources Purpose
Gateway/deploykf-istio-gateway The main gateway that exposes the platform to external traffic.
Gateway/deploykf-istio-gateway-https-redirect A special gateway for HTTP to HTTPS redirects.
VirtualService/https-redirect Redirects HTTP traffic to HTTPS, connected to Gateway/deploykf-istio-gateway-https-redirect.
VirtualService/deploykf-istio-gateway Routes for the central dashboard.

Last update: 2024-07-24
Created: 2024-01-16