Kubernetes Ingress Controllers: Why Your Traffic Routing Choice Affects More Than Latency

Most teams pick an ingress controller the same way they pick a font: grab whatever looks familiar, ship it, and move on. NGINX shows up everywhere, so NGINX it is. That decision is fine until it isn't — and when it stops being fine, the failure mode usually isn't a slow page load. It's a security gap you didn't know you had, a bill that doesn't make sense, or an on-call incident at 2am because your routing rules silently stopped working after a cluster upgrade.

The ingress controller is the front door to everything running in your cluster. It deserves more than five minutes of thought.

---

What an Ingress Controller Actually Does (and What It Doesn't)

Let's be precise about this, because the confusion here causes real problems downstream.

A Kubernetes Ingress resource is just a spec — a YAML object that says "route traffic for this hostname to this service on this port." By itself, it does nothing. An ingress controller is the software that watches those specs and actually implements them, usually by configuring a reverse proxy or load balancer running inside (or adjacent to) your cluster.

The Kubernetes project doesn't ship a default ingress controller. That's intentional. Different workloads have genuinely different requirements, and the ecosystem reflects that.

What the controller handles:

• TLS termination and certificate management
• Host-based and path-based routing
• Header manipulation and rewriting
• Rate limiting and connection throttling
• Authentication offloading (in some controllers)
• WebSocket and gRPC support

What it doesn't handle: east-west traffic between services, network policy enforcement, or service mesh functions. I see teams conflate ingress controllers with service meshes regularly. They're different tools solving different problems. Don't let a vendor's marketing blur that line.

---

The Four Controllers Worth Knowing

There are dozens of ingress controllers. Four of them cover the vast majority of real-world use cases.

ingress-nginx (the community one, not the NGINX Inc. one — yes, there are two) is the most widely deployed. It's backed by the Kubernetes community, it's well-documented, and it handles most standard HTTP/HTTPS routing without drama. The annotation-based configuration model is verbose but predictable. If you're running a general-purpose cluster and you don't have exotic requirements, this is a reasonable default.

Traefik is where I'd point teams that want automatic TLS via Let's Encrypt and a configuration model that feels less like writing XML in 2003. It discovers routes dynamically from Kubernetes resources and has a built-in dashboard that's actually useful for debugging. The tradeoff: the configuration can get complex fast when you start layering middlewares, and the documentation assumes you already know what you want.

HAProxy Ingress is the right answer when you need serious connection handling at high volume. HAProxy has been doing this for 20 years. The Kubernetes controller wraps that battle-tested core. If you're running something that handles tens of thousands of concurrent connections and you care deeply about connection queue behavior, HAProxy is worth the additional configuration overhead.

Contour with Envoy is the path forward if you're building toward a service mesh or you need HTTP/2 and gRPC support that actually works well. Envoy is the proxy underneath Istio, Linkerd (partially), and most of the serious service mesh implementations. Running Contour gives you Envoy's capabilities without the full service mesh operational complexity. The HTTPProxy CRD it introduces is more expressive than standard Ingress resources, though that means you're writing non-portable config.

A concrete example: a healthcare SaaS team running on a managed cluster here at IDACORE came in using ingress-nginx and hitting intermittent WebSocket disconnects on their real-time patient monitoring dashboard. The issue wasn't the controller itself — it was that the default proxy-read-timeout annotation wasn't set, so connections were dropping after 60 seconds. Five minutes of config change fixed it. But the point is: they didn't know to look there because they'd never thought carefully about what the controller was actually doing.

---

Where the Real Complexity Lives: TLS, Headers, and Auth

Latency gets all the attention. It's measurable, it shows up in dashboards, and it's easy to blame. But the ingress controller decisions that actually cost teams the most are in three other areas.

TLS termination is where you decide whether encrypted traffic gets decrypted at the ingress layer or passed through to the pod. Terminating at ingress is simpler and handles certificate management in one place. Passing through (TLS passthrough mode) means the pod handles its own certs, which is more complex but necessary if your compliance requirements mandate end-to-end encryption — which they do if you're handling PHI under HIPAA. Know which mode you're in. I've seen teams assume they had end-to-end encryption because they saw a padlock in the browser.

Header manipulation is where routing logic creeps in and becomes invisible. Stripping or adding headers at the ingress layer for authentication, routing hints, or rate limiting identifiers is powerful and also a great way to create debugging nightmares. Document every annotation that modifies headers. Seriously. Put it in the repo. Your future self will thank you.

Authentication offloading — using the ingress controller to handle OAuth2, JWT validation, or basic auth before traffic reaches your application — works well until it doesn't. ingress-nginx supports external authentication via the auth-url annotation, which proxies auth decisions to an external service. This is fine for simple cases. For anything complex, you're better off handling auth in your application or using a proper API gateway in front of the cluster.

# ingress-nginx external auth example
metadata:
  annotations:
    nginx.ingress.kubernetes.io/auth-url: "https://auth.internal/validate"
    nginx.ingress.kubernetes.io/auth-signin: "https://auth.internal/login"
    nginx.ingress.kubernetes.io/auth-response-headers: "X-User-ID,X-User-Email"

That works. But if auth.internal goes down, every request to every service behind that ingress returns 401. Plan accordingly.

---

The Operational Costs Nobody Mentions in the Docs

The controller you choose affects your cluster upgrade path, your monitoring setup, and how much time your team spends on ingress-related incidents.

Upgrade compatibility is a real concern. The Ingress API moved from extensions/v1beta1 to networking.k8s.io/v1 in Kubernetes 1.19 and the old version was removed in 1.22. Teams that hadn't updated their manifests hit this hard. Every controller has its own release cadence and its own compatibility matrix with Kubernetes versions. Before upgrading your cluster, check the controller's compatibility table. Not the blog post — the actual table in the docs.

Metrics and observability vary significantly between controllers. ingress-nginx exposes Prometheus metrics out of the box and there are solid Grafana dashboard templates for it. Traefik has a built-in dashboard and also exposes Prometheus metrics. If you're running a monitoring stack — and you should be — verify that your controller's metrics integrate cleanly before you're trying to debug a production incident.

Configuration drift is subtle but expensive. Ingress controllers configured heavily via annotations are hard to audit. You end up with routing logic scattered across dozens of Ingress resources, each with its own set of annotations that modify behavior in ways that aren't obvious from reading the resource. Consider whether a Gateway API implementation (the newer, more expressive successor to Ingress) makes sense for your cluster. It separates infrastructure concerns (handled by cluster operators) from routing concerns (handled by application teams) in a way that scales better.

One thing that's specific to running Kubernetes in an environment like ours in Weiser: sub-5ms latency to Boise means the round-trip cost of external auth calls, health checks, and other controller-adjacent network operations is genuinely low. When you're routing through an Oregon-based hyperscaler region, that same auth call adds 20-40ms per request. It sounds small. At scale, it's not.

---

Making the Decision

Here's how I'd approach this if I were starting fresh:

If your team is new to Kubernetes and you need something that works without extensive tuning, start with ingress-nginx. The community support is extensive and most of the problems you'll hit have documented solutions.

If you're building a platform that other teams deploy applications onto, look seriously at Contour or the Gateway API. The separation of concerns it provides makes life significantly better as the number of teams and services grows.

If you have high-throughput requirements or you're running something where connection handling characteristics matter — financial services, real-time data pipelines, anything with lots of concurrent long-lived connections — benchmark HAProxy Ingress against your actual workload before committing.

If you're in a regulated environment handling PHI or financial data and you're running in Idaho, the data residency question matters before the controller question. Where does your traffic terminate? Where does your TLS session end? If the answer is "somewhere in Oregon," you have a harder compliance conversation than if it's in a local data center where you can verify the physical path.

The ingress controller isn't glamorous infrastructure. It doesn't show up in architecture diagrams as often as it should. But it sits in the path of every external request your application handles, and the decisions you make there — about TLS, about authentication, about configuration management — compound over time. Get it wrong and you're debugging routing issues in production. Get it right and it disappears into the background, which is exactly where it belongs.

---

If you're running Kubernetes workloads in Idaho and you're tired of debugging ingress behavior across a 40ms round trip to an Oregon availability zone, we should talk. Our infrastructure in Weiser puts you under 5ms from Boise with full Idaho data residency — which matters if you're in healthcare or finance and your compliance team has opinions about where traffic terminates. Tell us what you're running and we'll show you what the architecture looks like here.