Why Docker Container Networking Gets Weird When Your Registry Is Far Away

If your container deployments feel sluggish — slow pulls, flaky image caches, timeout errors during CI/CD runs — the first thing most engineers check is their Dockerfile. Layer optimization, multi-stage builds, base image size. All reasonable places to look. But if your registry is sitting in an AWS region 600 miles away, you're fighting physics, not your Dockerfile.

This is a problem I see consistently with teams running workloads out of hyperscaler regions that weren't designed with the Pacific Northwest in mind. Oregon is the closest major AWS region to Idaho, and it's still 20-40ms away from Boise on a good day. That sounds small until you understand what Docker is actually doing when it pulls an image.

What Docker Pull Actually Does to Your Network

A docker pull isn't a single HTTP request. It's a sequence of API calls — manifest fetch, layer existence checks, individual layer downloads — and each one requires a round trip to the registry before the next one starts. The Docker registry API (OCI Distribution Spec) is inherently chatty.

Here's a simplified version of what happens when you run docker pull myapp:latest against a remote registry:

GET /v2/                          # auth challenge
POST /v2/auth/token               # token exchange
GET /v2/myapp/manifests/latest    # fetch manifest
GET /v2/myapp/blobs/<layer1-sha>  # download layer 1
GET /v2/myapp/blobs/<layer2-sha>  # download layer 2
...

Each of those requests carries round-trip latency. If you have 10 layers and 30ms RTT to your registry, you're looking at 300ms of pure latency overhead before a single byte of actual image data moves. Add in TLS handshake overhead, TCP slow start on each connection, and the fact that Docker's default behavior doesn't pipeline these requests aggressively, and you've got a real problem.

On a local registry with sub-5ms latency, that same sequence completes in under 50ms of overhead. That's not a minor difference. In a Kubernetes environment doing rolling deployments with 20 pods pulling simultaneously, it's the difference between a 2-minute deploy and a 10-minute one.

The Cache Miss Problem Nobody Talks About

Most teams know they should run a local registry mirror. Fewer understand why cache misses are so expensive in a high-latency environment.

When your node's local Docker cache is warm, none of this matters — the image is already there. The problem is cold starts: new nodes spinning up, CI runners that don't persist state, or any time you deploy a new image tag. In those cases, you're back to full pull latency.

Here's a real scenario. A Boise-based SaaS company was running their CI/CD pipeline on self-hosted runners in AWS us-west-2. Every job started with a docker pull of their build image — about 2.1GB uncompressed, 12 layers. Average pull time was 4 minutes. They moved their runners to infrastructure co-located closer to Boise, dropped registry RTT from 28ms to 4ms, and pull times fell to under 90 seconds. Same image. Same network bandwidth. Just less round-trip overhead across a chatty protocol.

The bandwidth wasn't the bottleneck. The latency was.

Running Your Own Registry Mirror Actually Works

The fix isn't complicated, but it does require owning some infrastructure. A pull-through cache registry (Harbor, or even a simple registry:2 container configured as a proxy) sitting close to your compute eliminates the latency problem for warm cache hits and reduces it significantly for cold ones.

Here's a minimal Docker registry mirror config that proxies to Docker Hub:

# config.yml for registry:2 as pull-through cache
version: 0.1
log:
  fields:
    service: registry
storage:
  cache:
    blobdescriptor: inmemory
  filesystem:
    rootdirectory: /var/lib/registry
http:
  addr: :5000
proxy:
  remoteurl: https://registry-1.docker.io

Then configure your Docker daemon to use it:

{
  "registry-mirrors": ["https://your-local-mirror:5000"]
}

For Kubernetes, you'd set this in the containerd config on each node:

[plugins."io.containerd.grpc.v1.cri".registry.mirrors]
  [plugins."io.containerd.grpc.v1.cri".registry.mirrors."docker.io"]
    endpoint = ["https://your-local-mirror:5000"]

This works well, but it has a catch: your mirror is now a critical piece of infrastructure. If it goes down, pulls fall back to the upstream registry — which is fine — but if it's misconfigured or has storage issues, you'll spend a frustrating afternoon debugging why images are serving stale layers. Run it on reliable infrastructure with persistent storage and monitor it like you would any other production service.

Private Registry Placement Is a Topology Decision

If you're running a private registry for your own images (not just a Docker Hub mirror), where you put it matters even more. Your build system pushes to it, your deploy targets pull from it, and in a GitOps workflow, that traffic is constant.

The right answer is to co-locate your registry with your compute. Not in the same data center necessarily, but on the same network segment with low-latency paths between them. If your Kubernetes nodes are in a Boise-area data center, your registry should be too — not in us-west-2 because that's where your AWS account lives.

This is a topology decision, not a storage decision. A lot of teams treat registry placement like an S3 bucket — just put it somewhere and point at it. That works fine for large files with long transfer times where latency is a rounding error. It doesn't work for a protocol that makes a dozen sequential round trips before it starts moving data.

One thing worth testing in your own environment: run docker pull with --debug flag and look at the timing on each layer fetch. You'll see exactly where the time is going. On a high-latency registry, you'll notice the gaps between layer downloads — that's your RTT showing up in the timeline.

dockerd --debug &
docker pull --disable-content-trust your-registry/your-image:tag 2>&1 | grep -E "pulling|downloaded|complete"

Compare that output against a local registry pull. The difference is usually obvious.

When Egress Fees Make the Problem Worse

There's a second problem hiding behind the latency issue, and it's financial. If your registry is in AWS and your compute is pulling from it, you're paying egress fees every time a node pulls an image. AWS charges $0.09/GB for data out of us-west-2. A 2GB image pull across 20 nodes during a deployment is 40GB of egress — $3.60 for a single deployment event. Do that a few times a day across multiple services and it adds up fast.

This isn't hypothetical. A development team running 50 deploys per day with a 1.5GB average image size was paying around $2,700/month in egress fees alone — just for image pulls. They weren't tracking it because it was buried in their AWS bill under "Data Transfer." Once they identified it and moved their registry to infrastructure without per-GB egress charges, that line item went to zero.

The latency problem and the cost problem have the same solution: put your registry close to your compute, on infrastructure that doesn't charge you to move your own data around.

The Broader Point About Infrastructure Proximity

Container networking weirdness — slow pulls, flaky deploys, timeout cascades during high-load deployments — is often a symptom of infrastructure that wasn't designed with proximity in mind. Hyperscaler regions are built for global reach, not for minimizing latency to a specific metro area. When your users and your compute are both in the Treasure Valley, routing through Oregon adds latency at every layer: registry pulls, inter-service calls, database connections, everything.

This is why infrastructure placement decisions matter beyond just cost. A 25ms RTT difference sounds trivial in isolation. Multiply it across every API call, every health check, every image pull in a microservices environment, and it shapes your entire system's behavior.

---

If you're running Docker workloads in or around Boise and dealing with slow pulls or unpredictable deploy times, it's worth looking at where your registry actually lives relative to your compute. IDACORE runs infrastructure out of a Weiser, Idaho data center with sub-5ms latency to the Boise metro — no egress fees, no surprise bills, and no ticket queue when something's actually broken. If you want to talk through your current registry topology and whether moving it makes sense, reach out and walk us through your setup.