One tilt up: getting a 40-service AI inference stack onto a laptop

When I took over the platform domain, setting up a working dev environment took the better part of a day, and even then it wasn't the real system. The stack is 40+ services and jobs — vLLM and Triton for inference, Keycloak for identity, Istio for the mesh, Minio for object storage, plus the algorithm, data, and prediction services that sit on top, and the setup jobs that seed schemas, buckets, and model repositories. Locally, most people ran two or three services and mocked the rest.

That works until it doesn't. The bugs that hurt us were never inside a single service — they lived at the seams. A token audience mismatch between Keycloak and a service behind Istio. A presigned URL that worked against real S3 but not Minio. Sidecar injection changing how a health check resolved. None of those are visible in a mocked environment, so they surfaced in staging, days after the code was written, attached to someone else's deploy.

The bet

The bet was simple to state and annoying to execute: the local environment should be the production topology, not an approximation of it. Same services, same mesh, same identity provider, same object store — scaled down, not stubbed out.

We built it on k3d (k3s in Docker) with Tilt orchestrating the build-deploy-watch loop. tilt up brings up the full stack: the mesh, Keycloak with realm config, Minio with seeded buckets, both inference servers, every service in between, and the jobs that validate the tedious preparation steps people used to do by hand. Code changes hot-reload into the running cluster, and you get the whole system's logs and status in one dashboard. Need less? Subgroup targets stand up just one project's slice of the stack.

A few things mattered more than I expected:

• Real dependencies beat fakes, even bad ones. A slightly under-resourced local Keycloak still catches an audience mismatch that a mocked auth layer never will. If staging passes and prod fails, your dev environment lied to you.
• The hard 20% is identity and the mesh. Databases and object stores containerize trivially. Keycloak realm bootstrap and Istio's interaction with local networking were where most of the platform work actually went.
• Resource budgets are a feature. Getting 40+ services and jobs to coexist on a laptop forces you to know what every service actually needs — knowledge that transferred directly back to right-sizing production.
• The one-command constraint is load-bearing. Anything that requires a second command eventually rots. If tilt up is the only entry point, tilt up is the thing everyone keeps working.

What changed

Onboarding went from half a day of “set up your laptop” to one command. More importantly, integration bugs moved left: the seam failures that used to appear in staging now show up on the laptop of the person writing the code, minutes after they write it.

The same pattern now runs my homelab: a self-hosted Kubernetes cluster where cluster state is git history (Flux), identity is one SSO surface (Authelia), and deploys are commits. Once you've had an environment you can rebuild from scratch with one command, every environment that can't do that feels haunted.

If you're building one of these

Start from the production manifests, not from docker-compose. Scale replicas and resources down; do not swap components out. The moment you replace a real dependency with a lookalike, you've re-created the gap you were trying to close — you've just moved it somewhere you won't look until staging.