Passthrough a GPU to a VM and you've just turned one cluster node back into a pet. That's the part nobody puts in the "run your own LLM on Proxmox" tutorials. Getting the card into the VM takes an afternoon. Living with it is the real job. Here's what actually changes, and I checked each of these against current vendor docs. Passthrough (vSphere DirectPath I/O, Proxmox hostpci): • No live migration. VMware's docs say no vMotion, no suspend, no snapshots for a DirectPath VM. Proxmox's wiki says VMs with passed-through devices can't be migrated, full stop. • vSphere reserves the VM's full memory as soon as you add a PCI device. Your overcommit maths just broke. • Big-VRAM cards need 64-bit MMIO configured (pciPassthru.use64bitMMIO on vSphere, a bigger MMIO window on Proxmox/OVMF). Skip it and you'll be googling "DevicePowerOn failed" at 11pm. • The IOMMU group has to be clean. If your GPU shares a group with a SATA controller, the ACS override is a last resort, not a fix. • NUMA matters. Put the vCPUs on the socket the GPU is physically attached to. vGPU gives you some of the cluster back. vSphere 8 can vMotion vGPU VMs, with stun-time limits and some cluster settings to tune. Proxmox 8.4 added live migration for vGPU mediated devices. Here's what it costs you: • licensing and a license server you now have to keep alive • a supported-GPU list, and no GeForce cards on it • host driver and guest driver versions that have to stay compatible, so every upgrade becomes a choreographed one MIG only exists on specific datacenter cards (A100, A30, H100/H200, Blackwell datacenter parts and some RTX PRO Blackwell). Time-slicing works on more cards but gives you no memory or fault isolation between tenants. How I'd run it: 1. Treat the GPU VM as cattle that just happens to be pinned to one host. Run N+1 GPU hosts, not one. 2. Health-check the inference server and restart the process, not the VM. A wedged GPU sometimes only comes back with a host reboot. 3. Put GPU hosts in their own maintenance window, because DRS won't save you.

Your tagging is fine. Your showback is still going to start a fight. Showback rarely dies on tag coverage. It dies on the remainder: the shared Kubernetes cluster, NAT and egress, the support plan, observability, the platform team, commitment discounts, idle capacity. Somebody has to decide who pays for that, and the formula you pick decides the invoice more than the usage does. ILLUSTRATIVE example. ExampleCo, four teams (Checkout, Search, Data, Platform), one month, total bill 200,000. I ran it through a small Python script so every number below is computed, not vibes. Tag coverage: 92%. Looks great on a dashboard. Directly attributable to the team that consumes it: 60%. The gap is shared pools: the cluster, NAT, observability, support and tooling are all "tagged" to Platform or to nobody. Now allocate the 54,000 of general shared cost three ways. Same bill, three invoices: Checkout: even 58,000 / proportional 62,500 / driver-based 63,300 Data: even 74,000 / proportional 83,000 / driver-based 80,060 Search: even 41,500 / proportional 39,250 / driver-based 38,850 Platform: even 24,500 / proportional 13,250 / driver-based 15,790 Data's invoice moves by 9,000 depending only on a spreadsheet decision. Nobody changed a single resource. Now the part that kills trust. Checkout cuts its direct spend by 30%. The company saves 12,000.00. Under proportional allocation (the usual default), Checkout's invoice drops by 16,000.01, and Data (+2,500.00), Search (+1,250.01) and Platform (+250.00) all get a higher bill for doing nothing. Under driver-based allocation the others absorb +444.45 in total, only because support is still split by spend. Under even split, zero. Proportional is zero-sum and circular. Every saving one team makes is partly invoiced to its neighbours. Do that twice and nobody believes the numbers. Two more traps in the same example: Commitment discounts. The 18,000 commitment fee was bought from the Platform account. "Buyer pays" puts all 18,000 on Platform. Amortized by covered usage it is Checkout 4,500, Search 3,000, Data 10,500, and those teams can actually see their discount.

In July 2026 the EU published the "Digital Omnibus on AI". It pushes the AI Act's stand-alone (Annex III) high-risk obligations from 2 August 2026 to 2 December 2027. Some teams read that as "logging can wait". Wrong lesson. Most infra agents probably aren't Annex III high-risk anyway. But the first time an agent changes something in prod, your security reviewer, your ISO 27001 or SOC 2 auditor, or your own incident review will ask the same questions. Regulation or not. Here's the uncomfortable bit: logs are not an audit trail. Tokens, latency and a stack trace are observability. Audit is being able to prove, after the fact, who did what, as whom, on whose authority, and that nobody edited the record since. The questions you will actually get, per agent action: 1. Who asked for it, and who authorised it? 2. Which identity did it run as? Service account, not "the agent". 3. Which model, which version, which prompt and policy version? 4. What did it see? Inputs and retrieved docs, by reference or hash, not necessarily the content. 5. Which tool calls, with which arguments? 6. What actually changed, on which target? 7. Who approved the human-in-the-loop step, and when? 8. Can you rebuild the whole chain from one correlation ID: gateway, agent, MCP/tool, database? 9. How do you know the record wasn't edited or trimmed? 10. How long do you keep it, and does that clash with your GDPR retention? ILLUSTRATIVE example, invented org: ExampleCo's change-bot scales payments-api from 3 to 6 replicas. Their logs show a 200 from the Kubernetes API and a token count. They can't show that Bob approved it, which prompt version was loaded, or that the agent's own service account couldn't have deleted the log line. That's a vibe, not an audit trail. The traps I keep seeing: Logging full prompts and tool output "for audit", turning your log store into an unmanaged PII copy. Sampling. Fine for traces, fatal for audit. You don't get to sample which deletions you can prove. Logs writable by the same identity the agent runs as.

Your secrets manager doesn't bill you for the secret. It bills you for every time your code reads it. AWS Secrets Manager list price: $0.40 per secret per month, plus $0.05 per 10,000 API calls. Google Secret Manager: about $0.03 per 10,000 access operations after a small free tier. Azure Key Vault also bills per 10,000 operations (check the current number for your region, I didn't verify it). Prices are US list, checked 2026-10-13. The storage line is a rounding error. The read line is where it goes wrong. Illustrative example (invented service, not a real client): one API, one database password, 500 requests per second, and the code calls GetSecretValue on every request because "it's always fresh". 500 req/s x 2,592,000 seconds in a month = 1.296 billion reads. - AWS: 129,600 units of 10k x $0.05 = about $6,480/month for ONE secret - GCP: about $3,888/month - At 50 req/s: about $648 on AWS Now cache it in memory and refresh every 300 seconds, across 200 pods: 200 pods x 2,592,000 / 300 = 1.73 million reads = $8.64/month on AWS. With a one-hour TTL, $0.72. Same secret. Same security posture. Roughly 750x cheaper. And money is the mild failure mode. Secret stores have API rate limits. Per-request fetch means a traffic spike or a retry storm turns into throttled secret reads, and throttled secret reads mean your service can't authenticate to its own database. You built an outage generator out of your credential store. What to do: 1. Measure first. CloudTrail logs every GetSecretValue. Count calls per secret per principal per day (Athena SQL below). Most teams find 2-3 secrets generating most of the reads. 2. Cache in-process with a TTL plus jitter, so 200 pods don't refresh in the same second. 3. Refresh on auth failure, not just on TTL. Rotation is the reason people don't cache. If a call fails with a bad credential, force one refresh and retry once. That makes rotation safe. 4. Keep stale-on-error: if the secrets API is down or throttled, keep serving the last good value for a bounded time.

Every routing slide says the same thing: local model for the bulk, frontier API for the hard 5%. Nice slide. Here is what it is missing: nobody on that slide measured the 5%. If you cannot say how you know a request belongs to the hard 5%, you do not have a router. You have a hope with a YAML file. Three ways teams actually decide where a request goes, most reliable first: 1. Route by task class. Summarize a ticket, classify an alert, extract fields from an invoice: local. Plan a migration across 40 services: frontier. You decide per workflow, in advance, with an eval behind it. Boring, auditable, and it captures most of the savings. 2. Cascade with a verifier. Try local first, then check the result with something that is not the model's opinion of itself: the JSON schema validates, the Terraform plan parses, the unit test passes, the tool call returned 200. Escalate on failure. Great when output is checkable, weak when it is prose. 3. Learned router. A classifier predicts which model a prompt needs. Published research exists and the headline savings look great, but they were measured on someone else's traffic and someone else's model pair. Your router is stale the day either model changes. The math that decides whether to bother (ILLUSTRATIVE numbers, not anyone's price list, as of 2026-10-12): a frontier call averages $0.010 for you. Your local stack costs $3,000 a month fixed (hardware amortization, power, hosting, the person who keeps it alive) and almost nothing per request. - 20% escalation: break-even is about 380,000 requests a month. Below that, all-frontier is cheaper and you should stop reading routing posts. - At 1M requests a month: 5% escalation is about $3.6k vs $10k all-frontier. 30% is about $6.1k. At about 70% you built an expensive way to call the API. Two things the cost dashboard will never show you: The cascade tax. An escalated request pays for the local attempt AND the frontier call. The silent miss. A hard request wrongly kept local returns a plausible wrong answer. Nothing errors, nothing alerts. Only a graded sample catches it.