Here's the math nobody walks you through. Take a 12-host cluster. 2 x 24-core CPUs per host = 48 cores per host = 576 cores. Under the old model you licensed 24 sockets and paid annual support. Under the new model you license 576 cores, every year, forever. Same tin. Same VMs. 24x the billable unit. That's why renewal quotes are commonly landing at 3x to 10x prior perpetual-plus-support cost. Industry reporting in 2026 puts the average increase around 150%, with outliers far above that. Now the part that actually hurts. Effective 10 April 2025 the minimum purchase became 72 cores per product, per order line. Up from 16. Your 12-host cluster: 576 real cores, 576 billed. No penalty. Your 3-host edge site with 2 x 8-core CPUs per host: 48 real cores, 72 billed. You just bought 24 cores of nothing. Run eight of those sites: 384 real cores, 576 billed. 192 phantom cores. You are now paying for as many cores at the edge as you are for the entire production datacentre. The 72-core floor doesn't hit the big cluster. It hits every ROBO site, every edge box, every two-node cluster in a factory cabinet. And it applies per product line, so a vSAN add-on hits the floor all over again. One more clock: miss your renewal anniversary date and you can eat roughly a 20% penalty on the first-year subscription price. What to do, starting now. 1. Run RVTools this week. Read-only vCenter service account, 20 minutes, free. You want vInfo, vCPU, vMemory, vDisk, vHost, vPartition, vSnapshot and vHealth. Do it before you reply to the quote, not after. 2. Count billed cores, not sockets. vHost gives you "# CPU" and "Cores per CPU". Multiply, sum per site, then apply the 72-core floor to every site that lands under it. That number is what you're actually negotiating against. 3. Find the dead weight. Sum "In Use MiB" for every VM where Powerstate = poweredOff. Pull snapshots older than 30 days from vSnapshot. Check vHealth for possible orphaned VMDKs. In most estates 15-25% of the VM count is doing nothing and a third of provisioned storage is air.

Kubecost will tell you that team-support/chat-api burned $4,100 last month. Ask it what your agent spent on tokens and it has nothing, because the money left the building. Two blind spots, and they fail differently. One: the vendor line item. LLM API spend arrives as a single opaque invoice row. No tags, no namespace, no cost allocation key. It never touches your CUR. Finance sees the total. Nobody sees who caused it. Two: GPU attribution that is technically correct and useless. Kubecost will happily split that $12/hr GPU node across namespaces by pod resource requests. But one vllm pod serving six product surfaces gets allocated as a lump, and requests are static regardless of whether a feature drove 2% or 60% of the tokens. You get a number with a decimal point that means nothing. The root cause is simple. The unit of consumption changed and nobody changed the meter. Cloud cost tooling meters resources over time. AI spend is metered in tokens per semantic operation. Those never reconcile, and no amount of tag hygiene will make them. The part that surprises people is where the money actually goes. Take a support agent: 12 model calls per ticket, a 3,500-token static prefix, 6,000 tokens of RAG on three of those calls, 900 tokens appended to the transcript per turn, 400 output tokens per call. Token accounting per run: 80,900 full-price input, 38,500 cached input, 4,800 output. Output is 3.9% of the token volume and 22% of the cost. But the single biggest chunk is the transcript re-read - 59,400 tokens per run that nobody ever wrote a line of code to spend. Agent loops are quadratic in transcript length. Every output token and every tool result gets re-read as input on every subsequent iteration. At 50k tickets/day that lands near $519k/month at illustrative frontier rates (check your provider's current pricing, these move constantly). Four changes that do not touch the product - cap the loop at 8 iterations, rerank retrieval to top-4, restructure the prompt so the cache prefix covers the stable head of the transcript, route the three mechanical calls to a mid-tier model - take it to about $139k. That is 73% off, and cost per resolved ticket goes from $0.56 to $0.15.

Nobody sets out to spend more watching the cluster than running it. It happens because telemetry cost scales with deploy frequency and pod churn, while infrastructure cost scales with load. Those are two different curves, and one crossed the other while you were shipping features. Here's a real shape of it. 40-node EKS cluster, mostly m6i.2xlarge on a savings plan, 8 TB of gp3, a couple of NAT gateways doing real work. Infrastructure lands around $12,400/month. The observability invoice that quarter was $17,900. And 62% of that was custom metrics - from three metric names, all carrying a pod_name label nobody had ever grouped by. The mechanic is multiplication, and it's worth doing the arithmetic once so you never forget it. http_requests_total{service, method, status_code, env} = 40 services x 5 methods x 12 status codes x 3 envs = 7,200 series. Completely fine. Now one engineer adds pod because they wanted to debug a noisy replica: x 250 pods = 1,800,000 series. A 250x multiplication from one label. And pod churns - every deploy retires the old names and mints new ones, so your active series stay at 1.8M while your billable series over a 30-day window climb into the tens of millions. Add request_id or an un-normalised URL path with an ID in it, and it stops being a multiplier and becomes unbounded: cardinality now grows with your traffic, forever. The rule I use: a label is safe if its value set is bounded by something YOU control - a config file, an enum, a deploy. It is unsafe if it's bounded by something your USERS control - traffic, IDs, URLs, error strings. Three commands that tell you where you stand. 1. Total active series: count({__name__=~".+"}) 2. Your worst metrics: topk(20, count by (__name__)({__name__=~".+"})) 3. The precomputed version that won't melt your Prometheus - hit /api/v1/status/tsdb and read seriesCountByMetricName and labelValueCountByLabelName. It's free, it's instant, and labelValueCountByLabelName will name your offender in about four seconds.

Most on-call runbooks are lies. They were written eighteen months ago by an engineer who has since left, they reference a load balancer that was replaced during the last migration, and the one command that actually mattered was never written down because the person who knew it just typed it from memory every time. We all know this. We still page people at 3am and point them at a wiki page nobody has opened since the last audit. I spent the last few weeks attacking this from a different angle. Instead of trying to keep a library of static runbooks fresh, I started generating them on demand from the alert payload itself. The alert already contains almost everything you need. A CloudWatch alarm tells you the metric, the namespace, the exact dimensions, the threshold, the evaluation window, the datapoints that breached, the account and the region. A PagerDuty incident carries the service, the escalation policy, the priority and the full trigger log entry. A Datadog monitor gives you the query, the scope and the tags. That is dense, structured, high signal context. It is exactly the kind of input a language model reasons over well. The pipeline is boring in the best way. Alert fires. Webhook or SNS topic hands the JSON to a small script. The script normalises the payload into a common shape, merges it with a static environment context file that describes your org, your clusters, your escalation tiers and your hard rules, and sends the whole thing to Claude with a strict output contract. Ninety seconds later the responder has a document with a summary, immediate actions, a ranked table of diagnostic commands, escalation criteria, three hypothesised root causes with remediation and rollback for each, and a pre-filled post-mortem template. The part that surprised me was the diagnostic ordering. I asked the model to sort commands by information gain per second rather than by category, and to state which hypothesis each command discriminates between. That single instruction turned a generic checklist into something that reads like a decision tree. For a checkout API 5xx alarm it opened with the target group health check and the deployment history, not with CPU graphs, because CPU is almost never the discriminator on a sudden error rate spike.