Below: three tools that turn "which model" into a routing decision instead of a hardcoded constant. One academic OSS router, one proxy/gateway that's become the de facto control plane, and one commercial marketplace that prices the decision for you.

RouteLLM

Summary. RouteLLM is a framework for serving and evaluating LLM routers — small models that decide whether a query should go to a strong (expensive) model or a weak (cheap) one.

• Maintained by LMSYS, the group behind Chatbot Arena.

• Ships pretrained routers plus a benchmark harness so you can tune the strong/weak threshold on your own traffic.

• Drop-in OpenAI-compatible server: point your SDK at the RouteLLM endpoint, get back a routed response.

• Exposes a single knob — the cost/quality threshold — that you can move based on observed quality on your prompts, not someone else's leaderboard.

Use case. Anywhere a single application sends a mix of hard and easy prompts to the same model.

• Imagine a support-ticket summarizer that currently hits GPT-4-class models for every message, including one-line "thanks, resolved" replies. With RouteLLM in front, you could route the trivial ones to a small open model and reserve the expensive call for genuinely ambiguous tickets.

• Imagine a code assistant where 70% of completions are boilerplate. With a tuned router you could send those to a 7B model and only escalate the architectural questions.

• The FinOps shape: cost per request becomes a distribution, not a flat rate, and you get a dial to shift the distribution without redeploying app code.

No invented savings numbers here — the actual ratio depends entirely on your prompt mix. Benchmark before you believe a vendor's headline figure.

LiteLLM

Summary. LiteLLM is a proxy and SDK that gives you one OpenAI-compatible interface in front of ~100 model providers (OpenAI, Anthropic, Bedrock, Vertex, Azure, local Ollama, and a long tail).

• Maintained by BerriAI.

• Two ways to run it: as a Python SDK inside your app, or as a standalone proxy server your services hit over HTTP.

• Built-in features that matter for cost control: per-key budgets, rate limits, fallback chains, spend tracking, and request/response logging to your own datastore.

• Routing rules support cost-aware fallbacks (e.g. "try cheaper model first, fall back to stronger one on failure") and load-balanced model groups.

Use case. The control plane for multi-model spend. Most teams don't need a clever router first — they need attribution and guardrails.

• Imagine three product teams sharing a single OpenAI org with no per-team budget. With LiteLLM you could mint a virtual key per team, set monthly spend caps, and get cost reports without waiting for the provider's invoice.

• Imagine a "use Claude for long context, GPT for tool calls" policy enforced by app-level if/else scattered across five services. With LiteLLM's model groups you could centralize that rule and change provider without a deploy.

• Imagine a provider outage. With fallback chains configured, traffic spills to the next model in the group instead of paging on-call.

If you only adopt one tool from this edition, this is the one with the broadest surface area. It doesn't make routing decisions for you — it makes routing decisions possible.

OpenRouter

Summary. OpenRouter is a commercial SaaS that aggregates hundreds of models behind one API and one bill, with public per-token pricing and live latency/throughput stats.

• Single API key, OpenAI-compatible endpoints, normalized model IDs across providers.

• Routes around provider outages and (optionally) chooses the cheapest available host for open-weights models served by multiple providers.

• Pricing is transparent on the model catalog page — useful if you want to see the cost delta between candidate models before you wire up A/B tests.

• Pay-as-you-go credits; no commit. Trade-off: you're adding a vendor in the path and a margin on top of underlying model cost.

Use case. Quick way to compare models on real traffic without onboarding to each provider separately.

• Imagine you want to test whether a Llama or Mistral variant is "good enough" for a classification job currently on GPT-4o. With OpenRouter you could swap the model ID in one config, run a shadow traffic split, and read cost-per-call straight off the dashboard.

• Imagine procurement says "no new vendor contracts this quarter." OpenRouter lets you reach a dozen model families under one existing commercial relationship — useful for evaluation, less useful once you're at volume and direct contracts beat the markup.

• No direct FinOps mapping for production scale: at high volume, going direct to the underlying provider is almost always cheaper. Treat OpenRouter as a comparison harness, not a forever-home.

The pattern worth stealing

These three tools occupy different layers of the same stack:

• RouteLLM = the decision (which model for this prompt).

• LiteLLM = the control plane (keys, budgets, fallbacks, logging).

• OpenRouter = the catalog (try many models without many contracts).

The mistake is reaching for the router first. Without per-team attribution and spend logs, you can't tell whether routing helped — you just see the next invoice and hope. Start with the control plane. Get cost per team, per feature, per endpoint. Then decide where intelligent routing earns its keep.

A few honest cautions before you go shopping:

• "Cheap model + retry" can be more expensive than "good model once" when retries cascade. Measure end-to-end cost per successful response, not per call.

• Quality regressions from routing are often invisible until users complain. Log inputs, outputs, and which model handled them, so you can audit later.

• Routers add latency. A 50ms classification step in front of a 400ms generation is fine; the same 50ms in front of a 60ms call is not.

• Any savings percentage a vendor quotes was measured on their benchmark. Yours will differ. Plan to A/B before you plan the win.

One decision for this week

Pick the layer you're missing. If you have no per-team or per-feature attribution on LLM spend, stand up LiteLLM as a proxy and mint scoped keys — that single change makes every later optimization measurable. If attribution is already solved, point a shadow stream at RouteLLM and find out what fraction of your traffic actually needs the expensive model.

asm (agent-skill-manager)

Summary. asm is a CLI and TUI for managing skills across every AI coding agent you run.

• One tool to install, search, audit, and organize skills for Claude Code, Codex, Cursor, Windsurf, and 10+ other agents

• Ships a catalog the project advertises as 2,800+ skills, browsable in a web UI

• TUI for interactive work, CLI for scripts and CI

• Replaces the per-tool skill directories that each agent ships with its own format

The pitch on the README is blunt: stop juggling skill directories. If you've ever copied a prompt template from Cursor into Claude and edited the frontmatter by hand, you know the tax.

Use case. Centralized agent configuration with an auditable inventory.

• Imagine you run a 40-engineer org where half the team uses Cursor and half uses Claude Code. Skills drift. Reviews drift. Costs drift, because nobody knows which prompt is firing which model.

• With asm you could pin a curated skill set per team, audit which skills are installed where, and treat the agent layer like any other managed dependency.

• For FinOps specifically: every skill is a potential token-spend pattern. A centralized manifest is the first step toward attributing agent spend to teams and projects, instead of letting it dissolve into "AI tools" on the invoice.

• The 2,800-skill catalog is a double-edged sword — discoverability is great, but governance is now your problem. A skills allowlist is something you'll want before week two.

cmux

Summary. cmux is a Ghostty-based macOS terminal built for AI coding agents and remote development.

• Vertical tabs, notification rings, integrated browser support

• Built on Ghostty, so you get the rendering performance of a modern terminal with workflow features layered on top

• Notification system: panes get a blue ring and tabs light up when an agent finishes a task in the background

• README is translated into 20+ languages, which says something about who's actually using it

• Top contributors include lawrencecchen, austinywang, and azooz2003-bit

The thesis is simple: agents run long, and you're not going to sit and watch the terminal. You need the terminal to tell you when something's done — or broken.

Use case. A terminal optimized for supervising long-running agent sessions in parallel.

• Imagine you've got Claude Code refactoring one service, an agent running migrations on a staging cluster, and a third tab tailing logs from a remote box. Without notifications you context-switch every 30 seconds. With cmux you could let panes run, get pinged when they need you, and stop the productivity bleed of compulsive tab-checking.

• FinOps angle: agent runtime is agent spend. Every minute a Claude session sits idle waiting for human approval is tokens you've already paid for and a context window you'll pay to rebuild. A notification-aware terminal is, weirdly, a cost-efficiency tool.

• Limits: macOS only, Ghostty-based. If your fleet is mostly Linux workstations, this isn't your tool.

codegraph

Summary. codegraph was shared in the channel as a code-structure tool relevant to the agent-tooling thread.

• No further detail was captured in the session material beyond the link

• Treat the link as a lead, not a recommendation — verify the README before adding it to your workflow

Use case. No direct FinOps mapping; this is general-purpose code-intelligence tooling worth a look if you're building agent context-injection pipelines.

• Imagine an agent that consistently burns tokens loading the wrong files because it doesn't understand the call graph. With a code-graph layer in the prompt context, you could route the agent to the 200 relevant lines instead of dumping 20,000.

• The token economics here are real: context windows are the most expensive thing about modern coding agents, and selective context is the cheapest optimization you'll make this quarter.

The pattern worth noticing

Three tools, three layers of the same stack:

• asm manages what the agent knows (skills, prompts, configuration)

• cmux manages how you supervise the agent (terminal, notifications, parallelism)

• codegraph manages what the agent sees (code structure, context)

If you're treating AI coding agents as a serious line item — and you should be, because the bill is already arriving — these are the three surfaces where you control spend. Skills determine which models fire and how often. Supervision determines how much idle time you pay for. Context determines token consumption per task.

None of this shows up in your cloud-cost dashboard. It shows up on your Anthropic, OpenAI, and Cursor invoices, usually under one undifferentiated line. The org that wins on agent FinOps over the next year is the org that has a manifest of skills, a measurable supervision workflow, and a context strategy — not the org with the most clever prompts.

Your move this week

Pick one. If you have more than three engineers using more than two agents, install asm and start with a single shared skill list. If you've ever lost an hour because you didn't notice an agent finished, try cmux on one machine before rolling it wider.

cmux is built by a small team — if it shaves real hours off your week, the GitHub Sponsors page is where you say thank-you in the currency maintainers actually care about.

CodeGraph

Summary. CodeGraph pre-indexes your repository into a semantic knowledge graph that Claude Code, Cursor, Codex, OpenCode, and Hermes Agent can query instead of grepping the filesystem token by token.

• Maintained by colbymchenry.

• Pitches ~35% cheaper runs and ~70% fewer tool calls per the project README.

• 100% local — the graph lives on your machine, no third-party index service.

• One-line installer for macOS, Linux, and Windows. No Node.js required.

Use case. Cut the per-prompt context tax on AI coding agents by replacing fan-out file reads with a single graph lookup.

Imagine a 400-file service repo where every "find the callers of X" question in Cursor triggers a dozen grep and read_file tool calls. With CodeGraph indexing the symbol graph ahead of time, the agent asks the graph once, gets a structured answer, and pays for the tokens of that answer instead of the tokens of twelve speculative reads. If your team is on Bedrock, Anthropic, or OpenAI pay-per-token plans, that delta lands directly on the AWS or vendor invoice you reconcile each month — not in some hand-wavy "developer productivity" bucket.

The honest FinOps test: turn on usage tracking for one sprint, install CodeGraph for half the team, and compare token spend per merged PR. If the README numbers hold even loosely, the savings should be obvious in the billing console.

lean-ctx

Summary. lean-ctx is a Rust binary that compresses file and shell output before it ever reaches Cursor, Claude Code, Copilot, Windsurf, Codex, or Gemini.

• Maintained by yvgude, with contributions from glemsom and frpboy.

• Ships as a single Rust binary — shell hook or MCP server, your pick.

• README claims 60–95% token reduction, up to 99% on cached reads.

• 59 tools, 10 read modes, 95+ compression patterns per the project description.

Use case. Sits between your terminal and your agent and strips the cruft — ANSI codes, repeated lines, low-signal stack frames — before tokens get billed.

Imagine an agent debugging a CloudFormation deploy. It tails the stack events, runs aws cloudformation describe-stack-events, then re-reads a 4,000-line Lambda log. Three calls, easily 50k tokens, half of it whitespace and ANSI escape sequences. With lean-ctx in the path, the agent sees a compressed view and the model bill drops accordingly. The Rust binary keeps the hook itself cheap — no Node process tax per shell invocation.

If you're running a small platform team where every engineer has Cursor or Claude Code wired to a corporate Anthropic key, this is the lowest-effort intervention on the list. Drop the binary in, point your shell hook at it, watch the daily token graph.

Graphify

Summary. Graphify turns any folder — app code, SQL schemas, R scripts, shell scripts, docs, papers, images, video — into a queryable knowledge graph your AI coding assistant can hit with a /graphify slash command.

• Maintained by safishamsi.

• Works with Claude Code, Codex, OpenCode, Cursor, Gemini CLI, and others.

• Unique angle: one graph spans app code + database schema + infrastructure — not just source.

• Mixed-media input set is broader than the other two tools in this post.

Use case. Give the agent one place to ask "what touches this table?" or "what infra backs this endpoint?" instead of letting it shotgun-read your Terraform, your migrations, and your handlers separately.

Imagine answering "if we drop column customer_status, what breaks?" The naive agent loop reads every .sql, every model file, every IaC module, then re-reads them when it loses focus. With a pre-built Graphify index, that's a graph traversal, not a tour of your repo. For cost-attribution work specifically — mapping a workload to its team, its Terraform module, and its CloudWatch namespace — having app + schema + infra in one graph is genuinely useful, because the relationships exist whether you've modeled them or not.

There is no published token-reduction number in the material, so don't promise leadership a percentage. Treat it as a structural bet: less re-reading, fewer tool calls, smaller invoice.

Pattern, not coincidence. All three tools attack the same FinOps problem from slightly different angles:

• lean-ctx compresses the bytes after the agent decides what to look at.

• CodeGraph changes what the agent decides to look at in the first place.

• Graphify widens "look at" to include schema and infra, not just code.

You can stack them. A graph-aware agent that also runs its shell output through a compressor is paying for the smallest possible surface area of tokens.

One decision this week. Pick the repo where AI-assisted spend is highest — usually the largest monorepo with the most active Cursor or Claude Code users — and run a one-week A/B with CodeGraph on half the team. Measure tool-call count and token spend per PR. If the numbers land anywhere near the README's claims and the tool saved your org real figures on the Anthropic or Bedrock invoice, sponsor yvgude for lean-ctx while you're at it; this category of work only continues if the maintainers can afford to keep shipping.

That question deserves a serious answer, because most FinOps tooling still treats cost as accounting, not as a system to simulate.

Pan et al., "Building energy simulation and its application for building performance optimization"

Summary. The 2023 review in Advances in Applied Energy is a peer-reviewed survey of building energy simulation as a discipline.

• Authored by Yiqun Pan, Mingya Zhu, Yan Lv, Yikun Yang, Yumin Liang, Ruxin Yin, Yiting Yang, Xiaoyu Jia, Xi Wang, Fei Zeng, Seng Huang, Danlin Hou, and Lei Xu

• Published as Advances in Applied Energy, Volume 10, June 2023, article 100135

• Covers methods, tools, and case studies for building performance simulation

• Scope spans design, retrofit, operations, and urban-scale energy planning

• Includes digital-twin approaches

Use case. The paper itself is a reference, not a deployable tool. But for FinOps practitioners, it reads like a parallel-universe playbook: the energy-engineering world has spent decades doing what cloud cost teams are only starting to attempt — simulating a stock of assets against time-varying demand, with retrofit options scored against a status quo. Imagine you are forecasting next year's spend across regions, instance families, and reserved-vs-spot mix. With a simulation-first frame borrowed from this literature, you would treat your fleet the way energy engineers treat a building stock: model the loads, model the supply curve, model the retrofits, then run the policy against "weather" — traffic patterns, seasonal demand, AI workload bursts.

Why energy is the binding input

Labor pays the engineers. Capital buys the hardware and the reservations. Data is the product. Energy is what makes the data move — and on GPU-heavy workloads, it is increasingly what dominates the marginal cost of an inference call or a training step.

Most cost dashboards still treat compute as the unit and ignore the watt. That was defensible when watts were buried inside the EC2 price and somebody else's problem. It is less defensible when accelerator pricing is explicitly energy-shaped and hyperscaler regions vary in carbon intensity by an order of magnitude. A FinOps practice that cannot reason about kilowatt-hours per workload is going to mis-forecast both the dollar and the carbon line.

This is the point the Slack thread was driving at. If you accept data as the fourth input, energy is what connects all four. Engineers consume energy. Capital expenditure is recouped over an energy-consuming life. Data is produced by burning watts. The economic geometry of cloud is starting to look much more like the economic geometry of a grid.

What grid-style simulation actually buys you

The Pan review is worth reading end-to-end, but the broad strokes that translate cleanly to cloud:

• Bottom-up load modeling. Building simulators model at the zone and equipment level, not the meter level. The cloud equivalent is modeling at the service, pod, or queue level — not the account level. Most FinOps reports still aggregate at the account.

• Weather-driven scenarios. Energy engineers run a typical meteorological year against a building. Cloud engineers should run a representative year of traffic and request mix against a fleet plan. The data exists in CloudWatch and the billing export. Almost nobody uses it that way.

• Retrofit vs new-build comparison. Energy modelers compare keeping a chiller against replacing it, with payback periods. FinOps should compare keeping a Reserved Instance footprint against committing to Savings Plans, against migrating to Graviton, against re-architecting to serverless — using the same payback discipline.

• Co-simulation. Buildings co-simulate HVAC, lighting, and occupancy because they interact. Cloud workloads should co-simulate compute, storage, egress, and queue depth — because those interact too, and pricing the parts in isolation gives you the wrong answer.

• Digital twins. A live model that updates against telemetry, instead of a quarterly spreadsheet. The cloud equivalent is the gap between a static forecast and a model wired to actual usage data.

What this changes about FinOps tooling

The honest answer: most FinOps tools today are reporting tools, not simulators. They tell you what you spent, what you will spend next month if nothing changes, and which resources are idle. They mostly do not let you ask "what if traffic doubles, 40% of inference moves to a smaller instance family at off-peak hours, and we move the batch job to a lower-carbon region."

A grid engineer would call that a missing capability, not a power feature. It is the basic question their tools answer.

There is an opening here for OSS. The energy world has open simulators — EnergyPlus and its lineage — built over decades of public investment. The cloud world has Kubecost, OpenCost, Cloud Carbon Footprint, and a long tail of vendor dashboards, but no widely adopted equivalent of a co-simulation engine. The thesis the Slack note implies is that someone is going to build it, and the methods will look a lot like the ones Pan et al. survey.

One decision this week

Read the Pan review — or at least the methods section — and then pick your single largest workload and write down its demand curve as if you were handing it to a simulation engineer. Hour of day on the x-axis, requests or GPU-seconds on the y-axis. Everything grid-style planning does starts from that one artifact, and almost no FinOps team has it written down.

Read more

Subscribe now

Read more