Kubernetes has become the default platform for modern application delivery, but its cost story remains messy. Most teams use Kubecost to instrument cluster spend \u2014 and Kubecost does that job well \u2014 yet the moment you open the dashboard, you are staring at hundreds of rows of namespace, workload, and resource allocations with very little guidance on what actually matters.<\/p>\n
FinOps engineers end up doing the same exercise every month: reading the tables, eye-balling idle workloads, identifying over-provisioning, building a list of recommendations, and writing an executive summary. The work is repetitive, time-consuming, and the quality varies with the experience of the engineer doing it.<\/p>\n
What if a local AI model could do the analysis for you, in the customer’s own environment, without sending a single line of cluster data to an external provider? That question led to Steward<\/strong>.<\/p>\n A completed Steward cost-analysis report against a test EKS cluster, showing the at-a-glance view, executive summary, and severity-graded findings.<\/p>\n A typical monthly review involves steps that don’t scale: reading 200-row allocation tables, identifying idle workloads and PVC waste by visual inspection, cross-referencing efficiency ratios against benchmarks, building a prioritised fix list by hand, and writing an executive summary. For a single cluster this takes half a day; across an MSP portfolio the effort compounds fast.<\/p>\n SaaS tools like CAST AI, Spot.io, and Vantage automate cost analysis, but they require pushing cluster data \u2014 namespace names, workload identifiers, cost figures, sometimes auth tokens \u2014 into the vendor’s environment. For regulated industries (healthcare, finance, government), that is a hard stop. External LLM providers have the same problem: even anonymised data leaving the customer environment can derail an evaluation.<\/p>\n Steward is an open-source, local-first FinOps reporting tool for Kubernetes. It connects to a customer’s Kubecost installation, runs analysis with a locally hosted LLM, and produces executive-grade cost reports through a modern web UI.<\/p>\n The key design constraint: no cluster cost data, namespace names, workload identifiers, or auth tokens ever leave the customer’s infrastructure<\/strong>. The LLM runs on a local Ollama daemon, the vector store on local ChromaDB, the database on local SQLite or Postgres. That’s the entire network surface.<\/p>\n Six containerised services, all on the customer’s own Docker host:<\/p>\n When the user clicks Scan<\/strong>, the worker runs an eight-phase pipeline. Each phase commits to the database so the frontend’s polling sees live progress: mark RUNNING \u2192 concurrent Kubecost fetches (allocation, assets, savings) \u2192 build a structured digest with grounding fields \u2192 retrieve RAG context from ChromaDB \u2192 send system prompt + digest + RAG snippets to Ollama \u2192 post-LLM validation \u2192 enrich findings with dollar impacts \u2192 persist the report.<\/p>\n Anyone who has tried to use a 7B-parameter LLM for structured analysis has hit the same problem: the model hallucinates. It claims “no idle workloads” when the input has four. It calls a 15% efficiency score “healthy.” Steward solves this with the grounded-LLM pattern<\/strong> \u2014 a three-step contract between the preprocessor, the prompt, and a deterministic validator.<\/p>\n Step 1 \u2014 Grounding fields.<\/strong> The preprocessor computes a digest with bucketed thresholds the model must use verbatim:<\/p>\n “trivial” means <$50\/mo run-rate. “critical” means 7\u00d7+ over-provisioning. These words become the vocabulary the LLM must use in its prose.<\/p>\n Step 2 \u2014 Grounded system prompt.<\/strong> The prompt enforces a strict contract: use exact grade and scale names from the digest, never describe a critical-grade cluster as “healthy,” and forbid AI filler phrases like “leverage” or “synergy.” Each finding must include a digest_reference pointer so the worker can resolve concrete dollar impacts.<\/p>\n Step 3 \u2014 Deterministic validator.<\/strong> After the LLM returns, a Python validator checks for negation contradictions, healthy-downplay phrases, boilerplate recommendations, severity violations on trivial clusters, and missing dollar impacts. Violations trigger a single repair round. If the model still fails, violations are logged but the report persists \u2014 a flawed report with a warning beats no report.<\/p>\n Every report opens with cards summarising cluster scale, per-resource efficiency (CPU \/ memory \/ overall with grade-driven colouring), and signal counts. A stacked bar shows the namespace cost breakdown \u2014 the “read the room in five seconds” view.<\/p>\n Below that, the LLM-written executive summary uses the digest’s exact grade names. Findings are sorted by severity then dollar impact, each with a severity-coloured border, a specific workload identifier, a concrete recommendation, and the dollar impact attached.<\/p>\n The Cost Analysis report with efficiency grade, signal counts, and severity-graded findings naming specific deployments. Multi-environment dashboard with at-a-glance status and aggregated metrics. Reports page showing cost trend over time and sortable scan history.<\/p>\n Steward runs entirely from Docker Compose:<\/p>\n Then open http:\/\/localhost:3000, add a Kubecost environment, and click Scan. You’ll need Docker with Compose v2, 8 GB RAM allocated to Docker, ~10 GB free disk, and a reachable Kubecost installation on an AWS EKS cluster (any v2.x).<\/p>\n Kubernetes cost optimisation does not have to mean reading endless allocation tables, and AI-driven FinOps does not have to mean shipping cluster data to a third party. By combining a structured digest, a grounded system prompt, a deterministic validator, and a locally hosted LLM, Steward demonstrates that small open-source models can produce reliable, executive-grade FinOps reports \u2014 entirely on customer infrastructure.<\/p>\n For MSPs and platform teams, this approach standardises a typically artisanal process and opens conversations with regulated-industry clients who previously could not adopt SaaS FinOps tools. The full source is on GitHub<\/a>.<\/p>\n Don’t just read about local AI for FinOps \u2014 experience it.<\/strong> Explore how TO THE NEW helps teams optimise and modernise their cloud at tothenew.com<\/a>, or schedule a call with us today<\/a>.<\/p>\nThe problem with manual cost reviews<\/h2>\n
What we built<\/h2>\n
System architecture<\/h2>\n
+---------------------+ +------------------+ +-----------------+\r\n | Next.js frontend | -----> | FastAPI backend | -----> | Kubecost API |\r\n | (TS + shadcn\/ui) | | (async) | | (per env) |\r\n +---------------------+ +--------+----------+ +-----------------+\r\n |\r\n +------------------+----------------+\r\n v v v\r\n +--------------+ +--------------+ +--------------+\r\n | SQLite\/PG | | Ollama | | ChromaDB |\r\n | (history) | | (LLM) | | (RAG) |\r\n +--------------+ +--------------+ +--------------+<\/pre>\n
\n
How a scan works<\/h2>\n
The clever bit: the grounded-LLM pattern<\/h2>\n
{\r\n \"cluster_scale\": \"trivial\", \/\/ trivial | small | production\r\n \"efficiency_grade\": \"critical\", \/\/ healthy | mediocre | poor | critical\r\n \"analysis_hints\": {\r\n \"idle_workload_count\": 3,\r\n \"over_provisioned_count\": 0,\r\n \"efficiency_grade\": \"critical\",\r\n \"cluster_scale\": \"trivial\"\r\n },\r\n \"cluster_efficiency\": { \"cpu\": 0.037, \"memory\": 0.825, \"overall\": 0.126 }\r\n}<\/pre>\nWhat the output looks like<\/h2>\n
Manual workflow vs. Steward<\/h2>\n
\n\n
\n Without Steward<\/strong><\/th>\n With Steward<\/strong><\/th>\n<\/tr>\n<\/thead>\n\n \n Half a day per cluster review<\/td>\n ~2 minutes per scan<\/td>\n<\/tr>\n \n Quality varies by engineer experience<\/td>\n Consistent executive reports<\/td>\n<\/tr>\n \n Manual waste identification<\/td>\n Automatic idle \/ over-provisioned \/ PVC detection<\/td>\n<\/tr>\n \n Generic recommendations<\/td>\n Specific workload names and CPU\/memory targets<\/td>\n<\/tr>\n \n Cluster data exposed to SaaS vendors<\/td>\n All data stays on customer infrastructure<\/td>\n<\/tr>\n \n Compliance objections in regulated industries<\/td>\n Compatible by default for healthcare\/finance\/gov<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n Getting started<\/h2>\n
git clone https:\/\/github.com\/Deepanshu846\/Steward_Scanner.git\r\ncd Steward\r\ncp .env.example .env\r\n\r\n# Generate a SECRET_KEY:\r\npython3 -c \"from cryptography.fernet import Fernet; print(Fernet.generate_key().decode())\"\r\n# Paste output into .env as SECRET_KEY=...\r\n\r\ndocker compose up -d\r\ndocker compose exec ollama ollama pull qwen2.5:7b-instruct<\/pre>\n
Conclusion<\/h2>\n
Want to take Kubernetes cost optimisation to the next level?<\/h2>\n