Rollout runs on a GKE cluster with a Cloud SQL database. We wanted the screen we check them on to live in Rollout, built the way a customer would build it. The only product code it needed was two general-purpose nodes.
The use case page has the screenshot and the queries.
The board
One app, Ops, with two panels and four tabs.
Database. Is Cloud SQL up, and its CPU, memory, disk and connections to the production database, each with a trend over the last hour, six hours or day.
Kubernetes. CPU and memory pressure on each node, container restarts, workload errors and Kubernetes warning events over the same range.
Every signal has a threshold. The header shows the worst one and names it. The
tabs below open the error log, the warning events, restarts by namespace, and
a Live tab that runs kubectl get pods, kubectl get events and
kubectl logs --tail 80 through the Rollout desktop client on one registered
MacBook, with that laptop's own kubeconfig.
What we threw away
The first version had a workflow that ran every 30 minutes, queried the same metrics and logs, derived healthy, degraded or failing, and wrote a record. We retired it the same day.
Cloud Monitoring already keeps these series, at a finer grain than a 30-minute record, and alerts belong in its alert policies. A copy in Rollout added a schedule to maintain and data to keep in sync, and bought nothing. So the app queries the source when it opens and stores nothing. Records in Rollout are for work with an owner and a next step. A CPU sample has neither.
Two new nodes
Rollout already had a Google Cloud service account connection. It had no node that read logs or metrics, so we added two to the Google Cloud integration, and they ship to every workspace: Query Cloud Logging and Query Cloud Monitoring.
A few choices in them:
- Every logging call carries a time window. Either a lookback or explicit
start and end times. The caller's filter is parenthesized before the window
and severity clauses are appended, so an
ORin the filter can't escape the window. - The node asks for the narrowest scope the connection allows.
logging.readbeforecloud-platform.read-onlybeforecloud-platform. If the connection allows none of them, the call is refused before a token is minted. - Big responses fail instead of truncating. Anything over 5 MiB is an error. A silently cut log page is worse than no page.
- Metrics go through PromQL. Cloud Monitoring exposes a
Prometheus-compatible API, so Cloud SQL and GKE system metrics work under
their PromQL names (
cloudsql_googleapis_com:database_cpu_utilization). A range query without a step gets one that yields about 240 points, and a step that would exceed Prometheus' 11,000-point limit is refused before the call.
Who could do what
The service account has Logs Viewer, Monitoring Viewer and Kubernetes Engine
Cluster Viewer on the project, declared in OpenTofu. The Rollout connection
narrows that again to logging.read and monitoring.read.
The app reaches Google Cloud and the laptop only through six granted tools. The project, the connection, the device and every kubectl command are fixed on those tools. The app fills a PromQL query, a log filter, a time range, a namespace and a pod name, each checked by the tool. Nothing it can call writes anything. The kubectl tools need a person's approval, and an approval lasts 24 hours.
How it got built
Claude Code wrote the two nodes in the Rollout codebase, with tests, through the same CI as everything else. Then it worked in the workspace over Rollout's MCP server: it configured the tools and wrote and published the app.
People did the parts an assistant shouldn't do alone. We approved the service account change, pasted its key into the connection form (the key is encrypted and never enters workflow data), activated the tools, and approved the kubectl runs and each app upload.
What broke on the way
The connection form blamed the wrong field. Scopes typed as short names
(logging.read) were rejected, and the error showed up as "is invalid" under
Name. Short names are now accepted and expanded, and a connection type can
now say which field a validation error belongs to.
The app's log tab failed on every call. Its tool fixes the logging lookback
to {{range}} and lets the app fill range. Tool dispatch resolved that
placeholder in the node's config but passed the raw preset in its inputs, and a
node reads inputs first. Graph runs had always resolved both. Tool calls now do
the same.
That second one was a platform bug that any tool using a template slot on an ordinary node would have hit. We found it because we used the product the way it tells customers to.
What it showed on day one
The warning events panel stayed amber, 43 in the hour we captured. Two ExternalSecrets fail to sync every few minutes because the Secret Manager entries they reference don't exist, a real misconfiguration. The rest were probe failures while deploys rolled Rollout's own pods. The database sat at 13% CPU and 66% memory.
kube-system logs a steady stream of error entries from cluster networking
components on a healthy cluster, so the workload errors panel leaves that
namespace out.
Limits
- Pod readiness and deployment rollout state come from the laptop's kubectl tab. The cluster doesn't export kube-state metrics to Cloud Monitoring.
- Billing isn't on the board yet. It needs Cloud Billing's export to BigQuery.
- The Live tab needs the laptop online with the desktop client running.
- The board is a screen, not a pager. Alerts stay in Cloud Monitoring.