Archives
All the articles I've archived.
The Sweep Could Not Reach the Project. The Silence Was a Finding.
When monitoring client GCP projects, a broken permission that stops a check is as dangerous as a failing metric. Why our observability platform treats an inability to read data as a critical finding.
The Code Rolled Back. The Schema Could Not.
A rollback restores the code and leaves the database where the migration put it. Why "no rollbacks" depends on schema changes the previous release can still run against.
Scope Every Read: Confidential Docs in a Shared Bucket
Per-matter prefixes, short-lived signed URLs, database-checked reads, and redacted logs for confidential documents in multi-tenant GCP apps.
The Reply Arrived Twice. The Record Updated Once.
Inbound replies and webhooks must link to the right record exactly once, and generated drafts must wait for a human. Two mechanisms from production systems: key on the sender id, hold behind a gate.
The Code Changed. The Description Had to Survive It.
Code changes flow back into the knowledge graph the generator reads: changed files only, merge not replace, both artifacts patched. Freshness as write back, not retrieval ranking.
The OAuth Succeeded. The Reports Still Had to Reconcile.
Hardening QuickBooks OAuth server side and structuring Profit and Loss, Balance Sheet and Cash Flow in Postgres so three reports read at different times still agree.
The Test Passed. The Agent Never Called the Tool.
Grading an agent on the artifact it produced tells you nothing about how it got there, which is the one thing a tool-use environment exists to measure.
I Wrote the Tool for a Human. An Agent Called It.
An agent has only your response bytes to reason from, which turns error messages into control flow and makes response size a correctness property.
Stripe Holds the Subscription. Your Database Holds a Copy.
Subscription state, idempotent Stripe webhooks and tenant scoped billing tables drawn from multi-tenant platform and ledger work.
The Policy Was Enabled. One Role Still Read Across Tenants.
Auditing Supabase Row Level Security for a multi-role SaaS pilot, testing tenant isolation role by role with the service key outside the test path.
The App Balanced the Ledger. The Database Refused the Write.
Enforcing double-entry balance, integer minor units and append-only audit in Postgres with constraints that refuse the forbidden write.
I Could See Every Resource. I Still Could Not Rebuild It.
Taking over cloud infrastructure someone else built, and why a read-only inventory answers the easy question rather than the one that decides the handover.
The Device Went Quiet. Quiet Is Not a Reading.
A thousand devices reporting over MQTT taught me that absent telemetry is ambiguous, and that the ambiguity gets worse the moment the device is allowed to move.
The Browser Never Sees the Token. The Edge Function Holds It.
Read-only social API integration through Supabase Edge Functions in Deno and TypeScript, with tokens sealed server side and scoped to the minimum.
Nobody Typed the 10,000 Lines. The Review Belonged Upstream.
In a generated system the unit of review is not the output, it is the template that emitted it, because a defect there ships in every instance at once.
The App Called OpenAI Directly. The Gateway Should Have Been in Between.
Putting a Go gateway in front of a Python RAG service to route across providers, stream safely, enforce per-customer budgets and cache without hiding failures.
The Model Wrote the Code. The Build Gate Was the Product.
An AI code generator that must compile and run its own output before it delivers, and the exit conditions that keep the loop from hanging.
Our Worst Terraform Deploy Took 71 Minutes. The Fix Was One Flag.
Finding it meant ignoring the median, and controlling for the thing that made my first comparison lie.
Page 1 Was Fast. Page 400 Timed Out.
Offset pagination has to produce every row it discards, and the fix has three parts that only work together: the sort key, the cursor, and the index.
One Shared Service Account Would Have Been 200 Lines Shorter
A generated graph of 131 services can share one identity or carry 131 of them. The second costs 216 IAM bindings, and it is the only version where a leaked credential is survivable.
Every Deploy Reported Success. The Endpoint Had Been Unreachable for Two Weeks.
A 404 is not one thing, and the difference between text/plain, text/html and application/json was the entire diagnosis.
Two Environments, One Resource. The Label Said 'dev'. The Name Said Otherwise.
Adding multi-environment support to a platform that names its resources by service, and the collision that every successful deploy silently confirmed.
Delete the State File. Redeploy. Nothing Should Happen.
How generated Terraform learns to adopt infrastructure it did not create, and the three ways a cloud inventory lies about what things are called.
Terraform Created the Service. The Image Did Not Exist Yet.
Generated infrastructure creates services before their containers exist, and a moving tag makes rollback a guess. Two fixes: read the live image before you write it, and pin to a digest.
Building an Internal Developer Platform That Deploys Into Someone Else's Cloud
Multi-tenant infrastructure generation, and the three design decisions that turned out to matter most.
The Devices Were Fine. The Primary Key Was the Problem.
Bridging 1,000+ MQTT devices into Pub/Sub, Spanner and BigQuery, and why a monotonic primary key quietly caps your write throughput no matter how many nodes you add.
We Deleted Old Readings Every Night. That Was the Expensive Part.
A telemetry table that only grows forces three decisions: how you remove data, how many resolutions you keep, and what happens when a device reconnects and replays six hours of buffer.
The Webhook Must Reply in Seconds. The Useful Work Takes Longer.
Designing a Go alerting service where every operation worth doing is slower than the budget you have to respond in, and the sender retries if you are late.
Why our Gitea went down every Sunday at midnight
A short post-mortem on recurring midnight outages on our internal PaaS, and why single-replica stateful services plus a maintenance window is a bad combination.
Welcome, how this blog is built and served
Why this site is a static Astro blog on GCS behind Cloud Run: near-zero cost, scale-to-zero, and deploys that are just git push.
Hello, world, a blog that costs almost nothing to run
Why this blog is a static Astro site served from a GCS bucket through Cloud Run, with zero footprint on any cluster and a scale-to-zero bill.