· 9 min
Rules born from mistakes
Nearly two in three commits carry an AI co-author. The memory, plans, hooks and audits that kept an AI-built app honest, and the mistake behind each.
Part 2 of 7 in the Building ProbaBaby series.
ProbaBaby’s repository has 2,876 commits. Lovable’s bot wrote 867 of them, all in the first three months. 1,833 more carry a Co-Authored-By: Claude line: nearly two in three of all commits, and 98% of everything committed in 2026. The trailers even record the models changing underneath me, ten of them in ten months, from Opus 4.5 to Opus 5.5.
every commit
- 867 by Lovable’s bot, all in the first three months
- 1,833 with a Co-Authored-By: Claude line
- 176 with neither
committed in 2026
- 98% with a Claude co-author
each square is about 1% of the commits in its grid
The diagram, step by step:
- The repository holds 2,876 commits.
- Lovable’s bot wrote 867 of them, all in the first three months.
- 1,833 carry a Co-Authored-By: Claude line. Nearly two in three.
- The other 176 have neither: not Lovable’s bot, and no Claude line.
- Of everything committed in 2026, 98% carries a Claude co-author.
So this isn’t a story about using AI a bit. I wrote the app with an AI partner, Esma tested it every day, and users reported what we both missed, but no second engineer read the code before it shipped. This part is about what went wrong with that, and the system that grew around it: a memory of lessons, rules with reasons, plans that outlive a session, and scripts that refuse a commit. Almost every piece of it traces back to a dated mistake.
The victory laps
January 2026 was the month of confident commit messages. On 29 January alone: “Complete final Filipino translations - 100% COMPLETE”, “Improve Filipino translations - FINAL 16 strings (100% COMPLETE!)”, and a session summary that reads “Quality: A+ across all metrics … Ready for deployment and continued implementation.” In February: “Achieve 100% global coverage with 241 countries & territories”.
Seven months later the Filipino column turned out to be partly seeded from Indonesian, with 591 cells still identical to English. The Ukrainian column was the Russian one run through find-and-replace. And the support data for those 241 countries was partly generated rather than researched. Part 5 has more of these.
That was the first lesson: “done” is a claim, not a check, whoever writes it.
Write the why down with the rule
The memory directory is a folder of lessons that Claude Code works from; its index loads at the start of every session. The first note is from March 2026. There are 173 now: 77 about how to work, 92 about the project, and a few references. Most lessons have a Why: line and a How to apply: line.
The project’s CLAUDE.md arrived on 1 May 2026, alongside its first rule: commit and push at the end of every phase without asking, because I had typed “please push all changes to github” one time too many. Every rule in that file now ends with a paragraph titled “Why this exists”.
The reason matters more than the rule. A rule with no reason gets optimized away by the next session that finds it inconvenient. A rule that names its incident gets applied to cases nobody predicted. The note gets written on the day of the mistake, while the commit hash and the numbers are still at hand.
Plans are the handoff, not the chat
A session forgets. On 2 May I added a convention for long work: a plan file in docs/plans/ with an append-only Progress log, one entry per phase: Done, Changed, Remaining, Risks. There are about 125 plans now, with 663 Progress-log entries between them. Since the end of August, multi-phase plans also carry a paste-ready kick-off prompt for each phase, to start in a clean session.
That is what lets a fresh session pick up a thirteen-phase audit cold, across hundreds of sessions and several model generations. The chat is gone by morning; the file isn’t.
Agents find examples, not every instance
In August I redesigned the app’s icon containers. Five discovery agents inventoried every place that drew one and listed 653 sites. I opened the Baby Care tab and found untouched icons right away. A plain regular-expression scan then found 134 more candidates in 57 files, some in files the inventory had never mentioned. A second spelling of the same pattern found 31 more.
- five discovery agents
- 653
- then a regex scan
- +134candidates, in 57 files
- then a second spelling
- +31
The diagram, step by step:
- Five discovery agents inventory every place that draws an icon container: 653 sites.
- I open the Baby Care tab and find survivors. A plain regex scan finds 134 more candidates, in 57 files.
- A second spelling of the same pattern finds 31 more.
The lesson that came out of it says it plainly: “LLM search agents satisfice. They find enough instances to characterise the pattern and then move on.” The rule now: agents for judgement, a mechanical scan for coverage, take the union, and re-scan until every remaining hit has a documented reason to exist. An agent’s list is a floor, not a ceiling.
“The real root cause”, three times in one night
On the night of 10 May 2026 a sheet in the contraction timer kept flashing open and closing again. Over twelve hours it got nine commits. At 01:15 one announced “the real root cause”. At 01:35 another was “going nuclear with three combined defenses”. At 01:53: “the actual root cause”. At 10:18 the sheet was “ironclad”. It still closed. Part 3 is the whole saga.
Every one of those commits fixed something real. None was the cause. What finally broke it open was an observation, not a theory: the console showed the app still believed the sheet was open while it was visibly gone.
So now a fix has to prove itself. Translation repairs report deltas against HEAD (introduced=0, fixed=6), not absolute claims. The worst live-event bugs are written into a script with an expected state, RED while broken and GREEN once fixed, and the script fails whenever reality and the marker disagree. A fix has to be marked on purpose, and a regression can’t slip back in quietly. Large plans carry a “Done means…” section written before the work starts.
From a note to a gate
Prose rules don’t hold under pressure. The migration-drift check says so in its own docstring: the rule “is in CLAUDE.md but is easy to forget — this script enforces it.” So anything that keeps breaking moves up a ladder: a memory note, then a CLAUDE.md rule, then an audit script, then a check in the pre-commit hook.
- memory notewritten the day of the mistake, with its why; 173 now
- breaks againCLAUDE.md ruleeach ends with “Why this exists”
- againaudit scripteighteen audit and check scripts
- againpre-commit hooktwo checks on 1 May, 15 now
The diagram, step by step:
- A mistake becomes a memory note, the day it happens, with its reason.
- If it breaks again, it becomes a rule in CLAUDE.md, ending in “Why this exists”.
- Again, and an audit script looks for it.
- Again, and the pre-commit hook refuses the commit.
The hook started with two checks on 1 May and has 15 now, with eighteen audit and check scripts behind it. Each one has the shape of a past bug:
- the Ukrainian column that was Russian → a gate that catches the wrong script and a sibling language’s marker words;
- translated keys that never reached the screen → a routing gate;
- 23 countries that showed no crisis hotlines → a reference-data audit, and a block on data changes staged without the rebuilt database;
- Live Activity triggers whose authentication regressed at least three times → a check that runs whenever those files change;
- notifications that bypassed the app’s coordinator and were silently swept → a bypass gate.
The gates that inventory old debt baseline it, so only new violations fail. That is what made fifteen checks livable on every commit. Cloud CI exists, but its triggers are off to save minutes; the local hook is the real gate.
When one session became many
By August I was running several Claude sessions against one working tree, and audits that fanned out to dozens of agents. It was faster, and it produced one of the longest lessons in the memory.
There are seven logged incidents of one session’s commit sweeping up another session’s staged work; twice, a commit that didn’t compile reached origin. A git stash wiped a peer’s staging. A parallel session rewrote a string catalog from a stale snapshot and deleted 19 keys, and no gate noticed; I caught it from a screenshot. Four build workers filled a 926 GB disk with about 95 GB of build products, until translator agents couldn’t even run a shell.
The rules that came out of it: commit with an explicit path list, every time; never stash while a peer is active; give every build worker its own build folder and delete it afterwards; keep edits in the shared tree to one main loop, and give parallel writers their own worktree. Background agents read, review and translate. One line in that lesson sums it up: “Printing the guard is not a guard.”
The audits scaled the same way. A September audit of the Planning module ran 42 agents and returned 699 findings. Three were refuted, and about 45% of the “high” ones were downgraded on review. Findings are now verified on a ladder: a three-reviewer panel for critical ones, a single refuter for high ones, batches for the rest. A second tool helps too: Codex caught a nested sheet that a twenty-agent Claude audit had missed.
Confident, and wrong
Some mistakes were plain wrong facts, delivered with confidence:
- An audit flagged
#available(iOS 26.0, *)as a bug and suggested iOS 18, not knowing that Apple went from iOS 18 straight to iOS 26. - Two sessions switched the app and its widgets to iPhone-only to postpone iPad work, and one told me that was fine. App Store Connect rejected the next build: an update can never support fewer devices than the live version.
- An agent was told “commit + push, do NOT deploy” for the paid prediction engine. The GitHub integration deployed it to production about a minute after the push.
- One plan logged “no local Postgres or Docker on this machine”. Docker had been installed all along.
The fix is boring: hard facts live in the memory index and the global instructions, where every session reads them, and sessions check the real environment instead of reasoning about it.
The look of AI
Some rules are about taste. The icon in a pale tinted circle is the house style of AI-generated apps, and ProbaBaby had it on almost every screen. CLAUDE.md now calls it “the generic AI-generated look this app deliberately replaced”.
Replacing it took a design decision first: earned things, like badges, get a scalloped seal; functional things get a neutral glass chip with the colour in the glyph. I approved both on a design canvas before any code was written. The first migration still “finished” with survivors, because its scan knew only one of the four ways the tinted circle can be written in SwiftUI. That evening CLAUDE.md banned all four, and a script now keeps the inventory. Taste stayed my job. The tooling just made it stick.
What I still had to do
- Look at real screens. Most of the worst misses were caught by my eyes on a device, not by a gate: surviving icons, raw keys on a screenshot, a white-on-white tooltip in dark mode.
- Decide. Design direction, product calls and medical policy are mine. Plans stop and wait when a decision would otherwise be invented.
- Operate. Database pushes, deploys, App Store Connect, build numbers, the release checklist.
- Read Turkish. I’m a native speaker, and new Turkish marketing lines are flagged for my approval, though on the website I’ve approved some in bulk (Part 7). One of the few 2026 commits without an AI co-author is mine: “fix(prediction): Turkey is not in the Middle East”.
Five things I’d pass on
- Write the reason with the rule, on the day of the mistake.
- Promote anything that breaks twice from prose to a gate, and baseline today’s debt so the gate is livable.
- Treat an agent’s list as a floor and its “done” as a claim. Ask for the delta, the log line and the device.
- Hand off through files, not chat.
- Name the AI look, then ban it.
The bug that taught me most of this is next: the sheet that closed itself.