SecretoCho
menu

· 14 min

Research before anything

73,756 lines of research, audits and spikes: how scouts, verifiers and skeptics checked it, what it changed, and where a ruling overruled it.

Part 2 of 7 in the Building Novi Kvadrat series.

Before Novi Kvadrat’s professional portal had a single screen, it had a folder called research/. By October, that folder and its successors held 73,756 lines of research, audits and spikes. To me this research step is really important, so this is the long part: what the research was, how it was checked, what it changed, and where a ruling went the other way.

Why research came first

I can design screens. What I couldn’t tell you was how Serbian agencies pay the portals, which official registers are open to reuse, or whether any model could read a scanned floor plan with Cyrillic labels. Each of those became a study before it became a screen.

By September it was how I asked for things. For a site-plan editor, I wrote: “We have very good floor plan — the studio. Same approach can we have site plan editor. Please first make a research about it.” (My own prompts are quoted with the spelling fixed.) The brief for the 3D work said it more firmly: “Prove the risky techniques with a throwaway spike in scratch/, write the research, and publish a Claude Design canvas. No app code.”

One April afternoon

It starts on 23 April 2026. At 13:40, local time, one commit added nine files: “Add research notes on competitor platforms and current app audits”.

Six were competitor studies: Zillow, Idealista, Rightmove, the Serbian portals, Compass, and ImmoScout24 with Realtor.com. They ran to 1,677 lines and 253 web links, with their own hedge: “Items we could not directly verify are marked (needs verification)”. Two audited what we already had, the public site and the admin. The ninth was a plan.

The most useful thing in them was what not to do. Zillow’s study listed “Non-exclusive leads sold to multiple agents simultaneously — the loudest agent complaint in every review we read”. Rightmove’s noted “No open API. Forces a scraper black market”, and Idealista’s said “Plan a JSON ingest endpoint from day one”. The Serbian study said “Publish all pricing on the public site”, and asked for an “English-first toggle, not ‘English added.’” The plan answered in its summary with lines such as “No pay-to-play lead resale — leads are exclusive to the listing owner”, and with a table, “Things we WILL NOT do (anti-patterns)”, each row with its reason; its first row is “Selling the same lead to multiple agents simultaneously”.

Then the plan sized itself: “Total: ~19 weeks of active engineering”.

Phase 0 landed at 14:29: “Phase 0 multi-tenant foundation, auth overhaul, and portal scaffolding”. Phase 8 landed at 17:23. That is eight phase commits in 2 hours 54 minutes, with 32,789 inserted lines. Commit time is not work time: some of it may have been prepared earlier, and the scope wasn’t the plan’s. Phase 5 was meant to bring Stripe. Its commit subject says “billing, credits, boosts, pricing page”; Stripe is in that code only behind a switch, and the switch stayed off.

The next day a “5-agent deep audit” read what had landed. At 13:49 came “Harden portal write paths across 5 audit findings”, and at 14:07 “Close remaining audit items: dispatcher cron, bulk-import upsert, invite fallback, + 8 polish fixes”. The backlog the audit left behind opens with “The correctness / security findings from that audit are all shipped”.

That afternoon and the day after hold the whole method in small: research, a plan, a fast build, and an audit that finds what the speed missed.

How much, and what kind

I had the corpus counted for this series: research, audits, analyses and spikes, wherever they live, including documents deleted later and a folder outside the repository. That comes to 141 documents, or 470 if each of the public-data survey’s 330 source cards counts on its own. They hold 73,756 lines and cite 4,467 unique web sources.

That last number counts only http links, so it undercounts: the code audits, the second-largest kind, cite path:line instead. Sorting documents into kinds is a judgment call too, and mixed ones went to their main kind.

  • Technical desk research, 14,832 lines: how to stack building floors on MapLibre, say, or which icon set has filled variants.
  • Code and security audits, 11,527: reading our own code before changing it.
  • Data-source surveys, 10,227: data sources, checked live.
  • Competitor studies, 8,796.
  • Market and legal research, 7,697: rentals, ratings, payments, and privacy impact assessments.
  • Synthesis, 6,339: the documents that turn several tracks into decisions.
  • UX research, 5,765.
  • Product audits, 5,040: our own screens, a few of them measured at phone widths.
  • Papers, 1,888.
  • Spikes, 1,454: throwaway code that measures one thing.
  • One risk register, 191: what three adversarial reviews of the floor-plan plan found and no phase absorbed.
technical desk research
14,832
code and security audits
11,527
data-source surveys
10,227
competitor studies
8,796
market and legal research
7,697
synthesis
6,339
UX research
5,765
product audits
5,040
papers
1,888
spikes
1,454
one risk register
191

The diagram, step by step:

  1. Technical desk research is the largest kind, at 14,832 lines: how to stack building floors on MapLibre, say.
  2. Code and security audits come second, at 11,527. They cite path:line, not web links.
  3. Data-source surveys, checked live: 10,227 lines. Competitor studies: 8,796.
  4. Market and legal research: 7,697. Synthesis, the documents that turn several tracks into decisions: 6,339.
  5. UX research: 5,765. Product audits of our own screens: 5,040.
  6. Papers: 1,888. Spikes, throwaway code that measures one thing: 1,454.
  7. One risk register: 191 lines. In all, 73,756.
spikes are one of the smallest kinds, and one afternoon’s spike shaped the phone design.

By month of first commit, April holds 10,898 lines and May 17,526, mostly a task-boards study and a blog study. June has none, July 2,418 and August 1,997. September holds 15,912 and October 21,532, as big programs began to start with their own research folders. Another 3,473 lines sit outside git.

April
10,898
May
17,526
June
none
July
2,418
August
1,997
September
15,912
October
21,532
outside git
3,473

The diagram, step by step:

  1. By month of first commit, April holds 10,898 lines, starting on 23 April.
  2. May holds 17,526, mostly a task-boards study and a blog study.
  3. June has none. July holds 2,418 and August 1,997.
  4. September holds 15,912 and October 21,532, as big programs began to start with their own research folders.
  5. Another 3,473 lines sit outside git. In all, 73,756.
by September, starting with research was how the big programs began.

For scale, the plans run to 168,076 lines. Research is about a quarter (24%) of the Markdown in the docs and research folders. Against the whole corpus, deleted documents and the outside folder included, plans outweigh it by about 2.3 to 1.

Scouts, verifiers and fact-checkers

The survey I’d show another builder first covered Serbian public data, on 5 October. Its question: every usable Serbian public API and dataset, verified live. It began by dumping the national open-data catalog through its API (3,510 datasets, 7,108 resources), then went looking for the rest. Each stage had its own job:

  1. Scouts. Eight of them, one per domain, found 403 candidate rows. Deduped, those were 311 sources; 19 more turned up while verifying, so 330 in all.
  2. Verifiers. 29 verifier batches wrote one card per source, each “after reading the terms and making a live GET probe”. 317 sources were verified live. 13 weren’t reachable that day and were marked UNVERIFIED.
  3. Fact-checkers. Three of them “re-probed all 113 key cards plus 75 others and refuted 13 claims (2 verdict changes)”.
  4. A completeness critic. It found 9 gaps. One more round ran on those, and then the loop stopped.
  1. eight scoutsone per domain: 403 candidate rows
  2. 311 after dedupe29 verifier batchesa card per source, after a live probe
  3. 330 in all317 verified live13 unreachablemarked UNVERIFIED
  4. three fact-checkersre-probe 188 cards, refute 13 claims
  5. a completeness criticfinds 9 gaps; one more round

The diagram, step by step:

  1. Eight scouts, one per domain, find 403 candidate rows.
  2. Deduped, those are 311 sources. 29 verifier batches write a card per source, each after a live GET probe.
  3. 19 more sources turn up while verifying, so 330 in all. 317 are verified live; 13 aren’t reachable that day and are marked UNVERIFIED.
  4. Three fact-checkers re-probe all 113 key cards plus 75 others, and refute 13 claims.
  5. A completeness critic finds 9 gaps. One more round runs on those, and then the loop stops.
scouts find and fact-checkers refute; without the check, 13 claims would have read as facts.

The rules were written down as well: “GET only, ≤ 1 request/s per host, no credentials, no CAPTCHA or bot-wall bypass, nobody contacted, no terms accepted”. Bot walls were “recorded, not bypassed”.

The headline: “the open, commercially reusable core is bigger than expected”, from the RGZ address register (2.49 M house numbers, nightly) to official GTFS timetables for Belgrade, Niš and Kragujevac. And: “The legally decisive registers are the closed ones.” Building permits sit behind a reCAPTCHA, and the brokers’ register has no reuse terms.

Every card also says what the run couldn’t prove, in an “Unverified” row, and the README adds: “Treat those items as open questions, not facts.”

It also hit the usual wall. One verifier batch “failed on the weekly limit (no cards)”, and the log notes: “The owner’s weekly limit was at 98% at 17:30 on 2026-10-05, so a new session (another account) may pick this up.” When a limit lands, the work moves to another of my five accounts.

When the fact-checkers ran out

The largest run was in July, for the agent portal: “a 105-agent deep-research pass … and a 200-agent dossier pass”. The first pass triple-checked 25 claims: 24 were confirmed and 1 refuted. The second set out to check 90 numeric claims, and then “a session usage limit killed the fact-checkers mid-wave”. Only 25 of the 90 got votes.

The useful part is what the document did next. It told the reader that, where nobody re-checked, a “Confirmed” tag “means the researcher read it on the cited source … not that an independent fact-checker re-confirmed it”. The two passes also disagreed about one Rightmove revenue figure. The report didn’t pick a side. It marked the figure “contested — do not repeat it externally”.

Skeptics, a spike and papers

The 3D work in September ran the most careful pipeline. Eight read-only scouts first described the 3D map as it really was, and an adversarial pass re-checked every issue: 88 held up and 16 claims were refuted. The research brief that followed carried the rule “Never re-report a refuted claim.”

Then the research ran as a chain: a researcher per track, a skeptic, and a completeness critic. The brief described the skeptic’s job: “For each track, a fable skeptic tries to refute its 5–10 load-bearing claims (pipeline, not a barrier). Refuted claims are fixed or removed.” The skeptics ran on a different model from the researchers (“Builders are opus, reviewers are fable”). The brief’s acceptance line was “no claim lacks a source”, and vendor marketing numbers had to be marked “vendor claim”.

Some claims can’t be refuted by reading. For those there was a spike: throwaway code in scratch/, “not app code”. On the afternoon of 24 September, five experiments produced 81 runner rows on one laptop. In the pick test, over 7,743 screen points and 9 cameras, MapLibre’s own hit test agreed with a colour-coded GPU render on 99.65% of them, with 27 refusals and 0 wrong floors, on illustrative data.

The number that changed the product was smaller. At the overview camera a single floor is “5.7 px” tall, and it reaches 44 px only at about zoom 19.9. So on phones, a tap picks the building and a floor strip, not the map, picks the floor. The report is plain about its limits: “No row here predicts a phone.” A verifier ran 16 checks against the raw rows: 13 held, 3 were weakened, none was refuted. Part 5 has the rest.

Floor plans, in August, were decided by reading: a teardown of CityExpert’s own plans, vendor terms, and the papers. The ask was to “Replicate (and ideally beat) the CityExpert listing-detail floor-plan experience”, and the options on the table included vendors and models that read a plan image. The research read the latest papers:

  • The state of the art is weak on geometry. Raster2Seq, on real plans from CubiCasa5K, scored Room F1 88.7, Corner F1 59.4 and Angle F1 37.4 in the research’s table.
  • Nothing reads our labels. “No floor-plan model in this literature was trained on Cyrillic room labels.”
  • The one big open dataset can’t be used. “CubiCasa5K is CC BY-NC 4.0 — commercially unusable”.

The verdict: “To match CityExpert you need zero AI. You need an internal polygon-tracing tool and an operator.” The first floor-plan phase was committed that evening, and the floor plans launched on 13 August, eight days after the research.

Across the folders, evidence labels grew into a vocabulary: “(needs verification)”, “vendor claim”, “[unverified]”, “[inferred]”, “[P] = perishable” for prices and traffic that change, V and I for verified in the code or inferred, and a strong / some / weak grade. They look fussy. They are what lets a later reader, or a later session, tell a measurement from a guess.

Research that changed the plan

Two changes are above already: floor plans became a tracing editor, and phones got a floor strip. Four more:

  • Footprints, checked after the fact. On 8 May, Microsoft’s building footprints went into the map at 14:39 and came out at 15:53, 74 minutes later: “the data proved unsuitable (footprint-only, no heights, AI-generated noise)”. In October the public-data survey re-probed one tile: “11,845 buildings, 0 with height”.
  • Rent and sale. A “14-agent verified codebase audit”, which also read the live database, found in July: “We are not starting from zero — we are starting from ~60% built, but the built pieces don’t talk to each other.” A build became “a connection-and-decision job for rent, and a data-model job for sale”. The rent marketplace launched on 28 July.
  • Analytics. On 14 October, measuring our own live data showed that only 3% of the server-side “visitors” over the past week ran JavaScript, and the audit that followed put “about 90%” of “Visitors” down to crawlers. Customers now see verified numbers only, with a notice, because “showing both keeps selling crawler hits as demand”. I approved it that day: “yes these are good. please implement it”. Part 6 has the layers under that number.
  • Viewers. The listing research had assumed a “~200 KB gz viewer” for 3D walk-throughs. A laptop spike measured the two candidates at “4.6× over” and “3.3× over”, and “No library entered novi-kvadrat-web/package.json”.

The order of the work changed too. The site-plan studio was researched first, on 23 September. On 24 September I put the 3D studio ahead of it, and on 25 September I accepted all 45 of the 3D research’s decisions. The site-plan research wasn’t deleted but annotated in place, with notes dated “Superseded (2026-09-25)”.

beforeafter
floor plansvendors and models that read a plan image, on the tablea polygon-tracing tool and an operator
footprintsMicrosoft’s building footprints in the map, 8 Mayout 74 minutes later; in October, 0 of 11,845 with height
3D on phonesthe map picks the floora tap picks the building and a floor strip picks the floor; at the overview camera a floor is 5.7 px tall
rent and salea build~60% built: a connection-and-decision job for rent, a data-model job for sale
analytics“Visitors”, about 90% of them crawlerscustomers see verified numbers only, with a notice
walk-through viewersa ~200 KB gz viewer, assumedmeasured at 4.6× and 3.3× over; no library entered the web app
pricesthe research: publish all pricing“Price on request”, a ruling against the research

The diagram, step by step:

  1. Floor plans: the options included vendors and models that read a plan image. The verdict: a polygon-tracing tool and an operator.
  2. Footprints: Microsoft’s went into the map on 8 May and came out 74 minutes later. In October, one tile held 11,845 buildings, 0 with height.
  3. 3D on phones: at the overview camera a floor is 5.7 px tall, so a tap picks the building and a floor strip picks the floor.
  4. Rent and sale: ~60% was already built. A build became a connection-and-decision job for rent and a data-model job for sale.
  5. Analytics: about 90% of “Visitors” were crawlers. Customers now see verified numbers only, with a notice.
  6. Walk-through viewers: assumed at ~200 KB gz, measured at 4.6× and 3.3× over. No library entered the web app.
  7. Prices: the research said publish them all. The ruling went the other way: “Price on request”.
six changes came from the research; on prices, a ruling went the other way.

When the ruling went the other way

Research gives options, and rulings decide. Twice the ruling went against the research, and once a finding the research had killed came back on its own.

Prices. The April studies said it again and again: “Radical pricing transparency — public RSD price list, no sales call to discover tiers”, and “Publish all pricing on the public site”. On 5 October we chose the other way: we wanted no public price list yet, and I made the change in one commit: “Pricing: hide plan prices on the public side and in the portals”. Visitors now see “Price on request”. The code paths stayed, with a note: “to show prices again, flip the one constant; don’t remove the code paths.”

PostHog. The research for the admin’s Pages area recommended “PostHog Cloud EU (Frankfurt), loaded only after consent”. On 9 October I ruled: “We need more clean path, so we should not pay extra at the moment.” The plan wrote it down as “No new vendor and no new paid service. That rules out PostHog …”.

A number that came back. On 5 August, the floor-plan research found that “floor plans → 52% more click-throughs” has “no traceable primary source”. It said the citation should be replaced with Rightmove’s verified “12% more leads”, and added: “Do not put any of these in an internal deck.” Nearly six weeks later, new research for the listing hero carried the 52% again, as “CubiCasa, vendor-published”, and it reached a design board. Nothing in the record says how it came back; a code comment from July, older than the research, still carries it. The string that shipped carries no number: “The single attachment buyers ask for most.”

The rhythm

In July, an audit led to code within hours and to a finished program within days. The portal audit, a “17-agent workflow”, was committed at 14:35 on 30 July. It concluded: “The portal is deep but unfinished, and structurally not mobile-ready”. The first fix phase landed 69 minutes later, the seventh on 4 August.

By October, audit to ship was often the same day. In the SEO and GEO audit, “Four read-only audits ran in parallel”, and a live check made “59 polite GET/HEAD requests with real and spoofed crawler user agents”. It found “AI training crawlers are blocked at the Cloudflare edge, while robots.txt says Allow: /”. The audit was committed at 11:19 and the fixes shipped at 14:48. The analytics audit was committed at 17:05 and its program merged at 22:43. The path editor’s four research lanes were written on 16 October; its plan was committed that evening and the studio was deployed the next morning, before its migration. The slower ones were the two studios, whose research ended in a design canvas and a multi-week plan.

The estimates in these documents, like April’s “~19 weeks”, were written in a different unit:

  • Floor plans said “~100 engineer-days”, with “FP0+FP1 = 24 days” as “the minimum shippable slice”. They launched about 8 calendar days after the first phase.
  • The floor-plan studio redesign said “≈49 d”. Its phases were in git by 13 September, the day after the plan.
  • The Account Manager plan sized itself in “calendar days for one main session plus agents”, at “~28–30 days”. Its nine phases span about 25.5 hours of commits.

Read those pairs with care. A commit says when work landed, not when it started, how many sessions ran beside it, or what was left for later, and several of these commits carry other programs’ work too. Where an estimate’s finish line was a data step or someone outside the build, the commits stop short of it. What the pairs do show is the unit: most estimates were in person-days, and the work ran as agent sessions measured in hours.

Research dies with the session

A research session writes into a scratchpad, and the scratchpad is temporary. The public-data survey’s log says so: “session-local; lost after the session; the cards in sources/ are the durable record”. Three habits kept the research alive:

  • Write as you go. The survey wrote one card per source as it went; when batch 23 “failed on the weekly limit (no cards)”, every other batch’s cards were already on disk.
  • Keep unfinished work outside the repository. A folder called nk-durable sits “outside the repo, so a sibling git add -A never sweeps it”. The path editor’s four research lanes live only there.
  • Hand builders facts, not chat. The analytics program gave its builders 2,659 lines of facts/ files. The database one opens with “Surprises (read first)”.

Saved research still ages. The floor-plan study’s reading of how CityExpert draws its plans was later marked “WRONG”. The public-data survey found older research that no longer matched reality: a permit register once described as public now sits behind a reCAPTCHA. And on 10 August a cleanup, “Remove superseded plan, audit and test-flow docs”, took several July studies out of the tree. They survive in git history, which is where this note quotes them from.

What I’d pass on

Ask a question that can change a decision. The 3D brief asked every track to say “what each answer decides”. The floor-plan study “deliberately stops short of a plan, because two of the three decisions are product/budget decisions”. A study that ends in a decision gets used; one that ends in a pile of facts gets read once.

Pay for the check, not only the search. Scouts find, and fact-checkers refute. One survey refuted 13 claims and one scout report 16; without that second step, they would have read as facts. When a check is cut short, write down where it stopped, as the July study did.

Measure the risky part before you plan around it. One afternoon’s spike found the 5.7 px floor that shaped the phone design. One laptop run put two viewers at 4.6× and 3.3× their budget. Both numbers came from running code, not from reading.

Research gives options, and a ruling decides, so write the ruling down. Prices and PostHog went against the research on purpose, and both rulings are recorded with their reasons. A refuted number needs the same care, or it comes back, as the 52% did.

Save research outside the session, with a date and what it replaced. Scratchpads die, limits cut runs short, and findings age. Cards written as you go, a folder no sibling session can sweep and a dated “Superseded” note let the next session trust what it reads.

Research told us what the product should be. Before any plan, I still needed to see it. That’s Part 3.

The series

  1. Three flats and a platform
  2. Research before anything
  3. See it before you build it
  4. Five languages, five of every mistake
  5. Drawn, not recognized
  6. Silently
  7. More hours than I should
cd ..