A team of AI coding agents constructed a working 3D browser replica of San Francisco's Union Square in two hours for roughly $33 in API costs, according to an experiment published recently by PhiloLabs. The company deployed Claude Fable 5.1 agents to build the scene from actual geographic data and reference photographs, then used automated screenshot testing through Playwright to catch visual errors that traditional code tests would overlook. The entire run consumed approximately 8 million tokens and produced a Three.js application featuring 453 building footprints, 75 custom façades, 129 named storefronts, 220 pedestrians, and 109 vehicles including Powell Street's cable cars.
PhiloLabs split the reconstruction work among specialist subagents that handled geographic research, building geometry, textures, storefronts, and other scene elements. Once the application ran in a browser, Playwright moved through 34 predetermined camera positions and captured screenshots for comparison against real Union Square photographs. The agents generated 147 comparison sheets in total, allowing them to identify issues that were technically correct but visually inaccurate—such as buildings placed at proper coordinates but with incorrect proportions, or storefronts appearing on the wrong side of the street. Using identical camera positions across iterations made it easier to track changes from one development pass to the next.
The report explains that specialist agents then reviewed the comparison material, with some concentrating on architecture and geography while others examined technical art and interactions. Together these reviewer agents produced nine reports on the Union Square build, which became a punch list for the next development pass. According to PhiloLabs, this division proved useful because not every mistake translates cleanly into a test—while code can verify whether a building sits at the right coordinates, it's far harder to write a test that determines whether the street actually resembles Union Square. The company notes that another agent checking the work doesn't guarantee correct decisions, especially when source material is incomplete, because the reviewer can miss the same thing the first agent did.
The low cost came from splitting the job among subagents, running tasks in parallel, and reusing cached context rather than having a single agent work through the entire project from scratch, the report states. PhiloLabs acknowledges that Union Square was a fairly contained experiment and that screenshots become less useful once agents move into complicated applications—Spline recently rebuilt its 3D editor using Claude Code agents, but the finished interface only shows part of what those agents built, with problems buried in code or triggered by actual user behavior potentially never appearing in a screenshot. The agents had to assemble open geographic data from OpenStreetMap and USGS elevation data alongside information about the real location, then convert everything into geometry, façades, and objects that would run in a browser, filling gaps where geographic data showed where a building belonged without revealing what its façade looked like, while photographs only captured parts of buildings visible from particular angles. The approach demonstrates how visual testing can extend AI agent capabilities beyond pure code correctness, though the technique's limits emerge quickly as application complexity grows and the gap between what screenshots reveal and what users experience widens.