AWS Lambda needed a complete network ledger for every tenant workload, no matter how briefly it ran or how much traffic it generated, but its old system buckled under density. The company detailed its rebuild in a technical report published on The New Stack, explaining how it replaced an aging capture architecture with a purpose-built pipeline written in eBPF and Rust. The old design, borrowed from a less-dense EC2-era system, couldn't scale to Lambda's workload: thousands of microVMs running for a few hundred milliseconds each, then shutting down. Within that brief window, logs captured during those milliseconds became the only evidence left.
The old system broke at Lambda's density for two reasons, according to the report. First, rule explosion: one worker running roughly 2,000 microVMs required over 100,000 iptables rules just to maintain the record, and iptables walks its rules more or less linearly for every packet. Every packet paid a tax proportional to how crowded the host was, slowing down as the host got busier—exactly the wrong direction when the goal was to pack more microVMs onto a host, not fewer. Second, the borrowed kernel module didn't support IPv6, and once dual-stack IPv6 support was proposed for Lambda, the old approach was finished. The new system uses small eBPF programs attached to traffic-control hooks on each network's virtual devices, intercepting packets and emitting one compact event per packet into a ring buffer. Each tagger process, written in Rust, runs unprivileged and drains its own dedicated ring buffer, rolling raw per-packet events into per-flow records and writing them to disk in the legacy Amazon Ion format.
The report states that the replacement eliminated the linear rule walk that taxed every packet as the host filled up, swapping it for constant-time map lookups whose cost doesn't climb with host density. "Within milliseconds, the workload is done, and the logs captured during those few milliseconds are the only witness left," the authors write, describing the challenge Lambda faced. Each tagger process lives in a few hundred kilobytes of RAM against a roughly one-megabyte budget, small enough that thousands per host became practical. Activating a flow into steady-state recording stays under 2 milliseconds at the 90th percentile and under 10 milliseconds at the 99.9th percentile, matching the baseline of the system it replaced. The output records are byte-for-byte identical to the old format, so every downstream flow-log and metering consumer kept working with no change, and the two systems could be run side by side to confirm they agreed record for record.
The shift happened because the old architecture imposed costs that grew with density, while the new one holds constant. The linear tax from iptables meant that as Lambda packed more microVMs onto a host, each packet's bookkeeping slowed, limiting the ability to serve requests under load. The eBPF programs, by contrast, just watch: they can't copy, block, drop, or rewrite a packet, so there's no code path for interference. The ring buffer's size was derived from each microVM's packet rate ceiling—100,000 packets per second per direction—multiplied by the drain interval of roughly 100 milliseconds, by the event size of about 24 bytes, and by two directions, yielding a floor of around 300 KB and a default of 512 KiB. The report notes that in production, the buffer is currently provisioned more generously, on the order of a couple of megabytes, while the team finishes tuning the right per-workload value. The kernel decides when to wake the userspace process, checking how full the ring is and only forcing a wakeup once it crosses about one percent full, so a quiet flow just sits there until the next drain, basically free, while a busy one trips that threshold and gets read almost immediately.
The report concludes that IPv6 flows, invisible to the old tool, now get recorded like anything else, covering the whole address space instead of half. The capture layer is observe-only and formally checked with a model checker during every build, and the processes touching customer traffic hold no elevated privileges: the orchestrator opens the ring buffer and hands the open file descriptor to the tagger over a Unix domain socket, so the tagger never needs permission to create one. The authors argue that keeping tenants apart at the point of capture is critical, giving each tenant its own ring and devices so streams never touch and attribution happens on clean traffic instead of after the fact. They also recommend that anyone swapping out an engine keep the bolt pattern: byte-for-byte identical output allowed AWS to replace the entire capture path with zero downstream migration and handed them a record-for-record way to prove the new system saw everything the old one did. The platform can now target roughly double the microVMs per host without the burst-time gap a per-packet tax invites, and it added significant visibility while shrinking the trusted, privileged surface that could corrupt the record. Observability architectures that treat capture as a tax rather than a watch will eventually hit the same wall Lambda did; the lesson for high-density platforms is that privilege boundaries and buffer math aren't details to tune later—they're load-bearing from day one.