librats vs libp2p: what a 5 MB peer-to-peer node buys you

Every few years I end up needing the same thing: two processes, on two arbitrary machines, behind two arbitrary routers, talking to each other securely with nobody in the middle. It came up in rats-search, it came up again in a remote-desktop project, and both times I ended up gluing a DHT to some NAT traversal to some crypto by hand and hoping the seams held.

librats is what happened when I finally sat down and did it once, properly, as a library. It’s C++, with C, Node.js, Java, Python and Android bindings over the same core, and the whole point of it is to be small enough that putting peer-to-peer into an application is not an architectural decision.

The question I get, reasonably, is why not just use libp2p. So this year I stopped hand-waving and measured it.

Why I didn’t just use libp2p

I tried it first, and what stopped me wasn’t an argument, it was watching it run.

I had a node up with a not-very-large number of peers and a DHT search going, on a laptop that isn’t especially strong, and the CPU pegged at 100%. Memory was worse — I re-read the figure a couple of times because I assumed I had leaked something myself. For a workload that is fundamentally some Kademlia lookups and a stream of small messages, none of it added up. Whatever I was paying for, most of it wasn’t work I had asked for. That’s an impression rather than a measurement — the rest of this post is me going back and measuring it — but it’s the reason I started writing code instead of reading docs.

The second thing was languages. libp2p isn’t a library, it’s a specification with implementations, and those implementations are nowhere near equal. Go and JavaScript are the real ones, Rust is close behind. The C++ side was closer to a placeholder than to something I would ship, and I needed C++ — dropped into a desktop application, cross-compiled for Android and for cheap ARM boxes, shipped as a single binary. “Pick libp2p” in practice meant “pick Go or JavaScript”, which is not a language decision I wanted the networking library to make for me.

librats is the other arrangement. There is one implementation, in C++, and one C ABI underneath everything else; the Node.js, Java, Python and Android bindings are thin FFI layers over that same core. So they aren’t tiers. Whatever the C++ API can do, the Python one can do, on the same code, with the same behaviour — there is no “the good implementation” and “the other ones”.

The third thing is that I don’t need interop. Reaching any other libp2p node over any transport either side happens to support is what multistream-select, the muxer negotiation and the transport zoo are there to buy. If you need that, nothing else will do. I don’t: I want my application’s peers to find my application’s peers, and I’m paying for a negotiation whose outcome is known before it starts.

librats takes the other side of that trade. There is nothing to negotiate: your application’s protocol id is bound into the Noise handshake prologue, so a node from a different application doesn’t get told “no”, it fails to complete the handshake at all. One round trip instead of four.

What the thing actually is

A Node is a secure transport and nothing else — an encrypted channel, a self-certifying identity, manual dialing, raw channel messaging. Everything above that is a subsystem you attach explicitly before start():

librats::NodeConfig config;
config.listen_port = 8080;
librats::Node node(config);

node.add_subsystem(std::make_unique<librats::PubSub>());
node.add_subsystem(std::make_unique<librats::DhtDiscovery>(dht_config));

node.start();

The parts I’d defend as genuinely different:

• TCP and UDP as equals, on the same port. The UDP side is a real ordered/reliable stream — sequencing, cumulative and selective acks, RFC 6298 retransmission timing, Reno congestion control — over one socket shared by every peer. That’s what keeps a NAT holding one mapping instead of one per peer, and it’s the only reason hole punching works at all. A dial tries UDP first and races TCP alongside it, so a UDP-hostile network still connects.
• Identity is the key. Every node has a Curve25519 keypair and its PeerId is the public key. No PKI, no CA, no bootstrap of trust — both sides prove possession during Noise_XX or the connection doesn’t exist.
• The DHT is the BitTorrent mainline DHT. Kademlia, BEP 5, IPv4 and IPv6. It joins a network with millions of live nodes instead of needing its own populated overlay. That’s a bootstrap story libp2p’s kad-dht simply doesn’t have, and it’s the one thing I’d have written librats for on its own.

And a number that says most of what “lightweight” means here: a whole statically linked P2P node — DHT, NAT traversal, crypto, the lot — is 916 KB, and ldd on it prints libc, libstdc++, libm and the loader. Nothing else. The js-libp2p benchmark peer, for comparison, resolves 139 npm packages and 66 MB of files from 8 direct dependencies.

Measuring it without fooling myself

Two P2P libraries never agree on what a “connection” or a “message” is, so the harness gives every library the same five scenarios, the same wire, the same pacing and the same instruments, and treats whatever is left over as a difference between the libraries. The rules that mattered:

• Same wire. librats prefers UDP; the head-to-head runs TCP on both sides so the comparison isn’t smuggling in a transport difference. UDP is reported separately as an extra data point.
• Same pacing, swept. Both sides pace with an application credit window rather than each library’s own backpressure signal — pacing on those compares the signals, not the transports. More on this below, because it turned out to matter more than anything else.
• Nobody spins. The C++ sender blocks on a condition variable, the JS senders await. A spin-wait charges librats CPU that a single-threaded Node peer structurally cannot spend, and CPU-per-gigabyte is a headline number.
• CPU is both ends summed, read by each peer from its own getrusage / process.cpuUsage(), with process boot excluded.
• Three runs per cell, median reported. Nobody logs.

Intel Core Ultra 7 265KF, 36 GB, Linux, GCC 15.2 -O3 -DNDEBUG, Node.js v24.18.0, js-libp2p 3.3.8, TCP + Noise_XX on both sides, over loopback. That last word is a real caveat and I’ll come back to it.

Footprint

This is the part that isn’t close.

Five megabytes to be a running, listening, encrypted P2P node. A hundred and eleven for js-libp2p to do the same. With a hundred peers connected the gap widens rather than narrows, because the two sides are also paying very different amounts per peer:

Eight kilobytes against four hundred and seventy-five. That is the number I care about most, because it’s the one that decides whether an idea is possible on a router, a set-top box or a Raspberry Pi. A thousand peers costs librats about 8 MB of connection state; the same thousand peers costs js-libp2p about 475 MB, which on most of the hardware I care about is simply the end of the conversation.

Cold start is the one people underrate. If your P2P node lives inside a CLI, a short-lived worker or a desktop app the user launches, 188 ms of startup is 188 ms the user watches. Both Node figures include booting Node itself, which makes the comparison between them the interesting one: the librats addon is up in 22 ms, so what js-libp2p is spending is very largely the cost of resolving and evaluating that 139-package module graph before a node exists at all.

The connection-setup gap has a structural explanation rather than an implementation one. By spec, a libp2p TCP connection takes four round trips: TCP handshake, multistream-select for the security protocol, the Noise handshake, then muxer negotiation. Early muxer negotiation saves one of them, and there’s ongoing work on replacing multistream-select. librats negotiates nothing, so it’s one handshake and you’re connected. That’s a design difference rather than an implementation-quality one, and it is not the kind of thing you optimise your way out of — you can’t negotiate less than nothing.

Throughput is not the interesting number

One row in this study is not a win for librats, and I’d rather point at it myself:

Bulk is a tie. Run-to-run variance is ±3–5%, 600 against 603 is noise, and anyone reporting that as a win is selling something. What is not a tie is the small-message row: 863k against 93k, a factor of 9.3, with the same two libraries on the same wire in the same run.

That difference is the whole story, because most peer-to-peer traffic is not bulk. It’s gossip, presence, DHT queries, control messages, tiny state updates — hundreds of thousands of small things, not a handful of big ones. Per-message cost is what a P2P node actually pays.

There’s a detail in the bulk column I find more convincing than the throughput tie itself. librats’ ChaCha20-Poly1305 runs at 721 MB/s on this machine — portable C, no AVX assembly, benchmarked at parity with the noise-c reference it was ported from. Node reaches OpenSSL’s vectorised implementation, which is meaningfully faster. Both stacks are cipher-bound at these frame sizes. So librats matches js-libp2p’s bulk throughput while running a measurably slower cipher, which means its non-crypto overhead is much lower — and it does it at 3.27 CPU-seconds per gigabyte against 4.68.

Worth noting the other direction too: during the bulk transfer the js-libp2p listener used about 0.99 CPU-seconds over 0.44 s of wall clock, i.e. more than one core. Node spreads GC and libuv work across threads; librats’ single reactor thread does not. Part of that throughput parity is bought with more cores.

Three numbers that were wrong first

I want to spend a section on this, because the first numbers this study produced were wrong — one by 2.8x, one by 20x, and one in a way no ratio captures — and every one of them looked completely plausible.

The first bulk number was 68 MB/s for librats, against 419 for js-libp2p. It looked like a rout. It was the harness: I was pacing on librats’ own writable signal, which produced a send-one-frame / wait / send-one-frame ping-pong that spent 3.2 seconds of a 3.88-second run waiting. Switching to an application credit window fixed the ping-pong but left the ack cadence as a free variable, and that variable turned out to be enormous:

A 2.8x swing with no change to either library, on a knob that lives in the measuring instrument. js-libp2p barely moved across the same sweep, so any single fixed cadence would have handicapped librats specifically. The fix is to sweep it and report every library at its own best — the one setting nobody can be accused of being disadvantaged by. The generalisation, which I’d now apply to any cross-library transport benchmark I read: flow-control configuration is not a neutral setting, and a benchmark that doesn’t report its pacing isn’t interpretable.

The first handshake run reported js-libp2p at 100.9 connections/s. In fact 5 of 100 dials succeeded and the rate was computed over those 5. js-libp2p ships INBOUND_CONNECTION_THRESHOLD = 5 — inbound connections per second, per source IP — and on a loopback rig all 100 dialing identities share one source IP. A guard aimed at attackers fired on my measurement setup. Raised for the recorded runs, it gives 317/s.

It’s also a feature librats doesn’t have: it caps max_peers and nothing finer. That one is on my list.

And a quiet one. js-libp2p’s stream API is message-shaped — you send(bytes), the peer gets a 'message' event — which invites the assumption that one send is one message. It isn’t guaranteed. Probing it directly: 200 sends of 65,537 bytes arrived as 201 events, one message split into 65,533 + 4. It first showed up as byte counts twenty short of what was sent, which is small enough to dismiss as noise. That’s exactly what makes it dangerous. The libp2p peer now length-prefixes its own stream, and that framing cost is included in its results, because a real application would have to pay it too. librats’ frame layer delivers whole messages on a named channel, so there’s nothing to do — that’s an API-level difference, not a performance one.

Where librats loses

Three things, and I’d rather name them myself than have you find them:

• No cross-implementation interop. librats talks to librats. If you need to reach the IPFS network or somebody else’s node, this is the wrong library and there’s no clever workaround.
• No browser transports. No WebRTC, no WebTransport. That’s a category of application librats can’t serve today, and it’s the gap I’d close first.
• No QUIC — though librats’ own reliable stream over UDP already covers most of what I’d want QUIC for, on one shared socket, which is the part that matters for NAT.

Where this sits among the others

The honest framing of everything above is librats versus js-libp2p, with the word JavaScript left in. It says nothing about the other implementations:

• go-libp2p is the rematch I want most — it’s the implementation people usually mean, and the one I’d expect to make the best showing. There was no Go toolchain on that machine, and none for rust-libp2p either. One data point while we wait: go-libp2p’s resource manager defaults to a 1 MB per-connection memory limit. librats measures 8 KB.
• cpp-libp2p is the comparison that would settle how much of the gap is “C++ beats JavaScript” and how much is design. Everything above is confounded by that, and I’d rather say so than let the charts imply otherwise.
• Hyperswarm couldn’t be measured on a single-host rig at all: its connect() is a DHT rendezvous by public key rather than an address dial, so reaching a peer means also measuring a DHT lookup. That’s a gap in the study, not a verdict on the library.

One more number that has nothing to do with libp2p and everything to do with what it’s like to work in these libraries: creating 100 js-libp2p nodes in one process takes about 63 seconds. A hundred librats nodes take about one. That’s load-generator cost, not a benchmark result, but it’s the kind of thing you feel every day.

So what would I actually reach for

Need to reach libp2p nodes, IPFS, or browsers	libp2p — that is exactly what it’s for
Embedding P2P in a C++/Qt/Electron application	librats
Router, set-top box, Android, anything with a memory budget	librats — the per-peer number decides it on its own
Mostly small messages: gossip, presence, control traffic	librats
Want the existing mainline DHT instead of bootstrapping your own overlay	librats
Want Python, Java or Node to get the same feature set as C++	librats

And the caveat I’d want attached to every chart above: these are loopback numbers on one machine. No propagation delay, no loss, no reordering, no MTU, no middlebox — every congestion controller involved sits in slow start and never leaves it. Throughput here is a floor on cost, not a prediction of a network. The figures that survive a real path are the per-unit ones: CPU per gigabyte, CPU per message, bytes per peer. Those are the ones I’d read first, and they’re the ones librats wins by the widest margin.