The question that broke Dave

I wrote about Dave here in May 2024 — a peer-to-peer network that disseminates packets of data, without accounts, without a token, and without anyone in charge. That article carries an edit note admitting it went out of date within months. This is what happened in those months, and why I’ve just thrown most of it away.

Underneath every version of it there’s one question, and it’s the reason I keep coming back to this instead of using something that exists:

Can you build a network that behaves like a shared pool of memory — anyone writes, anyone reads, the data stays around — with no payment system, no token, and no accounting of who is storing what? Just an incentive structure simple enough that nothing has to be coordinated at all?

The constraint is what makes it interesting. Filecoin, Arweave and Storj all answer the durability half with money and proofs of storage, and they work. Take money off the table and something else has to make a node want to keep your bytes, or the pool is only ever as good as the goodwill sitting in it.

Two years on, Dave has produced three different answers to that question. Two of them are in this post because I built them. The third I only recognised while writing it, and it had been sitting in the code since 2024.

The question in this article’s title is a much smaller one, and much more boring. It’s the one that broke it.

One thing to be clear about before any of the criticism, because it changes how you should read it. This was months of full-time work, and the decisions in it were deliberate — argued through, mostly written down, often revised more than once. What follows isn’t a list of things I hadn’t considered. It’s largely the opposite: mechanisms I designed carefully, which fail for reasons that only surface when you push a formula to its limit, or think like an attacker for five minutes, or run the thing at a scale I never reached. That’s a more useful kind of mistake to read about than carelessness, and a far more common one.

Where the code is now

I’ve reset godave’s main branch back to commit 4bc87d5, from 19 November 2024, and tagged it pre-signed-dats. Everything after it — seventy-nine commits, three more weeks — lives on a branch called signed-dats. Nothing is lost, but the default branch is now the version I actually want people to read.

That version is 1,074 lines. It builds, and its tests pass. The version I abandoned in December is 3,573 lines across nine packages, and I could no longer hold it in my head.

The whole divergence is one commit, and you can see it in a single diff of the protobuf spec:

 enum Op {
-    DAT = 2;                    a thing that exists
+    PUT = 2;                    an operation on a named slot
 }
 message M {
-    bytes v = 3;   // value
-    bytes t = 4;   // time
-    bytes s = 5;   // salt
-    bytes w = 6;   // work
+    bytes datKey = 3;           addressable
+    bytes val = 4;
+    bytes time = 5;
+    bytes salt = 6;
+    bytes work = 7;
+    bytes pubKey = 8;           ownership
+    bytes sig = 9;              asymmetric crypto
 }

Addressability, ownership and signatures all arrive together. Before that commit a dat was a value, a time, a salt and a proof of work — content-addressed by its own hash, with no key and no owner. Nobody could update it, because there was no it to update; there was just the data, and the data was its own name.

The rename tells the story on its own. DAT is a thing that exists. PUT is an operation on a named slot. I changed one into the other in an afternoon and spent the next three weeks discovering what that cost.

What it did

Each node keeps a table of peers and a table of dats. Every epoch it picks one dat at random and sends it to one peer at random. That’s the entire dissemination mechanism, and it’s the part I still like.

Storage was governed by something I called mass:

func Mass(work []byte, t time.Time) float64 {
	return float64(nzerobit(work)) * (1 / float64(time.Since(t).Milliseconds()))
}

Difficulty divided by age. The idea was that a dat with a harder proof of work should persist longer, and that everything should eventually decay unless somebody keeps re-publishing it. Trust worked the same way: when a peer sent you a dat you didn’t already have, its trust score went up by the mass of that dat, and trusted peers got picked more often for gossip.

That is the first answer to the question at the top: measure what a participant contributes, and pay them in the only currencies the network has — a share of its disk, and a share of its bandwidth. No token, no ledger, and a pleasing symmetry between the two halves.

The part that worked

Before the things that didn’t, the part I’d write again unchanged — and which survives into the rewrite more or less intact.

Peer discovery and liveness are the same pair of messages. You send GETPEER; a well-behaved node replies PEER with a couple of addresses. If a peer hasn’t been heard from in DROP, it’s deleted. There’s no separate heartbeat, no separate bootstrap protocol, and no membership list — the packet that discovers peers is the packet that proves they’re alive.

That doubling-up turns out to do a third job as well, and it’s the one I’m most pleased with.

Most nodes are behind a home router, and two of them can’t exchange UDP without help. The usual answer is a rendezvous server that tells both sides to transmit at the same moment, so each one’s outbound packet opens a return path through its own router. Dave has no such server and no code that mentions NAT at all. It gets there anyway, out of three unremarkable decisions:

// a packet from a stranger adds the stranger
_, ok := peers[pkfp]
if !ok { peers[pkfp] = &peer{pd: pkpd, fp: pkfp, added: time.Now(), seen: time.Now()} }

// gossiped addresses are added unconditionally
case dave.Op_PEER:
    for _, mpd := range pk.msg.Pds { if !ok { peers[mpdfp] = &peer{...} } }

// and the tick pings every peer in the table, verified or not
case <-pingTick.C:
    for pid, p := range peers {
        if !p.edge && time.Since(p.seen) > DROP { delete(peers, pid) }
        else if time.Since(p.peermsg) > PING { pktout <- &pkt{&dave.M{Op: dave.Op_GETPEER}, addr} }
    }

A newly gossiped peer has never sent a PEER message, so time.Since(p.peermsg) is enormous and it gets a GETPEER on the very next tick. Which means: once an edge node has told A about B and B about A, both sides start pinging each other within eight seconds, independently. Each one’s outbound GETPEER opens its own router for the other. Whichever packet arrives once both mappings exist gets through, and the receiver adds the sender on sight.

That’s a simultaneous open with nobody coordinating it. It works through full-cone and address- or port-restricted routers, which is most domestic kit; symmetric NAT and carrier-grade NAT still defeat it, because there the external port differs per destination and the address the edge saw is no use to anyone else. Good enough that I ran Dave across EC2 instances and Raspberry Pis at home without ever writing traversal code.

There’s a defence in there too, and it’s the detail I’d most like to have back if I’d lost it. Look at the two constants:

DROP        = 16 * time.Second // Time until silent peers are dropped from the peer table.
SHARE_DELAY = 20 * time.Second // Time until new peers are shared. Must be greater than DROP.

When you reply to a GETPEER, you only offer peers you’ve known for at least SHARE_DELAY. A peer that never answers is deleted at DROP. Sixteen is less than twenty, so an address that doesn’t respond is dropped before it ever becomes eligible to be passed on. You can inject a bogus address into one node’s table — anyone can, it’s a UDP packet — but it dies there, and no amount of injecting makes it spread. That’s peer-table poisoning closed off by an inequality between two constants rather than by any code that knows what poisoning is.

The same section has a rate limit on PEER messages: one accepted per PING interval, so a malicious peer can’t flood you with addresses to drown out the honest ones. I found that hole while writing the 2024 post and fixed it in the same afternoon, which is the only time writing an article has directly found me a bug.

Two honest limits on that, before I sound too pleased with myself. It stops addresses that are dead or invented; it does nothing about a live malicious node, which responds and therefore gets shared like anyone else. And discv5’s endpoint proof — the grown-up version of this — has exactly the same limitation, so at least the ceiling is in the right place.

I should be calibrated about the rest of it too. Combining discovery with liveness is what Kademlia’s FIND_NODE does. Refusing to gossip reputation is a call several deployed networks got wrong, but it isn’t new. Implicit hole punching happens incidentally in most DHTs — what’s unusual here is only that it was designed for rather than stumbled into. The delay inequality is the one I’d defend as properly neat, and even there the goal is standard; it’s getting it from an ordering between two timeouts rather than a round trip and some cryptography that I haven’t seen elsewhere.

So: not novel, mostly. I’ve stopped thinking that matters much. Landing independently on the same answers as the systems that got it right is the outcome you want from a first attempt at something, and it’s a better one than a genuinely new idea that doesn’t work.

None of this is what went wrong. The dissemination and storage side is where it comes apart, and it does so in three separate ways.

One packet is enough

Look at what happens to mass as age approaches zero.

Trust accumulates like this:

if novel {
    trust := peers[pkfp].trust
    if trust < MAXTRUST {
        trust += Mass(pk.msg.W, Btt(pk.msg.T))

MAXTRUST is 25. The cheapest dat the network accepts has eight leading zero bits, which costs about 256 hashes to find — call it free. Mass for that dat is 8 / age_in_ms, so it exceeds 25 at any age below 0.32 milliseconds.

An attacker mints a dat, hands it straight to the socket, and it arrives at a peer sub-millisecond on a LAN or from a colocated machine. It is novel by construction, because they invented it. One packet takes them from nothing to the trust ceiling.

And if the timestamp on the dat is the current millisecond, time.Since(t).Milliseconds() is zero, 1 / 0 in float64 is +Inf, and mass is infinite. The cap saves the arithmetic from doing any damage, but only by discarding whatever the number was supposed to mean.

The mistake underneath the arithmetic is the interesting one. I was rewarding peers for supplying novel data with valid proof of work, and I’d convinced myself that was evidence of useful contribution. It isn’t. It’s evidence of having spent CPU, and the attacker has CPU. A reputation metric only works if the thing it rewards is scarce to an adversary and abundant to an honest node, and “bytes nobody has seen before” is the opposite: trivially abundant to whoever is willing to make some up.

MAXTRUST exists precisely because I’d thought about farming — an uncapped score is obviously gameable, and the cap bounds what any peer can accumulate no matter how much they send. What defeats it is the interaction between the cap and the divergence, and that only shows up if you take the formula to its limit and ask what mass is worth at an age of zero. A cap you can reach in one packet isn’t a reputation system; it’s a single bit, and the bit says “has sent me something”, which every attacker can also say. The defence was real. It was aimed at the wrong axis.

Mass was recency wearing a costume

Dats are stored in shards, and the shard is chosen by difficulty:

func keys(work []byte) (uint8, uint64) {
	return min(MAXWORK, nzerobit(work)) - MINWORK, xxhash.Sum64(work)
}

Every dat in shard k has exactly k + 8 leading zero bits. When the pruner keeps the most massive dats in a shard, the difficulty term in mass is the same constant for every dat it’s comparing — so the comparison collapses to one over age. Within a shard, mass is recency. The difficulty half of the formula does no work at all.

It isn’t quite doing nothing globally, and what it is doing is the best idea in the codebase — which I only worked out properly while writing this.

Difficulty picks your shard, and each shard has its own independent capacity. So a harder proof doesn’t buy you priority directly; it moves you into a less contended queue. And because every node prunes its own shards independently, the effect compounds across the network into a self-clearing price for permanence. Want a dat to survive? Do more work and land somewhere emptier. As more people do sixteen bits, that tier fills and the price drifts to twenty. Nobody sets the price, nobody verifies anything, no payment changes hands, and there is no ledger, no contract and no challenge-response anywhere in it. It falls out of local pruning decisions alone.

That’s worth sitting with, given where the project went. Every other way of buying durability in a decentralised store is heavy: Filecoin and Arweave use money and proofs of storage; Freenet uses popularity. The version of Dave I abandoned used storage challenges — the mechanism that dragged in replica tracking and reassignment and eventually ate the whole design. I already had a mechanism that gets a weaker version of the same property for nothing, and I walked past it to build the expensive one.

I can’t claim it’s unprecedented; I know the well-known systems rather than every paper. But I haven’t seen proof-of-work difficulty used as a storage tier with per-tier caps anywhere else, and I’d have expected to.

What defeats it in practice is mundane. Almost nobody does more work than the minimum, because nothing in the interface suggests a reason to, so everything piles into the first shard and fights over the same slots while fifty-five tiers above it sit empty. A market with one participant doesn’t clear. Combined with the mass formula collapsing to recency, the effective policy was “keep the most recent few thousand dats”, which is not what the documentation said. The interaction is subtle enough that I wrote both halves and still didn’t see it: partitioning by difficulty and then ranking by a formula containing difficulty looks like belt and braces, right up until you notice the second term is identical across everything it’s being asked to compare.

The idea deserves a better implementation than I gave it, and it’s separable from all of this — see below.

The anonymity claim was too strong

From the README on that pinned commit:

Propagating dats in this way ensures that an adversary is unable to create a timing attack to discern the source of a dat, even with a broad view of network traffic.

The mechanism is sound and I still like it. Because every node emits one random dat to one random peer every epoch, at a constant rate, there’s no correlation between when a node received something and when it passes it on. A forwarder is genuinely indistinguishable from another forwarder.

But that’s forwarding. Origination is a different act, and the code does it differently:

case dave.Op_DAT:
	put(dats, &Dat{m.V, m.S, m.W, Btt(m.T)})
	for _, p := range rndpeers(peerList, trustSum, FANOUT, 0, 0) {
		pktout <- &pkt{m, addr}
	}

When an application writes a dat, it goes out to five peers immediately, and then repeats on a timer for however many rounds the caller asked for. The originator is the first node to transmit it and the most frequent. Random push hides everyone downstream of the source. It doesn’t hide the source, which is precisely what the sentence claims it does.

I don’t think I was being dishonest when I wrote that. I think I’d designed one mechanism carefully, and then let the claim it supported creep one step wider than the evidence.

The one I never solved

Those three are things I got wrong. This one is different — it’s the problem I worked hardest on, never cracked, and still think is the most interesting thing in the project.

Every epoch a node sends one dat to one peer. Which dat?

v1’s answer is: pick one at random. That works beautifully at the scale I ran it — data moved around exactly as it should across a handful of machines. But the selection probability for any given dat is one in N, so as the store grows a newly-arrived dat waits proportionally longer for its turn. At a thousand dats it’s fine. At a million, a new dat is essentially never chosen, and propagation stops working at all.

I tried a lot of things. The one still in the pinned code is a ring buffer: every novel dat is written to a thousand-entry ring, and each push epoch sends one dat from the ring as well as one from the store at large. That genuinely fixes the acute half — new data propagates immediately, however big the store gets.

What it doesn’t fix is the tail. A dat that falls out of the ring is back to one in N, so anything a node missed while offline, or anything that existed before it joined, only ever reaches it by luck. I also tried cuckoo filters: track what each peer has already seen, and don’t waste a push sending them something they’ve got. That work is on a branch called filter and it isn’t in the pinned version, which tells you how it went.

None of it worked because I was asking one mechanism to do two jobs with opposite requirements. Getting new data out fast wants a small working set, aggressive sending, and a short life. Repairing the long tail wants completeness, an unbounded working set, and can afford to be slow. A single random pick from a single pool is a compromise that is bad at both.

The literature has known this since 1987. Demers et al., Epidemic Algorithms for Replicated Database Maintenance, splits it exactly along that line: rumour mongering for new items — push them hard, and each time you push to someone who already has it, probabilistically stop, so the hot set limits itself and stays small — and anti-entropy in the background, periodically reconciling state with a random peer. You run both. My ring buffer is rumour mongering with its stopping rule missing; I had the instinct without the feedback that makes it terminate.

And the cuckoo filter was simply the wrong primitive, which is the part I’d most want to tell 2024 me. A membership filter answers “do you have this one?”, an item at a time. The question you actually need answered is “what are you missing?” — in one round trip, in bytes proportional to the size of the difference rather than the size of the set. That’s set reconciliation, and it’s a different tool entirely: Merkle trees over sorted ranges, as Cassandra and Dynamo use for repair; IBLTs, as Graphene uses for blocks; or PinSketch, which is what Bitcoin’s Erlay uses — having hit precisely this problem with transaction relay and solved it by pairing low-fanout flooding with periodic reconciliation.

The rewrite changes the shape of the question rather than answering it head-on. Flooding doesn’t select: every cell goes to every neighbour exactly once, deduplicated by the seen-set, so for new data there’s no choice to make and the problem doesn’t arise. What’s left is the tail — a node that was offline, or has just joined — and that is now a separate mechanism with its own name, the time-range sync. It’s the same split Demers describes, arrived at from the other direction.

Which puts the interesting work in exactly one place. That sync is currently naive and re-sends things you already have. Replacing it with real range-based reconciliation is the piece I most want to build, and it’ll be the first time in this project that the hard problem in front of me is one with a well-understood answer.

The question

Here’s the one that actually killed it.

A dat is addressed by its own work hash. To retrieve one, you send GET with that hash, and any peer that has it replies. So: how do you get the hash?

You don’t. Not from the network. There’s no index, no enumeration, no namespace, nothing to browse. You can only ask for something you were already told about through some other channel — and if that channel exists, a lot of the reason for the network existing has quietly moved into it.

It’s worse than that, because GET doesn’t route. A read checks your own store first, and on a hit it returns without sending a single packet:

case dave.Op_GET:
	shardKey, datKey := keys(m.W)
	dat, ok := dats[shardKey][datKey]
	if ok {
		found = true
		apprecv <- &dat
	}
	if !found {
		for _, p := range rndpeers(peerList, trustSum, FANOUT, 0, 0) {
			pktout <- &pkt{&dave.M{Op: dave.Op_GET, W: m.W}, addr}
		}
	}

On a miss it goes to five peers picked at random and repeats on a timer. No key space, no distance metric, nothing pointing the query towards whoever holds the data.

That local check is worth dwelling on, because it looks like the answer and isn’t. It’s a cache lookup, and a cache is only as good as the thing it misses to. What sits in a node’s store is a random sample of recent traffic — you receive one random dat per peer per epoch, and the pruner discards everything past the shard capacity every minute. The chance that an arbitrary hash happens to be in there is low, and it falls as the network holds more data. The fallback is doing the real work, and the fallback is the broken part.

Which is also why I didn’t see this for months. On a dozen nodes on my desk, with capacity far exceeding the data, the local hit rate is close to one and the network path almost never fires — and when it does, five out of twelve peers is most of the network. Dave behaved correctly at every scale I ever ran it at. The lookup only fails once the data in the network exceeds what a single node can hold, which is precisely the point at which you would want it to work. I called Dave a distributed hash table in that first article. It was a distributed hash table with no table, and a small enough network to hide it.

So I gave dats a key, and an owner, and a signature — the commit at the top of this piece. Now data had a name you could form without being told it, and a namespace per public key you could enumerate. That is a genuine fix for a genuine problem, and I’d make the same call again with the information I had.

What followed was mechanical. If dats have keys, the sensible place to put a dat is near its key, so I added XOR distance. If placement is deterministic, some node is supposed to hold each dat, so I added replica tracking. If replicas are tracked, peers leaving means reassigning them, so I added that. If replicas can be reassigned, I need to know whether a peer is actually storing what it claims, so I added storage challenges. Values then outgrew a UDP datagram, so I added a TCP fallback — a second transport, with its own code path for the same operation.

Every one of those follows from the one above it. Not one of them was a bad decision on its own terms. Together they deleted the property the whole thing was for: once placement is deterministic, an observer knows exactly which nodes should hold what, and “random push for anonymity” no longer hides anything at all.

By December I was sketching a third system inside the second — direct messaging, layered on the keyed store, with recipient-keyed namespaces. I never committed it. That’s where I stopped.

Deleting the question

I’m writing it again, in Zig, and the whole design rests on one rule I wrote down before any code:

A node never needs to know anything about the network beyond its immediate neighbours.

The point of the rule isn’t what it permits. It’s what it refuses. GET(key) means routing a query towards a node you can’t see, so there is no lookup — and with no lookup there is no placement, no replica set, no reassignment, and no storage challenge. The entire collapse is rejected by one sentence, and it’s rejected automatically, before I have a chance to talk myself into each step.

So how do you read anything, with no lookup?

You already have it. Everything floods to everyone, and reading is a query against your own disk. Following someone is a filter over your own store by their key. There’s no relay to ask, no query to route, and nothing to place, because nothing was ever anywhere in particular.

That is the same first move v1 made, with a completely different guarantee behind it. v1 checked a cache and fell back to a broken query; this checks a replica, and there is no fallback because none is needed. The difference isn’t the lookup — both start by looking in the same place. The difference is whether the design is working for or against that lookup succeeding. Random push and mass pruning actively hold a v1 node’s store down to a small sample; flooding and a retention window deliberately fill it.

That sounds like it can’t possibly scale, and it scales further than I expected. At a million users each posting ten times a day, a node carries about 0.6 Mbit/s and 2% of one core. Ingress doesn’t bind until several hundred million users on a normal home connection. Storage is the real cost, and it’s each node’s own choice of how far back to keep — a phone keeps a week, a machine with a spare disk keeps everything and becomes an archive.

That last part is deliberately not a protocol feature. There’s no mechanism to declare an archive, verify one, reward one or challenge one, and there is never going to be, because that mechanism is exactly what “storage challenges” were. Data stays available for as long as somebody chooses to keep it, which is how Usenet worked and how Nostr works. Any protocol promising more than that is either lying or has an accounting system that will eventually eat it, and I’ve now written the accounting system that ate mine.

That is the second answer, and it works by refusing the question. Nobody is doing anyone a favour, so there is nothing to pay for: you receive everything regardless, and you keep the part you want to read. The simplest possible incentive structure is no incentive structure — just self-interest, and no coordination because none is required.

It’s an honest answer and I think it’s the right one to build on. It also has a hole in it, which took me a while to see. Self-interest keeps the data somebody wants. It does nothing at all for data that nobody in particular wants yet — the unpopular, the archival, the thing you publish today that matters in five years. Under this scheme that data quietly evaporates, and there is no way to pay to keep it, because paying is the thing we removed.

Everything on the wire is one fixed size:

There’s no recipient field. Every node tries to open every cell against its own key, and the authentication tag says whether it worked — so nobody, including the node holding a cell, can tell who it’s for. On my machine that costs 31 microseconds a cell, essentially all of it the X25519 scalar multiplication.

The peer layer comes across almost unchanged: discovery and liveness in one message pair, gossip only in replies, sharing gated behind a delay longer than the drop timeout, and reputation kept strictly local and never gossiped. The one correction is that identity becomes the public key rather than the address, because a table keyed by ip:port treats a node that moves as a stranger — and on IP, moving is the normal case, not the exception.

The traversal needs writing down this time, which it never was. Nothing in v1 records that the ping interval is load-bearing for anything but liveness, and the obvious optimisation — back off the tick, skip unverified peers — is correct for liveness and quietly fatal for hole punching. I nearly made exactly that change while reviewing this code, having read the same three lines and concluded the traversal was accidental. It wasn’t; it was the reason the discovery design looks the way it does. The mechanism survived, and the note explaining it didn’t.

Why Zig

Partly because I want to, which is a sufficient reason for a hobby project. But there’s a real argument too, and it’s about where this code can run.

The protocol has no business needing an allocator on the receive path: a cell is a fixed 224 bytes, and opening one is arithmetic on a fixed-size buffer. Zig lets me write that once and compile it for a server or for a microcontroller with no operating system underneath it. I checked rather than assumed — the whole receive path, X25519 and BLAKE3 and ChaCha20-Poly1305 and the framing, cross-compiles to a Cortex-M4 in an eighteen-kilobyte object, with no allocator and no OS.

I’m not building the radio version now; I scoped this rewrite to IP so that it might actually get finished. But it means the door stays open, and it’s the one thing this project can do that the mature alternatives can’t.

Zig 0.16 is also recent enough that most of what’s written about the language online no longer compiles. main takes a std.process.Init, randomness comes through std.Io, std.time.Timer is gone. I’ve been finding these by compiling rather than by reading.

A loose end worth picking up

The third answer is the one I didn’t recognise until I sat down to write this, and it had been in the code since 2024. It’s also the piece that fills the hole in the second: a way to buy permanence for data nobody else wants, paid for in work rather than money, with nothing keeping accounts.

I want to write it down properly, because it doesn’t need a peer-to-peer network to be interesting and I’d like to come back to it.

The mechanism. Every stored item carries a proof of work of difficulty d, in leading zero bits. The store is partitioned into tiers by d, and each tier has its own capacity. When a tier is full, it evicts within itself — oldest first is fine. A writer chooses d before publishing, so a writer is choosing which queue to stand in.

Why it should clear. Treat each tier as a queue with capacity C and arrival rate λ. In steady state, an item’s expected residency is T = C / λ — Little’s law, and for evict-oldest it’s exact. So the writer faces a straight trade: pay 2^d hashes, receive C_d / λ_d seconds of retention.

That’s a market, and it clears without anyone running it. If a tier gets popular its arrival rate climbs, residency falls, and the marginal writer moves up a bit. Cost doubles per bit while capacity stays put, so demand spreads itself across the tiers geometrically. No payment, no ledger, no proof of storage, no contract — just local eviction, on every node independently, adding up to a price.

What was missing, and it’s the whole failure. A market needs four things. A cost function: proof of work, present. Differentiated goods: the tiers, present. Independent clearing per good: per-tier caps, present. And a price signal — which was entirely absent. Nothing in Dave ever told a writer that tier 0 was full and tier 12 was empty. There was no way to observe occupancy, so there was no way to act on it, so every writer rationally paid the minimum. The market had a price and no way to display it, and a market with one participant doesn’t clear.

The fix is small: have nodes report how full each of their tiers is, and let a writer sample a few and pick a difficulty from how long they want the thing to live. Occupancy is aggregate information about your own disk, so publishing it leaks very little.

How to test it with no network at all. This is why it’s worth doing separately. Simulate a population of writers with some distribution of how much they care about persistence, and one node’s store with tiers and caps. Feed arrivals in, evict, and measure whether residency time ends up proportional to cost. That’s a discrete-event simulation of a caching policy in a couple of hundred lines, and it answers the question completely — no sockets, no peers, no protocol.

Four things I’d want it to tell me:

• How to divide capacity across tiers. v1 gave every tier the same cap, which wastes most of the disk when demand is concentrated at the bottom. Allocating in proportion to observed demand is self-defeating, because the tier you escape to grows to meet you. There’s a right answer here and I don’t know what it is.
• Whether it oscillates. Publish occupancy and everyone can see the same empty tier. They all move there, it fills, they all move again. That’s textbook herding, and it probably needs damping or hysteresis to settle.
• What lying buys you. A node that reports tier 20 as empty makes writers burn work for nothing. Sampling several nodes and taking a median helps, and a writer can check empirically whether the item actually survived, but this needs thinking about.
• Whether the absolute numbers work at all. Dave’s minimum was eight bits — 256 hashes, which is nothing. Twenty bits is about a million, still only seconds of CPU, so the “expensive” tiers can be squatted cheaply. The scheme only prices anything if the top tiers are genuinely costly, which means calibrating against current hardware rather than against a number I picked in 2024.

If it survives that, it’s a retention policy any store could use, and it never needed to be part of a network in the first place.

What I don’t know

Whether the question at the top has a good answer at all. Three attempts so far. Reward measured contribution — which failed, because anything I could measure locally was something an attacker could manufacture. Refuse the premise and rely on self-interest — which works, and leaves anything unpopular to rot. Price permanence in work and let local eviction clear a market nobody runs — which is untested, and is the only one of the three that’s an incentive structure in the sense I originally meant. One is built, one is a cautionary tale, and one is a weekend of simulation away from being either interesting or dead.

Whether spam is survivable. Flooding means anyone can inject, everyone must process, and now that data is retained, spam persists as well as propagates. My current answer is to ration rather than reward — per-neighbour rate limits, and a single shared budget that all unknown authors compete for, so ten thousand fake identities get one stranger’s worth of throughput between them. That’s an argument, not a result. So was the trust metric.

Whether anyone will use it. Almost certainly not many, and that’s fine. I want it to be good, and used by a handful of people I know, which is a different target from the one I was implicitly aiming at in 2024 and a much more reachable one.

What I do know is the thing I’d tell 2024 me. The first version had no rule that could reject a feature. Every addition answered “is this good?”, and each one honestly was. None of them ever had to answer “is this ours?” — and by the time that question would have been useful, there was no longer a clear enough idea of what “ours” meant to answer it with.

The code is at github.com/intob/godave, pinned where it was still one idea.