openspec/swarm: Revise and review spec

openspec/changes/swarm/design.md

@@ -1,425 +1,685 @@
-# Design: swarm overlay for subnet discovery
+# Design: swarm overlay for subnet rendezvous
 
 ## Context
 
-Today a "network" is one `P2p` instance identified by `{magic_bytes,
-app_name, app_version}`. Magic bytes are checked at channel setup
-(`src/net/channel.rs`, raw frame read) before the version/verack handshake
-checks `app_name`. Hostlists, refinement, and datastores are per-`P2p`.
-Lilith spawns one `P2p` per configured network (`bin/lilith/src/main.rs`,
-`spawn_net()`), each needing its own listener, datastore, and config section.
-Apps construct their own `P2p` at startup from static seed lists.
-
-Extension points already public in `src/net` and sufficient for an overlay
-without core surgery:
-
-- `ProtocolRegistry::register(session_flags, constructor)` — attach custom
-  protocols per session type (`protocol_registry.rs`).
-- `#[macro_export] impl_p2p_message!` (`message.rs`) — define new wire
-  messages with metering; already used by `event_graph` and fud.
-- `GetAddrsMessage`/`AddrsMessage` gossip via `ProtocolAddress` establishes
-  the pattern ads should follow: relayed, unsigned, refinement-filtered.
-- `src/dht` exists for content-keyed lookup (Kademlia) and is used by fud;
-  it is the wrong tool for membership (structured lookup paths are linkable).
-
-See proposal.md for motivation. Constraints that shape this design: no
-changes to channel framing, magic-byte gating, or handshake semantics; the
-overlay must not become a cross-subnet correlation point; no stable node
-identity may cross subnets.
+See `proposal.md` for motivation and delta specs for normative behavior. Current
+implementation constraints are:
+
+- One DarkFi network is one `P2p`, isolated by magic bytes and an app-name plus
+  major/minor handshake. Host state, sessions, registry, and persistence belong
+  to that instance.
+- `Settings.seeds` uses short-lived `SESSION_SEED`; it cannot carry swarm
+  queries. `Settings.peers` creates ordinary manual channels.
+- Current manual peers carry only `Url` and transports resolve internally.
+  Exact cached sockets therefore require an explicit pre-start manual-target
+  API and connector path; they cannot be represented by `Settings.peers`.
+- `ManualSession::reload()`, `SeedSyncSession::reload()`, and
+  `InboundSession::reload()` currently do not reconcile changed addresses.
+  Bootstrap fallback, source fallback, and serving promotion cannot depend on
+  reload.
+- `Hosts::subscribe_channel()` publishes completed seed/refinement channels as
+  well as ordinary channels. Join completion must filter session type.
+- `VersionMessage.features` is retained remotely but sent locally as empty.
+  Existing variable version fields can combine past `VERSION_MAX_BYTES`, so
+  feature validation alone is insufficient.
+- An ad store knows only one-way `SubnetId` and addresses. It cannot perform a
+  subnet handshake and must not become an attacker-controlled dialer.
+- Current Tor state is process-global and does not guarantee independent onion
+  identities per subnet; I2P does not provide a general inbound listener.
+- Apps such as darkirc/fud construct substantial state from `P2pPtr` before
+  protocol registration. A registration closure alone is insufficient.
 
 ## Goals / Non-Goals
 
 **Goals:**
 
-- One overlay seed list bootstraps discovery for all subnets, forever.
-- Subnets remain ordinary `P2p` networks: their own magic bytes, hostlist,
-  refinement, datastore — joined by direct dial after discovery.
-- Subnet spawn/stop at runtime, driven by `SubnetId`.
-- Anti-linkability as a designed property, not a config flag.
-- Lilith collapses to one listener + persistent ad store.
-- First-class transient (mobile) participants: cheap lookups, no relay
-  obligations, no on-disk state beyond an optional overlay hostlist cache,
-  invisible to third parties.
-
-**Non-Goals (design-level):**
-
-- No connection multiplexing (one connection carrying multiple subnets).
-- No DHT lookup for membership; gossip only.
-- No signatures on ads; authenticity is refinement's job.
-- No changes to `src/net` semantics — additive exports only.
-- No automatic serving of every joined subnet; serving is explicit per
-  subnet.
+- Reuse ordinary `P2p` instances without changing framing, compatibility, or
+  seed-session semantics.
+- Keep all untrusted wire, persistence, queue, request, and work state bounded.
+- Make bootstrap/source attempts, join completion, serving creation, rollback,
+  recreation, and teardown explicit and testable.
+- Store/relay hints passively and validate only inside a joining subnet.
+- Add no third-party dependency.
+- State realistic protocol disclosure and persistence boundaries.
 
-## Decisions
-
-### D1. Thin overlay, not multiplexing or subnet-tagged address protocol
-
-Three alternatives were considered:
+**Non-Goals:**
 
-- **Multiplexed overlay** (one connection, virtual streams per subnet):
-  breaks the 1:1 channel↔network invariant across `session/`+`channel.rs`,
-  and mixes subnet traffic on one wire — a traffic-analysis surface that
-  violates the anonymity constraints.
-- **Subnet-tagged `GetAddrs`/`Addrs`** (one global network carrying all
-  subnets): smallest diff, but merges all hostlists into one refinement
-  state, exposing cross-subnet membership in a node's address book and
-  connection churn.
-- **Thin overlay (chosen)**: `Swarm` owns one overlay `P2p` plus a
-  `HashMap<SubnetId, SubnetEntry>`; each `SubnetEntry` owns an ordinary
-  spawned `P2p`. Overlay only bootstraps; subnet health remains
-  self-maintaining via existing refinery.
+- Multiplexing subnet traffic over overlay channels.
+- Authenticating ad authors or proving address ownership.
+- PIR, cover traffic, Sybil resistance, or global-observer resistance.
+- Automatic independent Tor/I2P provisioning.
+- Runtime reconciliation of manual, seed, or inbound session settings.
 
-### D2. `SubnetId` = `blake3(canonical descriptor)`
+## Decisions
 
-```
-descriptor := app_name || magic_bytes || version_constraint || secret?
-SubnetId   := blake3(descriptor)
+### D1. Isolated `net::swarm` module and feature
+
+```text
+src/net/swarm/
+├── mod.rs
+├── settings.rs
+├── descriptor.rs
+├── message.rs
+├── protocol.rs
+├── store.rs
+├── bootstrap.rs
+└── lifecycle.rs
 ```
 
-- Apps pin known IDs (e.g. darkirc mainnet) — a pin is a spec of the
-  descriptor fields.
-- `secret` present → unguessable ID: a non-member cannot even name the
-  subnet, giving obscurity-based access control (rendezvous-string style).
-- `version_constraint` is part of the descriptor so a subnet's version
-  policy is fixed at creation; exact pin initially, ranges deferred.
-
-Alternative: human-readable subnet names — rejected: global names leak the
-set of private subnets into gossip and invite squatting.
-
-### D3. Ad format and propagation: unsigned gossip with TTL
-
-```
-SubnetAd { subnet_id, addrs: Vec<(Url, u64)>, ttl_secs }
-```
+`src/net/mod.rs` exposes the module only with `feature = "swarm"`. The feature
+enables existing `net`, `blake3`, `kvdb-overlay`, and serialization facilities.
+Lilith and the pilot opt in explicitly. Any newly required dependency,
+`build.rs`, or proc-macro stops implementation for human supply-chain review.
 
-- Propagated by flood/gossip identical in spirit to `AddrsMessage`; the peer
-  an ad is received from is not its author → origin ambiguity.
-- Unsigned, no node identity. Poisoning is bounded by (a) refinement — ads
-  land in the target subnet's greylist and dead addrs are dropped by
-  handshake checks, and (b) per-message metering + ban policy for floods.
-- Overlay ad stores keep entries until `ttl` expiry + refinery liveness
-  checks (lilith's existing whitelist-refinery pattern, retargeted at ads).
-- Ads are re-gossiped on a slow, jittered cadence (like refinery intervals),
-  never event-triggered on subnet start — see R3.
+Nested placement gives `darkfi::net::swarm` and narrow crate-private host access
+without making orchestration core `P2p` behavior. A top-level module would need
+more public helper surface and a separate network-dependent root.
 
-Alternative: per-subnet signing keys. Gives poisoning resistance but tempts
-key reuse across subnets (linkability) and adds key management; deferred
-until refinement proves insufficient.
+### D2. Fixed overlay identity
 
-### D4. New messages and `ProtocolSwarm`, all outside `src/net` core
+All overlay instances use:
 
-```
-SubnetAd                       (gossip, unsolicited)
-GetSubnets       → Subnets             (list known subnet_ids)
-GetSubnetAddrs{subnet_id} → SubnetAddrs{subnet_id, addrs}
+```text
+app_name:     "darkfi-swarm"
+app_version:  1.0.0
+magic_bytes:  [0x78, 0x85, 0xa4, 0x2a]
 ```
 
-Defined via `impl_p2p_message!` with metering configurations and
-`MAX_BYTES` estimates in the existing style. `ProtocolSwarm` is registered
-via `ProtocolRegistry` on outbound+inbound sessions of the overlay `P2p`
-only. `GetSubnets` responses are built from local ad-store state; queries
-reveal participation in the overlay but not in any particular subnet.
-
-### D5. Serving vs joining
-
-- **Joining** (default): pull `SubnetAddrs`, seed the subnet `P2p`'s greylist
-  (via `Hosts::insert`, grey), dial. No ad is emitted.
-- **Serving** (opt-in per subnet): requires inbound addrs for that subnet —
-  one tor/i2p onion per subnet is the recommended deployment so overlay
-  observers cannot correlate a shared endpoint across subnets. Emits ads on
-  the D3 cadence.
-
-### D6. Subnet lifecycle under `Swarm`
-
-- `Swarm::join(subnet_descriptor)` → resolve via overlay → spawn subnet
-  `P2p` with per-subnet `p2p_datastore`/`hostlist` paths derived from
-  `SubnetId` under a swarm-managed directory; register app protocols onto
-  that `P2p`'s registry before `start()`.
-- `Swarm::serve(subnet_descriptor, inbound_cfg)` → join + advertise.
-- `Swarm::leave(subnet_id)` → `P2p::stop()` + deregister; ads simply expire
-  via TTL (no "leave" message — a departure broadcast would create a
-  timing-correlation surface).
-- Dynamic spawn after startup is the main new runtime pattern; apps
-  currently build all networks before `start()`. Watch item: executor
-  shutdown ordering when many subnet `P2p`s stop concurrently.
-
-### D7. Lilith becomes an overlay seed
-
-One config section (`[overlay]`: accept addrs, datastore), no per-network
-sections. Runs the overlay `P2p` with `inbound_connections` high,
-`outbound_connections` 0 (unchanged posture: no outbound dialing), plus:
-persistent ad store (ads survive restarts until TTL), refinery-based ad
-expiry, and the `spawns` RPC retargeted at overlay stats (known subnets,
-ad counts). Existing per-network sections keep working during migration
-(lilith simply spawns those nets as before, alongside the overlay).
-
-### D8. Overlay node roles: persistent vs transient, declared not negotiated
-
-Desktop daemons and lilith run on always-on machines and carry the overlay;
-mobile apps (and any short-lived client) join the overlay only to look
-subnets up and leave. The distinction is declared through the existing
-`VersionMessage.features` vector (`src/net/message.rs`), which is on the
-wire today but sent empty: a persistent node advertises
-`("swarm-store", 1)`; a transient node sends no swarm feature. Role is
-self-declared and unauthenticated — it is a hint for policy and load, never
-a privilege.
-
-|                      | persistent                     | transient                        |
-|----------------------|--------------------------------|----------------------------------|
-| typical host         | desktop daemon, lilith         | mobile app doing a lookup        |
-| inbound addrs        | typical (often onion)          | none (`inbound_connections: 0`)  |
-| ad store             | disk-backed, TTL + refinery    | none (optional in-memory cache)  |
-| gossip relay         | yes                            | only while connected (brief)     |
-| subnet serving       | per-subnet opt-in (D5)         | never                            |
-| overlay outbound     | default slots                  | minimal (1–2), query then leave  |
-| datastore/hostlist   | persisted                      | overlay hostlist cache (TSV); no ad store |
-| heartbeat tuning     | default                        | longer intervals (battery, NAT)  |
-
-- **Uniform protocol behavior**: both roles answer `GetSubnets`/
-  `GetSubnetAddrs` from whatever local state exists while connected. Role
-  changes *what state exists*, never message handling — role-specific wire
-  behavior would fingerprint peers and split the anonymity set.
-- **Transients are invisible to third parties**: no inbound addrs and no ads
-  means a transient node never appears in any hostlist or ad store. Its
-  overlay peers see only a short-lived connection — the same exposure a
-  client of today's per-network seeds has.
-- **Transient hostlist cache**: a transient node persists its overlay
-  hostlist (peer addresses only, via the existing `net::Settings.hostlist`
-  TSV — zero new machinery) so later sessions dial cached overlay peers
-  first and fall back to configured seeds only on miss/failure. The cache
-  MUST NOT record queried or joined subnets — it contains overlay peer
-  addresses and nothing else. Local-device forensics trade-off: the cache
-  proves overlay participation but not subnet membership; privacy-maximal
-  deployments disable it (also weaker against stale-entry churn, handled by
-  normal greylist refinement).
-- **Lilith is just the canonical persistent node**; any persistent daemon
-  relays ads and can cold-start others, which strengthens the R4 mitigation.
-- **Load spreading without new messages**: persistent nodes are reachable
-  addrs in the overlay's own hostlist (they advertise inbound via the normal
-  address protocol), so a transient node that wants to avoid hammering seeds
-  dials overlay peers from `GetAddrs` and simply keeps the ones whose
-  handshake carries the `swarm-store` feature. A dedicated
-  feature-filtered query can be added later inside swarm's message set if
-  wasted dials prove costly; not needed initially.
-- Mobile constraints shape defaults, not the protocol: battery (longer
-  heartbeat via `NetworkProfile`, disconnect after lookup), NAT (no inbound,
-  no hole punching required), metered data (small `GetSubnetAddrs` replies
-  bounded by metering).
-
-Alternative considered: no declared role at all (purely emergent — transients
-are just nodes that leave quickly). Rejected: without the feature bit,
-persistent nodes cannot preferentially keep slots for ad-carrying peers, and
-transients cannot find store-keeping peers without trial dialing everyone.
-
-## API Sketch
-
-Illustrative signatures — names may shift during implementation; the shape
-is what apps program against. Everything mirrors the existing `P2p` idiom:
-async constructors returning `Result`, `Arc` pointers, `StoppableTask`
-lifecycle, `net::Settings` for transport-level config.
-
-### Core types
-
-```rust
-/// Declared overlay role (D8)
-pub enum SwarmRole {
-    /// Disk-backed ad store, gossip relay, may serve subnets
-    Persistent { datastore: PathBuf },
-    /// Lookup client; no ads, no inbound, optional overlay hostlist cache
-    /// (`None` leaves no on-device overlay trace)
-    Transient { hostlist: Option<PathBuf> },
-}
+The magic is the first four bytes of
+`BLAKE3("darkfi-swarm-overlay-v1")`. Callers cannot override these fields.
+Future incompatible overlay changes follow existing major/minor rules.
 
-/// Canonical subnet descriptor (D2)
-pub struct SubnetDescriptor { /* app_name, magic_bytes, version, secret? */ }
+### D3. Manual canonical descriptor encoding
 
-impl SubnetDescriptor {
-    /// Pin a released network; shipped as constants in app code
-    pub const fn pinned(app_name: &str, magic_bytes: [u8; 4], version: &'static str) -> Self;
+`descriptor.rs` writes the exact spec bytes with checked lengths and explicit
+big-endian integers; it does not depend on general serializer stability.
+Application names are restricted to 32 UTF-8 bytes to remain valid in both
+version and verack bounds. Private construction accepts `[u8; 32]`, while
+generation fills it from `OsRng`. The golden vector is tested before any app pin
+is accepted.
 
-    /// Secret-bearing descriptor for private subnets
-    pub fn private(app_name: &str, magic_bytes: [u8; 4], version: &str, secret: &[u8]) -> Self;
+String concatenation and generic struct serialization are rejected because
+field/format ambiguity would split deployed IDs.
 
-    /// BLAKE3 of the canonical serialization
-    pub fn id(&self) -> SubnetId;
-}
+### D4. Correlated bounded wire protocol
 
-/// Handle to a joined or served subnet
-pub struct SubnetHandle { /* ... */ }
+Initial messages are:
 
-impl SubnetHandle {
-    pub fn id(&self) -> SubnetId;
-    /// The subnet's own P2p instance, for app-level messaging
-    pub fn p2p(&self) -> P2pPtr;
+```text
+SubnetAd {
+    subnet_id, visibility, ad_id: [u8; 32],
+    lifetime_secs, addrs: Vec<Url> // 1..=32
 }
-
-pub struct Swarm { /* overlay P2p + subnet registry + ad store */ }
-pub type SwarmPtr = Arc<Swarm>;
+GetSubnetAddrs { request_id: [u8; 16], subnet_id, cursor? }
+SubnetAddrs    { request_id: [u8; 16], subnet_id, addrs, next? }
+GetPublicSubnets { request_id: [u8; 16], cursor? }
+PublicSubnets    { request_id: [u8; 16], subnet_ids, next? }
+SwarmError       { request_id: [u8; 16], bounded_code }
 ```
 
-### Usage: persistent daemon (desktop, e.g. darkirc)
+Commands are fixed to `swarm.ad`, `swarm.geta`, `swarm.addrs`, `swarm.gets`,
+`swarm.subs`, and `swarm.err` respectively. Struct field order is exactly the
+order shown in `swarm-overlay`; existing DarkFi encoding is used. Visibility is
+`u8` (`0` public, `1` non-public), lifetime is `u32`, error codes are fixed
+`u8` values 0 through 3, and cursor version is one. No new serializer is added.
 
-```rust
-// Overlay settings: one seed list, forever
-let overlay = net::Settings {
-    app_name: "swarm".into(),
-    magic_bytes: OVERLAY_MAGIC,
-    seeds: OVERLAY_SEEDS.into(),
-    inbound_connections: 64,
-    ..Default::default()
-};
+Request IDs come from `OsRng`; a per-channel map permits at most 32 pending
+requests and removes entries on response, disconnect, or the default 10-second
+timeout (configurable to at most 60 seconds). Timeout is local; late, unknown,
+duplicate, or wrong-type responses are unsolicited and penalized.
 
-let role = SwarmRole::Persistent {
-    datastore: "~/.local/share/darkirc/swarm/ads".into(),
-};
-let swarm = Swarm::new(role, overlay, ex.clone()).await?;
-swarm.clone().start().await?;
+Every URL is at most 1,024 encoded bytes. Message maxima are fixed as in the
+spec: ad and address response 65,536; address/public requests 128; public
+response 16,384; error 128. Count and byte validation precede store/work.
 
-// Pinned descriptor shipped with the app (D2)
-const DARKIRC: SubnetDescriptor =
-    SubnetDescriptor::pinned("darkirc", [251, 229, 199, 181], "0.5.1");
+All swarm, version, and verack decoders are audited as attacker-input paths.
+They use checked reads and return errors for truncation/invalid structure; no
+`unwrap`, `expect`, explicit panic, unchecked slice/index, or allocation from an
+unvalidated declared size is permitted. Tests truncate valid payloads at every
+byte and feed hostile-length/arbitrary payloads under unwind and allocation
+instrumentation.
 
-// Client-only participation (default)
-let subnet = swarm.join(&DARKIRC, darkirc_protocols).await?;
+`PageCursor` is fixed 65 bytes:
 
-// Or serve it, with this subnet's own onion (D5)
-let inbound = vec![Url::parse("tor://darkirc-7.onion:9440")?];
-let subnet = swarm.serve(&DARKIRC, inbound, darkirc_protocols).await?;
-
-// App messaging rides the subnet's ordinary P2p — unchanged app code
-let _ = subnet.p2p();
+```text
+version: u8 | last_key: [u8; 32] | terminal_key: [u8; 32]
+```
 
-// Later: silent leave (D6) — no departure message, ads expire by TTL
-swarm.leave(DARKIRC.id()).await?;
+Address pages use BLAKE3 of canonical URL bytes as ordered key; public pages use
+`SubnetId`. The first page captures the greatest current live key as a terminal.
+Later pages return live keys strictly after `last_key` and no greater than that
+terminal, then advance `last_key`. Mutation may make a traversal include or omit
+records, but never invalidates a well-formed cursor, allocates a server snapshot,
+or extends traversal beyond its initial terminal. This trades snapshot
+consistency for bounded progress under adversarial mutation.
+
+Responses derive canonical item keys and require strict ascending uniqueness in
+the cursor window. `next.last_key` equals the greatest returned key; empty pages
+have no next cursor. Requesters independently derive keys, retain a bounded seen
+set, and reject within/across-page duplicates, unordered/out-of-window items,
+changed terminals, non-advancing cursors, or cursor/item disagreement.
+
+Public enumeration is disabled by default. It indexes IDs with at least one live
+normalized address record marked public and is available to all connected
+protocol-correct peers when enabled. Direct lookup ignores visibility. Because
+visibility is unsigned, an attacker can re-advertise an observed ID as public.
+
+### D5. Passive store with protected replay admission
+
+Persistent nodes use existing `kvdb-overlay` trees for address records, public
+index, seen IDs, and monotonic-epoch metadata. Address keys are
+`(SubnetId, hash(canonical_url))`. Atomic batches update records and indexes.
+Seen keys remain global by `ad_id`; their values bind the advertised `SubnetId`
+and general/local-reserve class only for quota/replay accounting. Reusing one ad
+ID under another subnet is therefore still a duplicate. No persistence API
+receives a source channel/address.
+
+Each key has one normalized record containing URL, current visibility, expiry,
+and accepting ad ID. A fresh ad atomically updates records for its included
+addresses; absent addresses remain until separately updated/expired/evicted.
+The public index contains an ID iff at least one live record is public, so a
+same-address visibility update can add/remove catalog membership while mixed
+records keep it public. Direct lookup reads all live records. Restart rebuilds
+or verifies the public index from normalized records.
+
+Defaults and hard maxima are:
+
+| Local limit | Default | Maximum |
+|---|---:|---:|
+| addresses per subnet | 256 | 1,024 |
+| total addresses | 16,384 | 65,536 |
+| general protected IDs per subnet | 256 | 1,024 |
+| protected ad IDs | 65,536 | 262,144 |
+| local-author reserve subnet partitions | 32 | 256 |
+| accepted/authored address lifetime | 7,200 s | 86,400 s |
+| replay checkpoint interval | 300 s | 600 s |
+| relay fanout | 16 | 64 |
+| bootstrap-stage timeout | 30 s | 300 s |
+| complete join timeout | 120 s | 900 s |
+
+The local expiry for each accepted address is the lesser of wire lifetime and
+the configured receive cap. Local author lifetime uses the same default and hard
+maximum. Relay preserves the original validated wire lifetime; each receiver
+applies its own cap.
+
+Address capacity evicts expired first, then earliest expiry, then lexical key.
+Seen IDs remain protected exactly through local address expiry plus 86,400
+seconds, for at most 172,800 seconds from acceptance. Expired IDs
+are removed first. Remote IDs occupy a general pool with a per-subnet quota; if
+that quota or the global general pool contains only protected IDs, the fresh ad
+is rejected before address mutation or relay. This prevents one claimed subnet
+from consuming the whole general pool, but generated subnet IDs can still cause
+distributed saturation.
+
+Authoring configuration reserves a default 32, at most 256, subnet partitions of
+256 slots each inside the global cap. Checked multiplication/subtraction derives
+reserve and nonzero general capacities. Remote ads cannot consume a partition;
+locally authored IDs use their subnet's partition until expiry. A serving
+transition atomically allocates/reuses a partition before listener/author start;
+stopping retains it until every protected local ID expires. Sequential churn may
+therefore return a typed capacity failure rather than overwrite protection. The
+last-ID expiry releases a stopped subnet's partition atomically; resumed serving
+retains it.
+
+Startup assigns persisted local IDs to partitions by distinct subnet and checks
+each partition's 256 slots separately. Persisted general IDs are checked only
+against the remaining general capacity and per-subnet quota; local IDs already
+inside reserve are not double-counted. The 256-slot partition exceeds the
+maximum IDs produced by the fixed 20-minute minimum cadence during the
+172,800-second maximum protection window. Protected IDs are never evicted early;
+replay semantics still take priority over general remote-ad availability.
+
+Reserve records necessarily identify to the local store which ephemeral ad IDs
+this process generated. They contain no peer/source address or stable author
+identity; reserve occupancy/use/failure/timing and local-origin classification
+are excluded from wire, RPC, status, metrics, and telemetry, including
+aggregate counters. This local-only authorship fact is an explicit cost of
+preventing remote saturation from blocking the process's own cadence.
+
+Ad acceptance atomically commits the global seen-ID record, pool accounting,
+address records, and public index before relay enqueue. A failed commit causes no
+mutation or relay. Restoring a database snapshot from before this commit removes
+the seen ID and can permit replay; rollback-resistant replay suppression would
+require external non-rollbackable state and is not claimed.
+
+Runtime expiry uses `Instant`. Persistence records accepted wall time, absolute
+expiry, original lifetime, and last-observed store wall time. Address records
+restore remaining time, clamped by local/original/protocol lifetime; rollback
+may expire addresses conservatively.
+
+Seen IDs instead use unsigned 64-bit seconds on a durable monotonic epoch. One
+metadata value atomically checkpoints elapsed ticks every 300 seconds by default,
+at most every 600 seconds, and on clean shutdown. Restart compares before
+subtracting: `deadline <= checkpoint` expires; otherwise checked subtraction must
+produce at most 173,400 seconds, valid remainder is clamped to 172,800, and
+checked duration conversion/`Instant::checked_add` builds the new deadline.
+Underflow, overflow, larger delta, or missing/incoherent metadata is a typed
+startup error.
+
+A checkpoint write that cannot complete before 600 seconds places persistent
+admission/authoring in fail-closed mode until checkpoint recovery or controlled
+shutdown; bounded reads may continue. This prevents new deadlines from exceeding
+the maximum validated delta.
+
+All surviving records and the new epoch replace the prior epoch atomically;
+interruption leaves the old epoch loadable. Uncheckpointed run time and downtime
+are not subtracted, so they can extend a record present in the loaded database.
+Repeated restart does not reset it to a fresh full horizon. Rollback before ID
+commit can remove the record entirely and is outside this guarantee. General and
+reserve partitions are validated independently before conversion.
+
+Transient stores use the same validation in bounded memory only. No store
+contains an active dialer. Active refinement is rejected because it cannot
+verify subnet attribution and creates scanning amplification.
+
+### D6. Validate features and full version size
+
+`net::Settings` gains a bounded local feature vector, empty by default.
+
+Feature validation rejects duplicate/overlong names, excess count, and invalid
+versions. Version permits at most 10 external addresses and 10 features;
+node ID is capped at 64 bytes, app name at 32, URL at 1,024, feature name at 32,
+and semver prerelease/build at 32 each. Before sending, protocol encodes and
+checks complete `VersionMessage` and `VerackMessage` against their maxima.
+Each overlay/subnet `P2p` receives a fresh CSPRNG node ID scoped to that instance;
+it is not persisted or reused across networks/restarts.
+
+Inbound `VersionMessage` and `VerackMessage` use manual bounded decoders that
+preserve existing field order and bytes while reading every declared
+string/vector length before reservation. They reject over-limit node/app and
+semver prerelease/build strings, external-address count/URL lengths, feature
+count, and feature-name length before `try_reserve` or allocation. Golden tests
+compare custom encoding/decoding with existing valid wire vectors.
+Compatibility rules remain unchanged.
+
+### D7. Protocol registration and independent work accounting
+
+`ProtocolSwarm` registers on `SESSION_DEFAULT` only: ordinary inbound,
+outbound, manual, and direct channels, excluding seed/refinement. Per-channel
+instances share store, work limiter, and bounded relay queue.
+
+Generic message metering is supplemented by token buckets keyed by ephemeral
+channel ID for validation/write, query/response bytes, pending/cursor work, and
+relay enqueue. Global semaphores bound durable writes, reads, and relay jobs.
+Channel accounting is removed on disconnect and never keyed by peer address.
+
+Protocol rates per channel are 32 ads, 16 direct requests, 16 direct responses,
+4 public-list requests, 4 public-list responses, and 16 errors per 10 seconds;
+work rates are 32 store writes and 32 relay enqueues per 10 seconds plus
+1,048,576 response bytes per 60 seconds. Initial local defaults/maxima are:
+
+| Resource | Default | Maximum |
+|---|---:|---:|
+| relay queue | 1,024 | 4,096 |
+| concurrent durable writes | 8 | 32 |
+| concurrent reads | 16 | 64 |
+| relay workers | 8 | 32 |
+| pages per direct join lookup | 16 | 16 |
+| pages per public enumeration | 4 | 16 |
+| candidate addresses per attempt | 64 | 256 |
+| previously compatible retries | 16 | 64 |
+| persisted compatible retry URLs/subnet | 64 | 256 |
+| local-author reserve subnet partitions | 32 | 256 |
+| active subnets | 32 | 256 |
+| concurrent lifecycle attempts | 8 | 32 |
+| shutdown deadline | 120 s | 600 s |
+| pending request timeout | 10 s | 60 s |
+| configured ordinary peers | 8 | 256 |
+| overlay bind addresses | 1 | 16 |
+| serving bind addresses/subnet | 1 | 16 |
+| serving external addresses/subnet | 1 | 32 |
+| overlay inbound channels | 64 | 256 |
+| overlay outbound channels | 8 | 64 |
+| overlay manual channels | 8 | 256 |
+| total overlay channels | 80 | 512 |
+| untrusted dial concurrency | 4 | 16 |
+| untrusted dial starts/minute | 32 | 128 |
+| DNS resolutions/join attempt | 64 | 256 |
+| dials/resolved destination/attempt | 1 | 1 |
+
+Each accepted ad queues at most one relay job and sends to no more than fanout
+eligible ordinary channels, excluding source by channel ID. Queries answer only
+their requesting channel. Overlay uses strict ban policy. This bounds work but
+does not claim Sybil resistance.
+
+### D8. Bootstrap by constructing one stage at a time
+
+Swarm owns at most one running overlay candidate. A cache record is a bounded
+pair of the original connect URL and exact resolved endpoint used by a
+successfully completed persistent-feature channel. It never comes from
+`VersionMessage.ext_send_addr`. For a transient:
+
+1. Parse the endpoint-only cache under file/count/URL/shareability bounds and
+   revalidate each stored socket without DNS.
+2. Construct overlay settings with `Settings.peers = []` and
+   `Settings.seeds = []`, then install cached targets through the explicit
+   pre-start manual-target API.
+3. Register protocol, subscribe to channels, start, and wait for a compatible
+   ordinary channel.
+4. On timeout/failure, fully stop and discard the candidate.
+5. Resolve/validate configured ordinary peers once, construct a fresh overlay
+   candidate, install those exact targets through the same pre-start API, and
+   repeat one bounded stage.
+6. Publish the successful `P2pPtr` as Swarm's active overlay only after success.
+
+No settings reload is used. Persistent nodes construct directly from configured
+ordinary topology. After an ordinary channel exposes `swarm-ad-store`, only its
+actual connect/resolved pair may enter the cache via atomic replacement. The
+peer's advertised external addresses are ignored for caching.
+
+Swarm does not configure a standard overlay hostlist/datastore for a
+privacy-maximal transient. Separately configured subnet and transport state is
+outside that overlay-cache guarantee and documented.
+
+Untrusted dial paths use a new narrow target model:
+
+```text
+ValidatedDialTarget { original_url, exact_socket, route }
+route = Direct | TrustedProxy { destination_kind }
+ManualSession::add_targets_before_start(Vec<ValidatedDialTarget>)
+ManualSession::add_target_plan_before_start({ first, second, switch_at })
+Connector::connect_validated(ValidatedDialTarget)
 ```
 
-`darkirc_protocols` is the registration closure the swarm runs against the
-subnet `P2p`'s protocol registry *before* `start()` — the same
-`registry.register(session_flags, init)` hook `register_default_protocols`
-uses internally. Apps keep registering their protocols exactly as today;
-they just do it through the closure.
-
-### Usage: transient lookup (mobile)
-
-```rust
-let overlay = net::Settings {
-    app_name: "swarm".into(),
-    magic_bytes: OVERLAY_MAGIC,
-    seeds: OVERLAY_SEEDS.into(),
-    outbound_connections: 2,
-    inbound_connections: 0,
-    ..Default::default()
-};
-
-let role = SwarmRole::Transient {
-    // Cache overlay peers between sessions so later sessions dial cached
-    // peers first and only fall back to seeds. `None` for a device free
-    // of overlay traces.
-    hostlist: Some(cache_dir.join("overlay_hostlist.tsv")),
-};
-let swarm = Swarm::new(role, overlay, ex.clone()).await?;
-
-// Lookup only: resolve addresses, no subnet participation, then disconnect
-let addrs: Vec<Url> = swarm.lookup(&FUD_CHANNEL).await?;
-
-// Or join for the duration of the app session (recommended over
-// per-lookup connections — R7 battery-vs-mixing guidance)
-let subnet = swarm.join(&FUD_CHANNEL, fud_protocols).await?;
+The pre-start method creates ordinary manual slots before `P2p::start()` and is
+not reload/reconciliation. The two-phase plan preinstalls already validated
+targets, activates only `first` at start, cancels it at a monotonic switch time,
+then activates `second`; a missing first phase activates second immediately.
+Overlay bootstrap uses the one-phase method. Fresh clearnet targets resolve once
+and cache their socket; cached targets skip DNS and revalidate their stored
+socket. In a direct route, `exact_socket` is the destination socket and the
+connector opens that exact socket without re-resolving; `original_url` supplies
+only TLS server-name identity.
+
+In a trusted-proxy route, `exact_socket` is the exact locally configured proxy
+socket, not an advertised destination. The advertised URL cannot choose or
+override it. The destination is restricted to a globally routable IP literal or
+a canonical Tor/I2P hidden-service name matching the transport. A hidden name is
+never locally DNS-resolved and is passed only inside proxy negotiation and, when
+applicable, TLS identity. Arbitrary clearnet hostnames are rejected rather than
+remotely resolved. The configured proxy socket may be loopback/private under
+local trust policy; that exception never applies to direct advertised targets.
+Missing/malformed proxy configuration is a candidate error. Production lilith
+forces direct-target local-test mode off.
+
+The full untrusted candidate pipeline—URL parse, allowlisted scheme, host/port,
+DNS result handling, address classification, target construction, proxy
+selection/negotiation, transport/TLS dial, and compatibility—is fallible and
+contains no panic, unchecked indexing, or unimplemented branch. Empty DNS sets,
+more than 16 results for one URL, malformed/missing proxy targets, unsupported
+or unaudited schemes, timeout, and cancellation return bounded errors. A
+bounded nonempty multi-address result is iterated safely, every address consumes
+the join resolution budget and is classified, and at most one allowed exact
+socket is selected for that URL. Only schemes whose adapters satisfy this rule
+are accepted from advertisements; enabled but unaudited transports are rejected
+before dialer construction.
+
+Resolution and dialing consume the D7 concurrency/rate/total budgets. A join
+attempt tries one resolved destination once. This does not prove endpoint
+ownership, but prevents local-network SSRF/DNS-rebinding and bounds public
+victim reflection.
+
+### D9. Keep overlay control channels and subnet data channels separate
+
+“Ordinary overlay session” means an inbound/outbound/manual/direct non-seed
+session on the overlay `P2p`; it does not mean every client keeps it for process
+lifetime. The channel remains bound to overlay magic/app identity, channel
+store, hosts, and `ProtocolSwarm`. Streams are never handed to another `P2p`,
+re-handshaken under subnet identity, or extended with subnet-tag multiplexing.
+
+Persistent nodes retain the overlay while storing/relaying. Serving nodes retain
+it while authoring ads. Transient settings expose two policies:
+
+```text
+SessionBound                 // default; retain overlay for application session
+ImmediateAfterOperation      // explicit reduced-privacy mode
 ```
 
-### Usage: lilith (the canonical persistent node)
+The default never reacts to lookup/join completion by disconnecting; it retains
+the overlay until an explicit `stop_overlay()` or full Swarm shutdown ends the
+application session. Immediate mode deterministically stops after every
+caller-visible lookup or join reaches a terminal outcome—success, empty result,
+error, timeout, or cancellation—but not after an internal lookup phase within a
+join. Its configuration warning states that the responder and a same-operator
+subnet server may correlate query, subnet connection, and teardown timing. In
+either mode the subnet handle owns an independent `P2p` and outlives overlay
+stop. Later discovery runs staged bootstrap again only when no active overlay
+remains.
+
+Swarm tracks overlay lifetime separately from subnet registry lifetime.
+`stop_overlay()` rejects persistent/serving duties, but for an eligible
+transient it stops only overlay tasks/channels and leaves subnet entries
+untouched. Full Swarm shutdown still stops every subnet and any active overlay.
+Transport reuse was rejected because it requires multiplexing or handoff,
+correlates overlay queries with subnet membership, mixes host/protocol state,
+and works only when the overlay peer also serves the subnet.
+
+### D10. Registry-owned lifecycle and explicit source attempts
+
+Registry states are:
+
+```text
+Initializing -> Joining -> Joined
+Initializing -> Serving
+any active state -> Stopping -> Absent
+```
 
-```rust
-let role = SwarmRole::Persistent { datastore: cfg.adstore };
-let swarm = Swarm::new(role, overlay_settings, ex).await?;
-swarm.clone().start().await?;
-// Nothing else. No per-subnet config, no join calls: ads arrive by
-// gossip, and the durable store + refinery (D7) make lilith the
-// cold-start anchor. Legacy per-network sections still spawn ordinary
-// P2p instances alongside, during migration.
+One lock owns transitions per ID; different IDs proceed concurrently. Registry
+entries retain `P2pPtr`, type-erased `Arc<dyn Any + Send + Sync>` app state, a
+shutdown hook, mode, and persistence policy. Caller handles clone the typed
+`Arc`; dropping them cannot drop registry ownership.
+
+A join attempt:
+
+1. validates descriptor and reserves `Initializing`;
+2. builds namespaced settings and `P2p` for that attempt's sources;
+3. subscribes to completed channels before start;
+4. runs the fallible initializer, stores app ownership/shutdown, and registers
+   protocols before start;
+5. loads the complete at-most-256-entry compatible retry index and performs
+   bounded `OsRng` reservoir sampling over every valid URL returned by the fresh
+   lookup's fixed-terminal traversal through completion or its 16-page cap;
+6. independently shuffles both tiers and resolves/validates them under a
+   candidate-preparation subdeadline capped at half the then-remaining overall
+   time; verified and fresh resolution each receive half that subdeadline, so
+   verified DNS/transport preparation cannot consume fresh preparation time;
+7. retains selected candidates as ephemeral `ValidatedDialTarget` values and
+   installs a two-phase verified/fresh plan through the subnet's pre-start manual-
+   target API, not its URL-only hostlist/refinery;
+8. starts and waits for a channel whose session flag is inbound, outbound, or
+   manual; at start it snapshots remaining dial time, cancels verified targets at
+   its monotonic midpoint, and activates fresh targets for the second half,
+   explicitly rejecting temporary direct, seed, and refinement notifications;
+9. confirms the channel remains an ordinary registered peer, then atomically
+   transitions to `Joined`; or
+10. on error/timeout/cancel, stops all attempt state and removes ownership.
+
+Failed pre-compatibility targets are dropped and never persisted. A successful
+ordinary channel may register its canonical peer URL in normal host state.
+Every later outbound retry/refinement resolves and validates a fresh exact
+socket under the same egress/rate rules before connection; no path may connect
+by reusing a prior validation followed by a second resolver call. Tests count
+DNS queries and inspect exact sockets across manual, outbound, retry, refine,
+and persisted-host paths. URL, store, hash, and DNS-answer order never selects
+the resolution/dial prefix: full bounded persisted state is shuffled, fresh URLs
+use reservoir sampling across the terminal traversal through completion or its
+16-page cap, and each bounded allowed DNS answer set is CSPRNG-shuffled before
+selection. The dedicated retry
+index defaults to 64 and never exceeds 256; when full, a newly compatible peer
+is usable now but does not evict an existing retry entry merely for admission.
+This reduces ordering/grinding bias but cannot force a malicious responder to
+return an honest candidate.
+
+Source policies:
+
+- overlay-only: no static seeds;
+- static-only: configured seeds, explicitly activated for that attempt;
+- combined: both sources are intentionally configured in one attempt; and
+- overlay-then-static: complete one overlay-only attempt; on failure stop it,
+  then create a fresh static-only `P2p`, rerun initializer, and remain under one
+  overall deadline.
+
+This avoids nonfunctional manual/seed reload. Rerunning initialization is an
+explicit observable cost and both attempts have independent rollback.
+
+### D11. Serving is initial configuration or controlled recreation
+
+`ServeSettings` separates:
+
+```text
+bind_addrs:       Vec<Url> // local listeners
+external_addrs:   Vec<Url> // advertised endpoints
+source_policy:    optional peer discovery after readiness
+visibility/lifetime
 ```
 
-### API-enforced invariants
-
-- `serve()` on a `Transient` swarm fails fast — the transient role has no
-  serving path (spec: swarm-overlay, node roles).
-- `join`/`serve` take a descriptor, never a raw id: a caller cannot join a
-  subnet it cannot describe, and the spawned `P2p` still enforces the
-  subnet's own magic bytes and `app_name` handshake independently of the
-  overlay.
-- Protocol registration happens only before subnet `start()` via the
-  closure — there is no window where a subnet accepts connections without
-  its app protocols attached.
-- `lookup()` answers from local state first (on a persistent node: the ad
-  store) and queries the overlay only on miss; on a transient node it may
-  reuse the session cache.
-- A transient swarm dials its cached overlay hostlist before configured
-  seeds; seeds are only the first-ever-run and fallback path.
+Create-and-serve validates persistent role and all fields, builds `P2p` with
+listeners before start, atomically allocates/reuses the subnet's local-author
+reserve partition, runs the initializer, and calls `P2p::start()`. Reserve
+exhaustion fails before initializer/listener/author activity. Success requires
+listener readiness, not an existing peer, enabling a first server. Peer
+discovery may continue afterward. Authoring is registered only after readiness
+and still waits for cadence. A newly allocated empty partition is released on
+pre-author failure; stopping retains a nonempty partition until its IDs expire.
+
+Promoting joined to serving marks it stopping, fully stops P2p/app state, and
+constructs a fresh serving-configured instance with another initializer call.
+It never calls inbound reload. Failure leaves no partial server and returns a
+typed stopped/recreation error. Namespaced persisted state may be reused.
+
+An externally provisioned onion/I2P endpoint may forward to a distinct local
+bind; the API does not conflate them. Built-in transport identity provisioning
+is not promised. Locally reused external endpoints produce a linkability
+warning.
+
+### D12. Cadence is independent of lifecycle events
+
+One author task uses a fixed version-one 30-minute base interval with
+independent uniformly sampled ±10-minute `OsRng` jitter; it is not configurable.
+Each emission uses a fresh 32-byte `OsRng` ad ID, configured lifetime capped at
+24 hours and defaulting to two hours, and only that subnet's external addresses.
+Multi-subnet emission order is shuffled with independent jitter.
+
+Initialization, listener readiness, peer connection, recreation, new overlay
+channel, and stop only mutate local author state. They never invoke immediate
+send. Stop removes future snapshots; relayed ads expire locally.
+
+### D13. Lilith uses ordinary persistent behavior
+
+Lilith `[overlay]` maps to normal persistent Swarm settings and strict policy. It
+may be inbound-only or have ordinary outbound peers; it never places overlay
+bootstrap into `Settings.seeds`.
+
+Lilith loads the durable store under caps, starts one ordinary overlay, and has
+no ad refinery/dialer or local authoring, so its reserve-partition count is zero.
+Corrupt records decode fallibly. Malformed/unverifiable seen-ID, quota/reserve,
+or epoch state fails startup; address/index state may be quarantined/rebuilt only
+when authoritative replay/accounting remains intact. Status RPC reads only
+aggregate listener/connection/capacity/address/dedup/eviction/expiry/rejection
+counters, including aggregate per-subnet-quota and epoch-checkpoint failures. It
+never reports local-author reserve occupancy/use/transition timing and never
+walks full IDs/addresses or query mappings.
+
+Legacy instances retain separate settings, paths, registry, policy, and
+shutdown.
+
+### D14. Scoped metadata threat model
+
+Protected properties are no stable author identity, no overlay-source-peer to
+subnet/authorship persistence, and isolated subnet state.
+
+Disclosed properties are requested ID to responder, connection-level query
+linkage, IDs and advertised endpoints observed/mapped by gossip/store peers,
+timing/topology evidence, public catalog, endpoint reuse, local full-ID paths
+when subnet persistence is enabled, and remote peer retention. The
+ID-to-endpoint mapping is intentional rendezvous output. Separate anonymity
+circuits may reduce linkage but are not provisioned or guaranteed by Swarm.
+
+Absolute cross-subnet unlinkability is rejected because direct query and shared
+connections make it false.
 
 ## Risks / Trade-offs
 
-- **R1 Overlay as correlation point** (new metadata surface) → per-subnet
-  addresses (D5), unsigned per-subnet ads with no node identity (D3),
-  gossip origin ambiguity (D3/D4), query design that reveals only overlay
-  participation (D4). Residual: a global adversary observing all overlay
-  traffic plus all subnet on/off timings can still correlate — documented
-  as a known limit; timing jitter is the only partial defense.
-- **R2 Ad poisoning / flood** → no signatures means ads are cheap to forge;
-  bounded by refinement liveness checks, metering thresholds, and ban
-  policy; per-subnet ad-store caps (like GREYLIST_MAX_LEN) bound memory.
-- **R3 Timing linkage of fresh serving nodes** → ads on jittered cadence
-  only, never event-driven; deployment guidance recommends pre-registered
-  onions.
-- **R4 Cold-start still depends on overlay seeds** → same trust profile as
-  today's per-network seeds, but strictly reduced: the seed sees overlay
-  participation only, never which subnets are joined (dials are direct and
-  subnet-scoped). Multiple overlay seeds can be listed, any serving node's
-  overlay connection also relays ads, and repeat sessions bootstrap from
-  the transient hostlist cache rather than seeds.
-- **R5 Private-subnet obscurity is not access control** → the secret names
-  the subnet; it does not encrypt subnet traffic. Documented; end-to-end
-  protections remain the apps' job (e.g. darkirc saltbox, event-graph RLN).
-- **R6 Tor/onion-per-subnet operational cost** → serving on clearnet tcp is
-  possible but exposes a shared endpoint; the design allows it, deployment
-  guidance should not recommend it.
-- **R7 Transient nodes are timing-fingerprintable** (connect → query →
-  leave) → while connected, transients send the same messages any node may
-  send, and third parties never observe them at all (no hostlist presence).
-  The connection-lifetime pattern itself is the residual fingerprint; cover
-  traffic is out of scope for battery-constrained devices. Documented
-  battery-vs-mixing tension: staying connected longer mixes better and costs
-  more — deployment guidance may suggest holding the overlay connection for
-  the app session rather than per lookup.
-- **R8 Transient load concentrates on overlay seeds** → transients spread
-  across persistent nodes via the overlay's own address gossip plus the
-  `swarm-store` handshake feature (D8, no new messages); after the first
-  session, the transient hostlist cache (D8) means seeds are fallback-only.
-  If concentration persists, add a feature-filtered query inside swarm's
-  message set.
-
+- **[Global metadata]** Responders/gossip peers observe IDs → Minimize fields,
+  disable public enumeration by default, document disclosure.
+- **[Unsigned poisoning]** Fresh forged ads and malicious compatible peers →
+  Bound state/work, reservoir-sample every URL in the at-most-16-page terminal
+  traversal, CSPRNG-shuffle bounded tiers, partition verified/fresh time and
+  attempts, validate compatibility, retain app authorization/static fallback,
+  make no authenticity claim.
+- **[Dedup saturation]** Protected IDs can fill capacity → Per-subnet general
+  quotas prevent one-ID monopolization, local-author partitions preserve
+  allocated local cadence, and strict global bounds reject distributed-ID floods
+  without early eviction; general remote-ad availability and serving transitions
+  can still fail under saturation/churn.
+- **[Stale addresses]** No probing retains offline hints → Two-hour receiver and
+  author defaults, 24-hour hard maximum, shuffled bounded candidates, deadlines,
+  pilot metrics, static fallback.
+- **[Client dialing/reflection]** Joiners still try attacker-selected public
+  addresses → Resolve once, reject local/reserved ranges, connect the exact
+  validated direct socket or configured trusted proxy socket, never locally
+  resolve hidden names, reject proxy DNS bypass, enforce per-destination/rate/
+  concurrency/total budgets, and retain subnet handshake checks.
+- **[Transport abort]** Existing transport constructors/dialers may assume
+  validated configuration → Reject unaudited schemes before construction and
+  require fallible no-unwind handling across every attacker-selected candidate
+  stage and accepted transport adapter.
+- **[Clock uncertainty]** Monotonic time does not survive reboot → Restore ad
+  address expiry from wall time and replay protection from a periodically saved
+  checked monotonic-epoch remainder; crash/downtime may extend records present in
+  loaded state, but restart does not reset each to the full horizon.
+- **[Storage rollback]** Same-database checkpoints cannot detect rollback before
+  an ID commit → Commit seen/address/index state before relay, test/document that
+  restoring an older snapshot can permit replay, and make no rollback-resistant
+  guarantee.
+- **[Cursor churn]** An attacker can mutate indexes between pages → Stateless
+  last/terminal-key traversal plus requester key/order/dedup validation guarantees
+  bounded forward progress while accepting non-snapshot omissions/additions.
+- **[Local reserve metadata]** Reserve records reveal local ephemeral authorship
+  to the local database → Store no peer/stable identity and exclude reserve use,
+  occupancy, subnet labels, and timing from RPC/status/telemetry.
+- **[Role Sybil]** Attackers claim persistent feature → Treat only as hint,
+  cache multiple peers, grant no privilege.
+- **[Bootstrap concentration]** Configured peers can observe/censor → Multiple
+  peers/cache, staged deadlines, static subnet fallback.
+- **[Reconstruction cost]** Failed stages rerun P2p/app initialization → Explicit
+  bounded attempts and complete cleanup; no unsupported reload semantics.
+- **[Serving downtime]** Promotion requires stop/recreate → Require serving mode
+  at initial creation where possible and return typed recreation failures.
+- **[Transport traces/endpoints]** Transport state or reused endpoints link →
+  Separate scope/config, document state, warn reuse, no provisioning claim.
+
+## Deferred Follow-up Changes
+
+These are deliberately not `swarm` completion criteria:
+
+- **Endpoint-reuse enforcement:** A later transport-identity change may reject
+  cross-subnet external-endpoint reuse by default and require an explicit
+  reduced-privacy override. This change only detects, warns, and documents reuse
+  because independent Tor/I2P identity provisioning is unresolved.
+- **Query-peer privacy budget:** A later discovery-policy change may specify
+  random single-responder lookup and bounded sequential fallback. This change
+  retains the current bounded query mechanism and explicitly discloses responder
+  and connection-level linkage; it does not promise a selection/fanout policy.
 ## Migration Plan
 
-1. Land `src/swarm` with the overlay protocol; lilith gains the overlay
-   section alongside legacy per-network sections. No app changes.
-2. Apps adopt `Swarm` optionally, keeping static seed lists as fallback;
-   pinned `SubnetId` constants added per app (values equal to existing
-   `{app_name, magic_bytes, version}` triples so current networks are
-   discoverable).
-3. Once overlay coverage is healthy, deprecate lilith's per-network
-   sections (warn, then refuse across a release boundary).
-4. Rollback: overlay is additive; apps revert to static seeds, lilith drops
-   the overlay section. Ads and `SubnetId` paths are all namespaced and
-   removable.
-
-## Open Questions
-
-- Exact `ttl_secs` default and per-subnet ad-store cap values (tune during
-  implementation against refinery intervals).
-- Whether `GetSubnets` responses should be rate-limited per peer beyond
-  generic metering (decide when metering thresholds are set).
-- Whether darkfid's consensus networks adopt `Swarm` at all, or only
-  latency-tolerant apps do (darkirc, taud, fud) — an adoption-policy
-  question, not a protocol one.
-- Whether persistent nodes should cap the share of inbound slots given to
-  transient (feature-less) overlay peers, and at what ratio — deployment
-  tuning once real transient traffic exists.
+1. Land the gated module, descriptor vectors, feature settings, full version size
+   validation, bounded messages, and in-memory tests with no app default.
+2. Add passive durable storage, protocol work limits, staged fresh-instance
+   bootstrap, and lifecycle attempts/recreation using local transports.
+3. Add lilith's optional overlay section alongside unchanged legacy sections.
+4. Run local multi-node tests for replay saturation, poisoning, no probing,
+   per-subnet quota/local reserve, monotonic-epoch restart, two-hour TTL clamp,
+   mutation-tolerant terminal cursors, CSPRNG candidate ordering, DNS rebinding/
+   local-range/reflection rejection, decoder truncation/hostile lengths, direct/
+   proxy exact routing, hidden-service no-local-DNS, full candidate-pipeline no-
+   unwind behavior, channel filtering, rollback, first-server creation,
+   recreation, and source/query-free stores.
+5. Add a default-off application pilot with overlay-then-static policy and
+   aggregate privacy-safe metrics.
+6. Run required Makefile gates, `@anon-security-review`, and human `src/net`
+   review. Broader adoption or legacy deprecation is a later change.
+
+Rollback disables pilot/overlay config and returns to static seeds and legacy
+lilith. State is namespaced and removable after shutdown; existing wire and
+subnet persistence formats are unchanged.

openspec/changes/swarm/proposal.md

@@ -1,107 +1,244 @@
-# Proposal: swarm overlay for subnet discovery
-
 ## Why
 
-Every DarkFi P2P network (darkirc, taud, darkfid testnet, fud, ...) is an
-isolated `P2p` instance separated by magic bytes and `app_name`. As a
-consequence:
-
-- Each app ships and maintains its own per-network seed lists.
-- Lilith must run one listener, datastore, hostlist, and config section per
-  network, and must be reconfigured and restarted to serve a new network.
-- There is no way to discover a subnet dynamically: creating a private
-  darkirc room, a testnet fork, or a fud channel with its own membership
-  requires baking new magic bytes into configs on every participating node.
-- Nodes that serve multiple networks expose one inbound endpoint per network,
-  and the seed infrastructure becomes the de-facto directory of all networks.
-
-We introduce a thin gossip overlay ("swarm") for subnet rendezvous: a single
-overlay network through which nodes learn which subnets exist and which peers
-serve them, then join each subnet directly as an ordinary `P2p` network. One
-overlay seed list serves all networks forever; lilith is demoted from an
-N-network zookeeper to a single overlay seed with a persistent ad store.
-
-Privacy is a primary driver, not an afterthought: the overlay must not become
-a cross-subnet correlation point (hard invariant: no peer-address/metadata
-leakage). Ads are per-subnet, carry per-subnet addresses, and propagate by
-gossip so origins stay ambiguous, mirroring the properties of the existing
-address protocol.
+DarkFi applications currently need independently configured bootstrap peers and
+seed lists for every subnet. This makes dynamic subnet discovery, first-server
+startup, and shared operational deployment unnecessarily difficult.
+
+Introduce one bounded rendezvous overlay that discovers subnet endpoints while
+keeping each subnet in its own ordinary `P2p` instance. Static seeds remain an
+explicit fallback and unchanged default during the pilot.
+
+The overlay is a metadata and availability dependency, not an anonymity,
+authentication, authorization, or access-control mechanism. It does not remove
+per-subnet handshakes, application authorization, endpoint poisoning, Sybil
+risk, or availability failure, and it cannot hide a lookup from its responder.
 
 ## What Changes
 
-- Add a new `src/swarm` subsystem: a `Swarm` that owns one overlay `P2p`
-  instance (fixed `app_name`, one magic-bytes constant) and dynamically
-  manages subnet `P2p` instances.
-- Define `SubnetId = blake3(canonical descriptor)`, where the descriptor
-  binds `{app_name, magic_bytes, version, optional secret}`. Known subnets
-  are pinned in app code; the optional secret yields unguessable private
-  subnets (obscurity-based access control).
-- New overlay wire messages (`SubnetAd`, `GetSubnets`/`Subnets`,
-  `GetSubnetAddrs`/`SubnetAddrs`) defined via the existing
-  `impl_p2p_message!` macro with metering configurations, gossiped by a new
-  `ProtocolSwarm` registered through the existing `ProtocolRegistry`.
-- Ads are unsigned, `{subnet_id, addrs, ttl}`, propagated by gossip/flood
-  (like `AddrsMessage`); poisoning is handled by the existing refinement and
-  ban machinery, not by signatures. No stable node key crosses subnets.
-- Subnet membership discovered on the overlay is seeded into each subnet's
-  greylist; per-subnet hostlists, refinement, and datastores remain unchanged
-  and self-maintaining.
-- Serving a subnet (advertising inbound addrs, e.g. one onion per subnet) is
-  opt-in per subnet; joining (query + dial) is the default.
-- Overlay node roles: **persistent** swarm nodes (desktop daemons, lilith)
-  keep disk-backed ad stores and relay gossip; **transient** swarm nodes
-  (e.g. mobile apps doing a lookup) run inbound-free overlay sessions,
-  emit no ads, and persist at most an overlay hostlist cache so repeat
-  sessions dial cached peers instead of always contacting seeds. The role
-  is declared via the existing `VersionMessage` features vector — no new
-  handshake semantics.
-- Lilith is reduced to a single overlay listener plus a persistent,
-  TTL-bounded ad store for cold-start; its config collapses from N network
-  sections to one overlay section and new subnets are learned without
-  operator action.
-- Minimal, non-breaking changes to `src/net` (exports/helpers only); no
-  changes to channel framing, magic-byte gating, version handshake
-  semantics, or the host ACL.
-
-Non-goals: multiplexing subnets over a single connection (rejected for
-traffic-analysis and architectural reasons); replacing gossip with structured
-DHT lookup for membership; subnet-scoped content DHT keys (possible later
-rider on the same overlay); changes to consensus, contracts, or ZK.
+- Add an isolated higher-level `src/net/swarm/` subsystem, exposed as
+  `darkfi::net::swarm` behind a dedicated `swarm` feature. `Swarm` owns one
+  overlay `P2p` instance at a time and manages independent subnet `P2p`
+  instances. Existing BLAKE3, `kvdb-overlay`, serialization, and networking
+  facilities are activated explicitly; no third-party dependency is added
+  silently.
+
+- Define a versioned, collision-resistant `SubnetId`. Its normative encoding is
+  domain-separated, length-delimited, byte-exact, and covered by a golden vector.
+  It binds application name, magic bytes, and the major/minor compatibility
+  boundary enforced by the current handshake. Patch/prerelease/build metadata do
+  not affect the ID. A private descriptor may include a high-entropy 32-byte
+  `OsRng` secret, making an unobserved ID difficult to guess without providing
+  authentication, encryption, authorization, or confidentiality after
+  disclosure.
+
+- Bootstrap through ordinary non-seed overlay sessions, never the existing
+  connect-exchange-close `SESSION_SEED` path. A transient first constructs a
+  `P2p` from its bounded endpoint-only cache. The cache records only the exact
+  connect URL/resolved endpoint pair used by a completed ordinary channel that
+  advertised the persistent feature; advertised external addresses are not
+  cache authority. On timeout/failure, that instance is fully stopped and a
+  fresh configured-peer instance is built. No runtime session reload is assumed.
+
+- Define bounded overlay advertisements, correlated direct lookup, optional
+  public enumeration, and bounded errors. Requests use fresh 16-byte IDs echoed
+  by responses. Ads contain a fresh 32-byte ad ID, bounded lifetime, visibility,
+  and 1..=32 addresses, with no sender timestamp or author identity. Every
+  message, URL, page, cursor, pending map, queue, candidate set, and work setting
+  has a fixed protocol or safe local maximum.
+
+- Use a stateless terminal-key cursor. Index mutation can add/omit records
+  relative to the first page but cannot invalidate the cursor or force restart
+  loops. Requesters validate canonical ordering, uniqueness, windows, terminal
+  stability, and advancement. Public enumeration is disabled by default and
+  requires explicit enablement. Direct lookup ignores visibility; unsigned
+  visibility cannot prevent an attacker from relabeling an observed ID public.
+
+- Treat advertisements as untrusted routing hints. Address/replay storage is
+  TTL-, cap-, and work-bounded. Replaying a retained global ad ID neither
+  refreshes it nor relays it again, including reuse under another subnet.
+  Protected IDs are never evicted early.
+
+- Partition replay capacity into a general pool with per-subnet quotas and a
+  fixed local-author reserve. Reserve subnet partitions default to 32, max 256,
+  and hold exactly 256 IDs each inside the global cap. Remote ads cannot consume
+  them. Serving allocates a partition before network activity; stopped subnets
+  retain nonempty partitions until expiry. General and reserve state are checked
+  separately at startup without double-counting. Reserve occupancy, use,
+  failure, and timing remain absent from wire, RPC, status, metrics, and
+  telemetry.
+
+- Persist replay deadlines on a monotonic epoch with a 300-second default and
+  600-second maximum checkpoint interval. Restart uses checked `u64` arithmetic,
+  expires `deadline <= checkpoint`, rejects incoherent deltas, and atomically
+  installs a new epoch. Checkpoint failure at 600 seconds fails fresh admission
+  and authoring closed while bounded reads may continue.
+
+- Atomically commit seen-ID, quota/reserve, address, and public-index state before
+  relay enqueue. Crash/downtime may conservatively extend records present in the
+  loaded database. Restoring a database snapshot from before commit can lose an
+  ID and permit replay; rollback-resistant storage is not claimed.
+
+- Clamp accepted and locally authored address lifetime to a two-hour default and
+  24-hour hard maximum. Relay preserves the validated wire lifetime and each
+  receiver applies its own cap. Passive expiry, bounded candidates, deadlines,
+  pilot metrics, and static fallback mitigate stale hints without probing.
+
+- Perform no advertisement liveness dialing from overlay stores or Lilith.
+  `SubnetId` is one-way, so a store cannot perform the subnet handshake, and a
+  transport-only probe would create attacker-controlled scanning/reflection.
+  Only a descriptor-holding joining subnet validates returned addresses through
+  ordinary magic, application-name, and major/minor compatibility. Compatibility
+  and possession of a private ID still grant no application authorization.
+
+- Prevent storage/key order from selecting the resolution/dial prefix. A join
+  loads at most 256 bounded previously compatible URLs and reservoir-samples all
+  valid URLs returned through terminal completion or the fixed 16-page cap.
+  Persisted/fresh tiers and allowed DNS answers are independently shuffled with
+  `OsRng`. Candidate preparation time is split between tiers; verified dialing
+  ends at a monotonic midpoint and consumes at most half the attempts, preserving
+  fresh time/capacity. This reduces ordering bias, not Sybil or authenticity risk.
+
+- Preserve each validated candidate as a typed original-URL/exact-socket target
+  through compatibility. It does not enter URL-only host/refinery state first.
+  Failed candidates are dropped; a compatible peer may enter bounded ordinary
+  retry state. Every later reconnect/refinement resolves, validates, budgets, and
+  dials a fresh exact target.
+
+- Apply exact-target egress policy before every untrusted dial. Fresh clearnet
+  names resolve once; cached targets revalidate their stored socket without DNS.
+  Direct loopback/private/shared/link-local/multicast/unspecified/documentation/
+  benchmarking/reserved ranges are rejected outside explicit local-test mode,
+  which production Lilith cannot enable. Connection uses the exact socket and
+  retains the hostname only for TLS identity.
+
+- For Tor/I2P, the exact socket is the trusted locally configured proxy; an ad
+  cannot choose or override it. Canonical hidden-service names are never locally
+  resolved and are passed only in matching proxy/TLS handling. Arbitrary
+  clearnet remote-proxy DNS is rejected. DNS answer count, resolution/dial rate,
+  destination repetition, concurrency, and total candidate work are bounded.
+
+- Standardize the persistent role feature as `("swarm-ad-store", 1)`.
+  Persistent nodes retain bounded durable state and relay ads. Transients accept
+  no inbound overlay connections, author no ads, and persist no swarm state
+  except their bounded successful-endpoint cache. The self-declared feature
+  grants no validation, metering, query, or storage privilege.
+
+- Keep overlay and subnet channels structurally separate. Overlay channels remain
+  owned by the fixed overlay identity and are never transferred, re-handshaken,
+  multiplexed, or reused for subnet data. Subnet `P2p` lifetime remains
+  independent.
+
+- Persistent store/gossip and serving-advertisement duties retain the overlay.
+  The transient default retains it until explicit stop or Swarm shutdown and
+  never disconnects because lookup/join completed. Explicit reduced-privacy mode
+  tears down after every caller-visible terminal result—but not an internal join
+  lookup—and documents timing correlation. Later discovery reconstructs an
+  overlay only when none remains.
+
+- Make metadata disclosure explicit. Ads contain no stable node/signing ID,
+  author, relay provenance, or intentional cross-subnet field. Nevertheless,
+  responders see requested IDs; requests on one channel are linkable; gossip
+  peers observe ID-to-endpoint mappings; timing/topology can suggest origin; and
+  endpoint reuse links subnets. The prohibited durable mapping is source peer to
+  subnet/authorship, not the rendezvous mapping itself. No PIR or global-observer
+  guarantee is made.
+
+- Generate `VersionMessage.node_id` independently with a CSPRNG for every
+  overlay/subnet `P2p` instance and process lifetime; do not persist or reuse it
+  across networks. Explicit endpoint reuse remains linkable and warning-only.
+
+- Plumb a validated local feature vector into `VersionMessage` without changing
+  compatibility or valid wire bytes. Complete outgoing version/verack messages
+  are size-checked. Inbound decoding checks every variable length/count before
+  reservation/allocation, including strings, semver metadata, addresses, URLs,
+  and features. Golden tests preserve existing encoding.
+
+- Audit every swarm, version, verack, and advertised-candidate path as bounded,
+  fallible attacker input. URL parsing, DNS, address classification, proxy
+  selection/negotiation, transport/TLS dialing, and compatibility contain no
+  `unwrap`, `expect`, panic, unchecked indexing/slicing, unvalidated allocation,
+  or reachable unimplemented branch. Unsupported/unaudited schemes fail before
+  dialer construction.
+
+- Implement registry-owned subnet attempts. Initialization constructs app state,
+  protocols, and a shutdown hook before network activity. Join succeeds only on
+  a compatible ordinary inbound/outbound/manual channel—not seed, direct, or
+  refinement—and failure/timeout/cancellation fully rolls back.
+
+- Support overlay-only, static-only, combined, and overlay-then-static sources.
+  Overlay-then-static fully stops the first attempt, constructs a fresh
+  static-configured `P2p`, reruns initialization, and remains under one overall
+  deadline; it never depends on session reload.
+
+- Select serving before initial start. A first server can initialize and bind
+  without an existing peer. Bind and external addresses remain distinct.
+  Promoting a joined instance uses serialized full stop/recreate and repeated
+  initialization, never inbound reload. Serving is persistent-role only.
+
+- Author ads only after listener readiness on a fixed 30-minute cadence with
+  independent uniform ±10-minute `OsRng` jitter. Lifecycle events never trigger
+  immediate emission; stop sends no withdrawal. Externally provisioned Tor/I2P
+  endpoints are supported without claiming automatic identity provisioning;
+  reused endpoints receive an explicit linkability warning.
+
+- Run Lilith as one optional ordinary persistent overlay peer with strict bounds,
+  passive durable storage, production egress policy, aggregate-only status, and
+  zero local-author reserve partitions. It has no ad refinery/dialer. Legacy
+  network sections remain isolated and supported during migration.
+
+- Pilot one application behind a default-off flag with unchanged static fallback.
+  Collect bounded aggregate lookup, stale/poisoning, remote store-pressure,
+  checkpoint, pagination, and fallback metrics. No metric may contain peer/query/
+  private-ID/local-author data.
+
+Deferred follow-up changes, intentionally not implemented here:
+
+- fail-closed cross-subnet endpoint-reuse policy;
+- query-peer selection/privacy budgets.
+
+Non-goals: connection multiplexing; authenticated ads; PIR, cover traffic,
+global-observer or Sybil resistance; using private IDs as access control; active
+ad scanning; automatic hidden-service provisioning; rollback-resistant storage;
+or consensus, contract, ZK, canonical serialization, host-ACL, framing, magic,
+compatibility, or seed-session changes.
 
 ## Capabilities
 
 ### New Capabilities
 
-- `swarm-overlay`: the overlay rendezvous protocol — SubnetId derivation,
-  advertisement format, gossip propagation, query handling, ad-store TTL and
-  bounds, metering, node roles (persistent vs transient and their
-  obligations), and the anti-linkability requirements (per-subnet addresses,
-  origin ambiguity, no cross-subnet identity).
-- `subnet-lifecycle`: dynamic subnet management from the application
-  perspective — resolving a SubnetId to reachable peers, spawning/stopping
-  subnet `P2p` instances at runtime, seeding subnet greylists from overlay
-  ads, and the serving-vs-joining modes with their advertisement obligations.
-- `lilith-overlay-seed`: lilith redeployed as a single overlay seed — one
-  listener, persistent ad store with refinery-based expiry, single-section
-  config, and dynamic learning of new subnets.
+- `swarm-overlay`: versioned IDs; bounded correlated messages/pagination;
+  passive replay storage; roles; ordinary-peer bootstrap; exact-target dialing;
+  resource limits; and privacy disclosure.
+- `subnet-lifecycle`: descriptor resolution; registry-owned app state; ordinary
+  join completion; bounded source fallback; isolated state; serving/recreation;
+  and teardown.
+- `lilith-overlay-seed`: optional persistent ordinary overlay peer with bounded
+  passive durable state, aggregate status, strict privacy/resource policy, and
+  legacy isolation.
 
 ### Modified Capabilities
 
-None. There are no existing specs under `openspec/specs/` to modify; `src/net`
-core behavior is deliberately left unchanged.
+None. No existing specifications under `openspec/specs/` are modified.
 
 ## Impact
 
-- **New code**: `src/swarm/` (overlay `P2p` ownership, `ProtocolSwarm`, ad
-  store, subnet registry/lifecycle) and its wire message definitions.
-- **`src/net`**: minor only — possibly exporting small `hosts` helpers and a
-  path-derivation helper for per-subnet datastores; no semantic changes to
-  sessions, hosts, channel, or transport layers.
-- **`bin/lilith`**: config format collapses to one overlay section; existing
-  per-network sections deprecated on a migration path.
-- **Apps** (`bin/darkirc`, `bin/tau`, `bin/darkfid`, `bin/fud`): opt-in
-  adoption — construct `Swarm` instead of (or in front of) individual `P2p`
-  instances; static seed lists remain as fallback during transition.
-- **Security review focus**: the overlay is a new metadata surface; the
-  anti-linkability requirements in `swarm-overlay` are binding and must pass
-  the anon-security-review gate before apply.
+- **New code:** `src/net/swarm/`, gated by feature `swarm`.
+- **Existing `src/net`:** module exposure; validated version-feature plumbing;
+  bounded version/verack decoding and outgoing size checks; exact validated
+  target dialing and bounded one-/two-phase pre-start plans; narrow channel/host
+  helpers; and fallible transport adapters or pre-construction rejection.
+  Existing valid wire bytes, framing, magic, compatibility, and manual/seed/
+  inbound reload semantics remain unchanged.
+- **Feature/dependencies:** existing optional facilities are activated explicitly.
+  Any new dependency/source, `build.rs`, or proc-macro requires separate human
+  supply-chain review.
+- **Lilith:** one optional overlay section and aggregate status; legacy sections
+  remain available.
+- **Pilot:** opt-in Swarm construction and descriptor pinning with static fallback
+  unchanged by default.
+- **Operational cost:** one additional bounded overlay connection set/store plus
+  independent per-subnet listeners.
+- **Security review:** this is a privacy-sensitive shared-network change.
+  Transport/privacy, persistence, descriptor hashing, and candidate fairness
+  require focused human review. `@anon-security-review`, CI, and final human
+  patch review remain mandatory.

openspec/changes/swarm/specs/lilith-overlay-seed/spec.md

@@ -1,84 +1,245 @@
 ## Purpose
 
-Defines lilith redeployed as a single persistent overlay seed: one overlay
-listener and datastore, a durable advertisement store with refinery-based
-expiry, dynamic learning of new subnets without operator action, and
-continued support for legacy per-network sections during migration.
+Defines lilith as a persistent ordinary overlay peer providing bounded subnet
+rendezvous through one listener and durable store without joining, probing, or
+serving advertised subnets.
 
 ## ADDED Requirements
 
-### Requirement: Single overlay seed configuration
+### Requirement: Single ordinary overlay configuration
 
-Lilith SHALL support an overlay configuration section that starts one
-overlay network instance with its own accept addresses, datastore, and
-hostlist paths. When the overlay section is present, lilith participates in
-the overlay as a persistent node: high inbound slot allowance, no outbound
-slot requirement, gossip relay, and the swarm-ad-store handshake feature.
-Per-network configuration sections SHALL NOT be required for the overlay to
-operate, and operating the overlay SHALL NOT require one listener, datastore
-section, or magic-bytes constant per served subnet.
+Lilith SHALL support one overlay section containing accept and external
+addresses, ordinary bootstrap peers, connection policy, datastore/hostlist/ad
+store paths, public-enumeration policy, and finite resource limits within
+`swarm-overlay` maxima. It SHALL start one ordinary
+overlay `P2p` advertising `("swarm-ad-store", 1)`.
 
-#### Scenario: Fresh subnet served without reconfiguration
-
-- **WHEN** participants begin advertising a subnet unknown to a running
-  lilith overlay seed
-- **THEN** lilith's advertisement store learns and serves the subnet with
-  no operator action and no restart
+Bootstrap addresses MUST be ordinary peers, not seed sessions. Lilith MAY run
+inbound-only with zero ordinary outbound slots as an operator topology choice.
+Production lilith overlay configuration MUST disable local-test egress mode;
+every configured outbound target SHALL use the exact resolved-target validation
+and dial budgets from `swarm-overlay`.
+It SHALL NOT require a per-subnet listener, descriptor, magic, datastore, or
+network instance to store ads and answer lookup. Learning an ID MUST NOT make
+lilith join or serve it.
 
 #### Scenario: Overlay-only deployment
 
-- **WHEN** lilith is configured with only the overlay section
-- **THEN** it starts, accepts overlay connections, and serves subnet
-  discovery
+- **WHEN** a valid overlay section exists without legacy sections
+- **THEN** lilith starts one persistent ordinary overlay peer
+
+#### Scenario: Inbound-only topology
+
+- **WHEN** an overlay listener has zero outbound slots
+- **THEN** inbound ordinary peers can maintain sessions, submit ads, and query
+
+#### Scenario: Invalid configuration
+
+- **WHEN** any path, address, privacy policy, or resource limit is invalid
+- **THEN** startup fails before overlay activity without panic or unbounded
+  fallback
+
+### Requirement: Unknown subnets need no reconfiguration
+
+Lilith SHALL validate, store, relay, and answer valid ads for previously unknown
+IDs within all message/store/work bounds and without descriptors. Learning new
+IDs MUST NOT create subnet listeners or app protocols.
+
+#### Scenario: Fresh subnet ad
+
+- **WHEN** a valid unknown-ID ad arrives
+- **THEN** it becomes available to bounded lookup without restart or operator
+  action
+
+#### Scenario: Rendezvous-only learning
+
+- **WHEN** many subnet IDs are learned
+- **THEN** lilith still runs one overlay and no subnet instance
+
+### Requirement: Durable state preserves bounded replay and expiry
+
+Lilith SHALL persist normalized per-subnet address records with their current
+visibility/expiry, local expiry metadata, stateless ordered indexes, protected
+replay IDs, and monotonic-epoch checkpoint metadata. It SHALL enforce configured
+caps no greater than 1,024 addresses per subnet, 65,536 total addresses, 1,024
+general-pool protected IDs per subnet, and 262,144 protected IDs globally.
+Accepted address lifetime SHALL be clamped to the configured local receive cap,
+defaulting to 7,200 seconds and never exceeding 86,400 seconds. Restart MUST
+restore only remaining ad-address lifetime and MUST NOT revive expired entries.
+
+At least every 600 seconds and on clean shutdown, lilith SHALL atomically persist
+the current monotonic epoch checkpoint, defaulting to 300 seconds. Restart SHALL
+compare unsigned 64-bit deadline/checkpoint ticks before checked subtraction:
+expired/equal records are removed; valid deltas over 173,400 seconds, arithmetic
+failure, or failed checked `Instant` addition are typed startup errors; remaining
+duration is capped at 172,800 seconds. Surviving records and the new epoch SHALL
+be replaced atomically. It MUST NOT reset every record present in the loaded
+database to a fresh horizon or shorten it. Crash/downtime MAY extend that
+remainder. Rollback before an ID's commit can remove replay state and permit
+replay; lilith makes no non-rollbackable guarantee.
+
+Checkpoint failure reaching the 600-second maximum SHALL make lilith reject
+fresh ads until checkpoint recovery or controlled overlay shutdown; existing
+bounded lookup MAY continue.
+
+Protected IDs MUST NOT be evicted early. Lilith authors no subnet ads and SHALL
+configure zero local-author reserve partitions. If a subnet quota or its global
+general pool has no expired slot, the applicable fresh remote ad SHALL be
+rejected rather than weakening replay protection.
+Persistence MUST NOT contain ad sources, queriers, query history, source-peer/
+subnet associations, or private secrets. Replay ID-to-advertised-subnet binding
+solely for quota accounting is allowed and MUST NOT contain a peer/source.
+Decoding malformed/truncated records SHALL be fallible and bounded. Malformed or
+unverifiable seen-ID, quota/reserve, or epoch state SHALL fail overlay startup.
+Address records MAY be quarantined and the public index rebuilt only when replay
+and accounting state remains intact. Capacity failure SHALL not evict/reset
+protected state.
+
+Seen-ID/quota state, address/index mutation, and acceptance SHALL commit
+atomically before relay enqueue. Commit failure performs neither mutation nor
+relay.
+
+#### Scenario: Restart preserves remaining lifetime
+
+- **WHEN** lilith restarts before expiry without wall-clock rollback
+- **THEN** only remaining lifetime is restored
+
+#### Scenario: Restart does not revive expiry
+
+- **WHEN** restart occurs after expiry
+- **THEN** the ad is not returned
+
+#### Scenario: Protected set is full
+
+- **WHEN** all dedup slots are protected and a fresh ad arrives
+- **THEN** lilith rejects it without evicting protected replay state
+
+#### Scenario: Restart restores seen-ID remainder
+
+- **WHEN** lilith loads valid persisted seen IDs after any clock movement
+- **THEN** each receives its conservative checkpointed remainder on a new
+  monotonic epoch rather than a fresh full horizon
+
+#### Scenario: One subnet reaches its replay quota
+
+- **WHEN** one claimed subnet consumes all of its unexpired general-pool slots
+- **THEN** lilith rejects another fresh ad for it without consuming other
+  subnet capacity
+
+#### Scenario: Store rollback loses accepted ID
+
+- **WHEN** lilith loads a database snapshot from before an ad's atomic commit
+- **THEN** replay may be accepted again and no rollback-resistant claim is made
+
+#### Scenario: Reduced dedup cap blocks startup
+
+- **WHEN** configured capacity cannot hold valid persisted seen IDs
+- **THEN** overlay startup fails without evicting them
+
+#### Scenario: Corrupt record
+
+- **WHEN** durable bytes are malformed or truncated
+- **THEN** authoritative replay/accounting corruption fails startup, while only
+  non-authoritative address/index state may be quarantined/rebuilt fallibly
+
+### Requirement: Lilith performs no advertisement liveness dialing
+
+Lilith MUST NOT connect to an advertised subnet address due to accepting,
+storing, relaying, expiring, or reporting an ad. It SHALL expire through local
+TTL, replay admission, and capacity policy only. It MUST NOT describe transport
+reachability as subnet compatibility; only a descriptor-holding joining app can
+perform the subnet handshake.
+
+#### Scenario: Attacker-selected address
+
+- **WHEN** an accepted ad contains an attacker-selected shareable address
+- **THEN** no lilith ad-store task connects to it
+
+#### Scenario: Passive expiry
+
+- **WHEN** an ad expires
+- **THEN** it stops being returned without a probe
+
+### Requirement: Cold-start lookup uses an ordinary correlated session
+
+A client SHALL be able to configure lilith as an ordinary peer, establish a
+long-lived session, and issue request-ID-correlated bounded lookups. Lilith MAY
+return locally unexpired addresses whose servers are offline; responses are
+untrusted hints, not reachability proof.
+
+#### Scenario: Fresh client queries directly
+
+- **WHEN** a no-cache client connects to lilith as an ordinary peer
+- **THEN** it can query without a seed-session exchange
+
+#### Scenario: Stored address is stale
+
+- **WHEN** a returned unexpired address is offline
+- **THEN** the joining app handles failure within its deadline and lilith makes
+  no availability guarantee
+
+### Requirement: Strict bounded resource policy
+
+Lilith's overlay SHALL enforce all protocol message, URL, page, pending request,
+store, dedup, work, relay-fanout, and configured-safe maxima with strict ban
+policy. Small requests MUST NOT induce unbounded response, cursor, write,
+relay, allocation, or connection work. Legacy policy MUST NOT weaken overlay
+policy.
+
+#### Scenario: Query flood
+
+- **WHEN** one channel exceeds message or work budgets
+- **THEN** strict penalties apply while state remains bounded
+
+#### Scenario: Ad flood
+
+- **WHEN** fresh ads reach address or protected-ID caps
+- **THEN** deterministic rejection/eviction rules preserve every bound and
+  replay guarantee
 
-### Requirement: Durable advertisement store
+#### Scenario: Legacy relaxed policy
 
-Lilith's overlay advertisement store SHALL persist to disk and survive
-restarts. Entries SHALL be served after restart until their TTL expires
-absent re-confirmation. Store persistence MUST NOT record any information
-about transient queriers (no logging of query sources into the store).
+- **WHEN** a legacy instance is relaxed
+- **THEN** overlay strict policy remains independent
 
-#### Scenario: Restart preserves cold-start service
+### Requirement: Aggregate-only overlay status
 
-- **WHEN** lilith restarts while a subnet's serving peers are offline
-- **THEN** the persisted advertisements are still available to cold-start
-  nodes that query afterward, within TTL bounds
+Status RPC SHALL expose listener state, aggregate connection counts, configured
+capacities, current address/dedup counts, evictions, rejections, and expiries.
+It MUST NOT expose peer or advertised addresses, queried/private subnet IDs,
+ad sources, per-peer counters, query history, or source/query associations.
+Public enumeration, if enabled, remains the bounded overlay protocol.
 
-### Requirement: Advertisement refinery
+#### Scenario: Operator reads health
 
-Lilith SHALL run a periodic refinery over its advertisement store,
-verifying reachability of advertised addresses; entries failing checks
-SHALL be downgraded or dropped. The refinery SHALL be rate-limited so it
-does not dial a burst of addresses simultaneously.
+- **WHEN** status is requested
+- **THEN** aggregate health/capacity/count metrics are returned without peer or
+  subnet-query identifiers
 
-#### Scenario: Dead advertisement expires early
+#### Scenario: Querier data is absent
 
-- **WHEN** an advertised address fails refinery liveness checks before its
-  TTL would expire
-- **THEN** lilith stops serving that address before TTL expiry
+- **WHEN** peers query different IDs
+- **THEN** status and durable metrics cannot identify which peer queried which
+  ID
 
-### Requirement: Legacy per-network sections honored during migration
+### Requirement: Legacy sections remain isolated during migration
 
-While the migration period is in effect, lilith SHALL keep accepting and
-spawning per-network sections as independent network instances alongside
-the overlay, preserving current seed behavior for apps that have not
-adopted the swarm. Deprecation of per-network sections, when it comes,
-SHALL be staged (warning first, refusal at a later release boundary).
+Lilith SHALL continue accepting valid legacy sections as independent `P2p`
+instances. Overlay and legacy settings, listeners, paths, protocol registries,
+policies, failures, and shutdown handles MUST remain isolated. Legacy refusal
+requires a later release-boundary plan.
 
-#### Scenario: Mixed config runs both
+#### Scenario: Mixed configuration
 
-- **WHEN** lilith is configured with both the overlay section and legacy
-  per-network sections
-- **THEN** the overlay seed and the legacy per-network instances all run
+- **WHEN** overlay and legacy sections coexist
+- **THEN** each runs with independent state and policy
 
-### Requirement: RPC reporting of overlay state
+#### Scenario: Overlay failure
 
-Lilith's JSON-RPC SHALL expose overlay seed status: listener health,
-participating subnets, and advertisement store statistics. The reported
-data MUST NOT include addresses or identifiers of transient queriers.
+- **WHEN** overlay startup or runtime fails
+- **THEN** failure is reported without silently changing legacy configuration
 
-#### Scenario: Operator inspects seed
+#### Scenario: Legacy-only deployment
 
-- **WHEN** an operator calls the lilith status RPC
-- **THEN** listener health, known subnet identifiers, and advertisement
-  counts are returned, with no record of which peers queried which subnets
+- **WHEN** currently valid legacy sections exist without overlay
+- **THEN** they remain accepted during migration

openspec/changes/swarm/specs/subnet-lifecycle/spec.md

@@ -1,138 +1,394 @@
 ## Purpose
 
-Defines how an application uses the swarm overlay to manage subnets
-dynamically: joining a subnet by resolving its identifier to reachable
-peers, serving a subnet with advertisement obligations, leaving a subnet,
-and keeping per-subnet state isolated — at runtime, without per-subnet seed
-configuration.
+Defines bounded, failure-safe application lifecycle behavior for resolving,
+initializing, joining, creating, serving, recreating, and stopping isolated
+swarm-managed subnets.
 
 ## ADDED Requirements
 
-### Requirement: Subnet join via overlay resolution
+### Requirement: Descriptor-based resolution uses untrusted candidates
+
+An application SHALL request a subnet using a valid descriptor, not a raw ID.
+The swarm SHALL derive the normative ID and request only addresses for it.
+Overlay results SHALL enter only an ephemeral typed unverified-target set for
+that subnet, preserving original URL and validated socket until compatibility.
+They MUST NOT enter URL-only persistent host/refinery state before success.
+Overlay and subnet connections MUST remain separate. Every subnet peer MUST
+independently pass magic-byte, application-name, and major/minor checks.
+An overlay channel MUST NOT be reused, transferred, re-handshaken, or
+multiplexed for subnet traffic. Default transient policy retains the overlay for
+the application session; immediate post-lookup/join stop requires explicit
+reduced-privacy policy and occurs after every caller-visible terminal outcome,
+not an internal lookup phase within join. Any overlay stop MUST NOT stop the
+independent subnet.
+
+Resolution and connection phases SHALL have finite deadlines. A complete join
+deadline MUST be configurable and no greater than 900 seconds.
+Every untrusted candidate SHALL pass the overlay's resolution-time egress
+policy before connection and consume the configured resolution/dial concurrency,
+rate, per-destination, and total-candidate budgets. Rejected destinations SHALL
+remain fallible candidate failures and MUST NOT trigger a second resolution in
+the connector. Direct candidates SHALL retain the exact validated destination
+socket. Tor/I2P candidates SHALL retain the exact trusted locally configured
+proxy socket plus canonical hidden-service destination, with no local DNS or
+advertisement-selected proxy. Every accepted transport path SHALL reject
+malformed/unsupported input without unwind before or during compatibility.
+
+Candidate ordering SHALL use two independently `OsRng`-shuffled bounded tiers:
+at most 256 URLs from the bounded previously-compatible retry index and a
+bounded reservoir sampled across the fresh terminal traversal through completion
+or its fixed 16-page cap.
+Persisted peers MUST be re-resolved and revalidated. After both tiers exist, the
+candidate-preparation subdeadline SHALL use at most half the then-remaining
+overall time and split resolution/validation time equally between tiers; unused
+persisted time MAY pass to fresh, not conversely. A two-phase pre-start target
+plan SHALL then split remaining dial time at a monotonic midpoint: persisted
+targets stop/cancel by it and consume no more than their configured limit or half
+the attempt budget; fresh targets activate for the second half. URL, wire, store,
+hash, DNS-answer, and lexical order MUST NOT choose either attempted prefix.
+Fresh candidates MUST NOT be persisted before compatibility succeeds.
+
+#### Scenario: Descriptor-only resolution
+
+- **WHEN** lookup returns a compatible reachable peer
+- **THEN** the swarm attempts it through a distinct subnet connection
+
+#### Scenario: Wrong-subnet candidate
+
+- **WHEN** a candidate fails any compatibility field
+- **THEN** it remains unverified and failure is not attributed to the relay
 
-An application SHALL be able to join a subnet given only its descriptor: it
-derives the subnet identifier, resolves the identifier to advertised
-addresses over the overlay, seeds those addresses into the subnet's
-unverified (greylist) peer set, and dials peers as an ordinary network
-instance using the subnet's own magic bytes and application identity. The
-overlay connection used for resolution MUST NOT become a connection of the
-joined subnet. If the overlay yields no reachable addresses, joining SHALL
-fail with a resolvable error rather than hang.
+#### Scenario: Unknown subnet
 
-#### Scenario: Descriptor-only join
+- **WHEN** no selected source yields a compatible ordinary peer
+- **THEN** join fails within its deadline and leaves no attempt running
 
-- **WHEN** an application requests joining a subnet for which it holds only
-  the descriptor
-- **THEN** the subnet network instance starts and establishes peer
-  connections using addresses discovered over the overlay, without any
-  subnet-specific seed configuration
+#### Scenario: Candidate resolves to prohibited destination
 
-#### Scenario: Unknown subnet
+- **WHEN** an advertised clearnet candidate resolves to loopback/private/
+  reserved space outside explicit local-test mode
+- **THEN** it is rejected before dialing and join continues within its budgets
+
+#### Scenario: Hidden-service candidate enters subnet connector
+
+- **WHEN** a canonical Tor/I2P candidate is selected for connection
+- **THEN** the connector uses the trusted configured proxy socket, passes the
+  hidden name only in proxy/TLS protocol, and has no direct fallback
+
+#### Scenario: Stored ordering is adversarial
+
+- **WHEN** returned URLs or persisted peers are arranged to control lexical or
+  insertion order
+- **THEN** bounded reservoir selection/CSPRNG shuffles choose attempted prefixes
+  and the verified tier cannot consume the fresh tier's reserved time or budget
+
+#### Scenario: Default join completion retains overlay
+
+- **WHEN** a default-policy transient completes an ordinary subnet join
+- **THEN** join completion does not itself stop the overlay
+
+#### Scenario: Explicit immediate overlay stop after join
+
+- **WHEN** reduced-privacy policy observes successful, failed, timed-out, or
+  cancelled join completion
+- **THEN** subnet network/app state continue and the caller was warned about
+  timing correlation
+
+### Requirement: Fallible initialization precedes network activity
+
+Before each subnet attempt starts, the swarm SHALL invoke a fallible app
+initializer with that attempt's `P2p` handle. It SHALL construct subnet-scoped
+app state, register protocols, and return both caller-visible state and a
+bounded shutdown hook. No listener, connection, or protocol job SHALL start
+before success.
+
+The registry SHALL retain ownership of the returned app state and shutdown hook
+for the active attempt's entire lifetime; dropping the caller handle MUST NOT
+drop required state. If initialization fails or is cancelled, partial state is
+released, nothing starts, and the original error is returned.
+
+#### Scenario: App state precedes connection
+
+- **WHEN** initialization succeeds
+- **THEN** protocols and registry-owned app state exist before network start
+
+#### Scenario: Caller drops handle
+
+- **WHEN** a caller drops its returned app handle while the subnet remains
+  active
+- **THEN** registry ownership keeps required app state alive
+
+#### Scenario: Initializer fails
+
+- **WHEN** initialization returns an error
+- **THEN** no network activity starts and partial state is released
+
+### Requirement: Join completion requires an ordinary persistent channel
+
+`P2p::start()` alone SHALL NOT complete join. Join succeeds only after an
+ordinary inbound, outbound, or manual subnet channel passes the compatibility
+handshake and remains registered as an ordinary peer. Temporary direct, seed,
+and refinement channels MUST NOT complete join, even when their handshake
+succeeds.
+
+Until then the registry SHALL expose a distinct joining state. Timeout,
+cancellation, or candidate exhaustion SHALL stop tasks and connections, run
+the shutdown hook, remove the attempt, and return a typed error without
+panicking on untrusted input.
+
+#### Scenario: Seed channel does not complete join
+
+- **WHEN** seed discovery completes but no ordinary channel exists
+- **THEN** join remains pending
+
+#### Scenario: Refinement channel does not complete join
+
+- **WHEN** a refinement probe succeeds but no ordinary channel exists
+- **THEN** join remains pending
+
+#### Scenario: Ordinary channel completes join
+
+- **WHEN** a compatible ordinary channel completes before deadline
+- **THEN** state atomically becomes joined
+
+#### Scenario: Failed join rolls back
+
+- **WHEN** deadline, cancellation, or failure ends an attempt
+- **THEN** its state is removed without disturbing overlay or other subnets
+
+### Requirement: Source policies use explicit bounded attempts
+
+Applications SHALL select overlay-only, static-only, combined, or
+overlay-then-static behavior. Sources MUST NOT activate implicitly outside the
+selected policy.
+
+- Overlay-only SHALL use overlay candidates with static seeds absent.
+- Static-only SHALL use configured subnet seed discovery with overlay lookup
+  absent.
+- Combined MAY configure both sources in one attempt.
+- Overlay-then-static SHALL complete and tear down one bounded overlay-only
+  attempt before creating a fresh static-only `P2p` attempt. It SHALL rerun the
+  initializer for the fresh attempt and remain inside one overall deadline.
+
+No policy SHALL rely on runtime manual- or seed-session reload. Every source
+uses the same compatibility and ordinary-channel completion rule.
+
+#### Scenario: Overlay-first succeeds
+
+- **WHEN** an ordinary overlay-discovered peer completes in the first attempt
+- **THEN** no static attempt starts
+
+#### Scenario: Overlay-first fails and static succeeds
 
-- **WHEN** no advertised address exists or none is reachable for a requested
-  subnet
-- **THEN** the join attempt reports failure in bounded time and no subnet
-  network instance is left running
+- **WHEN** the first attempt fully rolls back under overlay-then-static
+- **THEN** a fresh static-configured attempt reruns initialization and may join
 
-### Requirement: Per-subnet state isolation
+#### Scenario: First-attempt state does not leak
 
-Each subnet managed through the swarm SHALL have its own peer hostlists,
-refinery behavior, and on-disk datastore and hostlist files, namespaced by
-the subnet identifier. State of one subnet (hostlist entries, datastore,
-refinement outcomes) MUST NOT be shared with or leak into another subnet
-managed by the same node, and removing a subnet SHALL be possible without
-damaging other subnets' persisted state.
+- **WHEN** overlay-then-static creates its second attempt
+- **THEN** no task, channel, app state, or registry ownership from the first
+  attempt remains
+
+#### Scenario: Every source fails
+
+- **WHEN** all policy attempts fail within the overall deadline
+- **THEN** a typed aggregate failure is returned after complete rollback
+
+### Requirement: Per-subnet state remains isolated
+
+Each subnet SHALL have independent hostlists, refinement, app state, tasks,
+datastore, and hostlist files. Persistent paths SHALL be namespaced by full ID
+under a configured root. No state, address outcome, dispatch, or shutdown signal
+may cross subnets. Deleting one subnet MUST NOT alter another.
+
+Overlay role persistence rules apply only to overlay-owned state. Applications
+MAY separately configure subnet persistence for a transient overlay participant;
+such paths reveal local subnet history and MUST be documented. Private secrets
+MUST NOT be logged.
 
 #### Scenario: Independent refinement
 
-- **WHEN** an address is unreachable in subnet A but healthy in subnet B
-- **THEN** refinement in A downgrades it there while B's state for the same
-  address is unaffected
+- **WHEN** one address has different outcomes in A and B
+- **THEN** each subnet retains only its own outcome
 
 #### Scenario: Namespaced persistence
 
-- **WHEN** two subnets persist hostlists and datastores on one node
-- **THEN** their files are stored under distinct paths derived from their
-  subnet identifiers
+- **WHEN** two subnets persist under one root
+- **THEN** their files occupy distinct full-ID paths
+
+#### Scenario: Transient persists a subnet explicitly
+
+- **WHEN** a transient overlay caller enables subnet persistence
+- **THEN** only that separately configured subnet state is written and its
+  local-history implication is documented
+
+### Requirement: Serving mode is selected before initial start
+
+A subnet SHALL be created either join-only or serving. Serving creation SHALL
+require a persistent overlay role, at least one local listener bind address,
+and at least one externally advertised endpoint assigned to that subnet. Bind
+addresses and advertised endpoints SHALL be separate fields and MUST NOT be
+assumed identical. All addresses SHALL be validated before network start.
+
+Serving initialization SHALL configure listeners before `P2p::start()`. It
+completes when app initialization succeeds and required listeners bind; it does
+not require an existing peer. This permits the first member of a new subnet to
+serve. It may attempt peer discovery afterward under an explicit source policy.
+A transient SHALL reject serving before any network activity.
+
+Before initializer, listener, or author activation, serving SHALL atomically
+allocate or reuse one 256-slot local-author reserve partition for the subnet.
+If all configured partitions are occupied by subnets with protected local IDs,
+serving SHALL fail with a typed capacity error. A newly allocated empty partition
+SHALL be released on pre-authoring initialization/bind failure. Stopping serving
+SHALL retain a nonempty partition until all protected local IDs expire, then
+release it only if the subnet has not resumed serving.
+
+#### Scenario: First serving member
+
+- **WHEN** no subnet peer exists but a persistent caller supplies valid bind
+  and advertised addresses
+- **THEN** the serving subnet succeeds after listener readiness without a peer
+  handshake
+
+#### Scenario: Bind and advertised endpoint differ
+
+- **WHEN** an externally provisioned endpoint forwards to a distinct local bind
+  address
+- **THEN** the listener binds locally while ads contain only the external
+  endpoint
+
+#### Scenario: Transient serving is rejected
+
+- **WHEN** a transient requests serving
+- **THEN** failure occurs before initializer, listener, or author task starts
+
+#### Scenario: Author reserve is exhausted
+
+- **WHEN** a persistent caller requests serving while every reserve partition is
+  retained by protected local IDs for other subnets
+- **THEN** failure occurs before initializer, listener, or author task starts
+
+### Requirement: Serving promotion uses controlled recreation
+
+An already started join-only subnet MUST NOT be promoted by mutating inbound
+settings and calling session reload. Promotion SHALL require a controlled
+stop/recreate operation: stop the joined instance completely, then create a new
+serving-configured instance and rerun initialization. The operation SHALL be
+serialized with other lifecycle actions and return a typed result if recreation
+fails. Retained state MAY be reused only from that subnet's namespace.
+
+Listener binds and externally advertised endpoints MUST remain distinct.
+Swarm SHALL NOT claim automatic Tor or I2P identity provisioning. Reusing one
+advertised endpoint across local subnets SHALL produce an explicit linkability
+warning.
+
+#### Scenario: Promotion does not use reload
+
+- **WHEN** a joined subnet is promoted to serving
+- **THEN** its old instance fully stops before a serving-configured instance
+  starts
+
+#### Scenario: Recreation fails
+
+- **WHEN** the serving listener cannot bind or initialization fails
+- **THEN** no partial serving instance or author task remains
+
+#### Scenario: Shared endpoint warning
+
+- **WHEN** one external endpoint is assigned to two served subnets
+- **THEN** an explicit local linkability warning is produced
+
+### Requirement: Advertisement authoring is serving-only and cadence-only
+
+Join-only subnets SHALL author no ads. A successfully serving subnet SHALL
+author bounded ads only on the overlay jittered cadence and only with its
+advertised endpoints. Initialization, listener readiness, peer connection,
+recreation, and new overlay channels MUST NOT emit immediately. Stopping
+serving SHALL cease future authoring without a withdrawal.
+
+#### Scenario: Join-only remains silent
+
+- **WHEN** a subnet is joined without serving mode
+- **THEN** it authors no ad
+
+#### Scenario: First server waits
+
+- **WHEN** a first serving member becomes listener-ready
+- **THEN** its first ad waits for the next cadence tick
+
+#### Scenario: Stop is silent
+
+- **WHEN** a serving subnet stops
+- **THEN** no departure message is sent and existing ads expire locally
+
+### Requirement: Leave and recreation are idempotent and failure-safe
 
-### Requirement: Serving a subnet is opt-in with advertisement obligations
+Leave SHALL disable authoring, stop network producers/channels/protocol jobs,
+run app shutdown, and remove registry ownership. It MUST emit no withdrawal.
+Repeated leave SHALL be idempotent. Same-subnet join, serve, recreate, leave,
+and delete operations SHALL have one serialized owner.
 
-An application SHALL be able to declare, per subnet, that it serves the
-subnet. Serving requires inbound addresses for that subnet; when active,
-the node SHALL gossip advertisements for the subnet on the jittered cadence
-defined by the overlay capability, and those advertisements SHALL carry
-only the inbound addresses assigned to that subnet. A node that merely
-joins a subnet (default) SHALL emit no advertisement for it. A transient
-overlay node SHALL NOT serve any subnet.
+The caller SHALL choose to retain or delete namespaced state after shutdown.
+Deletion MUST occur only after stop and affect only that ID.
 
-#### Scenario: Opt-in serving advertises
+#### Scenario: Repeated leave
 
-- **WHEN** an application marks a joined subnet as served with configured
-  inbound addresses
-- **THEN** other overlay participants can resolve the subnet to those
-  addresses, and no advertisement is emitted before the first cadence tick
+- **WHEN** leave is called after stop
+- **THEN** it returns the already-stopped result without restarting work
 
-#### Scenario: Join-only stays silent
+#### Scenario: Retained rejoin
 
-- **WHEN** an application joins a subnet without declaring serving
-- **THEN** no advertisement naming that subnet is ever emitted by the node
+- **WHEN** state is retained
+- **THEN** a later attempt may reuse only that subnet's files
 
-### Requirement: Subnet leave without departure broadcast
+#### Scenario: Delete after stop
 
-Leaving a subnet SHALL stop the subnet's network instance, release its
-connections, and deregister it from the managing swarm. The leave MUST NOT
-emit any departure or withdrawal message on the overlay; the node's
-advertisements for that subnet simply cease and expire by TTL. Persisted
-per-subnet state MAY be retained for rejoining.
+- **WHEN** deletion is selected
+- **THEN** only that namespace is deleted after shutdown
 
-#### Scenario: Silent leave
+### Requirement: Concurrent lifecycle remains isolated
 
-- **WHEN** an application leaves a subnet it was serving
-- **THEN** no overlay message announces the departure, and other nodes'
-  advertisement stores still list its addresses until TTL expiry or failed
-  liveness checks remove them
+Different subnets MAY transition concurrently; one ID SHALL have one owner.
+Duplicate joins MUST NOT create duplicate instances. Swarm shutdown SHALL stop
+authoring, cancel attempts, stop every subnet despite individual failures, and
+stop the overlay last within a finite deadline.
 
-#### Scenario: Rejoin reuses state
+#### Scenario: Duplicate concurrent join
 
-- **WHEN** a node rejoins a subnet it previously left and retained state for
-- **THEN** its subnet peer hostlist resumes from the persisted state
+- **WHEN** two callers request the same descriptor
+- **THEN** at most one instance is created and deterministic state is returned
 
-### Requirement: Runtime subnet lifecycle
+#### Scenario: Late subnet operation
 
-A swarm-managed node SHALL support starting and stopping subnets at any time
-after the overlay is running, without restarting the overlay or other
-subnets. Concurrent shutdown of multiple subnets SHALL terminate cleanly
-without affecting surviving subnets.
+- **WHEN** a new join or serving creation starts while others run
+- **THEN** existing subnet and overlay operation is not restarted
 
-#### Scenario: Late subnet spawn
+#### Scenario: One shutdown hook fails
 
-- **WHEN** a new subnet join is requested while other subnets are already
-  running
-- **THEN** the new subnet starts without disruption to existing subnet
-  connections or the overlay
+- **WHEN** one app shutdown hook returns an error
+- **THEN** every other subnet still receives a stop attempt and failures are
+  aggregated without panic
 
-#### Scenario: Concurrent teardown
+### Requirement: Pinned and private descriptors share lifecycle rules
 
-- **WHEN** several subnets are stopped at once
-- **THEN** all stop cleanly and remaining subnets continue operating
+Applications SHALL pin public descriptors with golden IDs and accept valid
+private descriptors. Private secrets MUST NOT be logged, advertised, sent in
+lookup, or exposed by public status; overlay messages use only the derived ID.
+Possession of the descriptor SHALL NOT bypass compatibility or app-level
+authorization.
 
-### Requirement: Pinned and private subnets
+#### Scenario: Pinned interoperability
 
-Applications SHALL be able to pin known subnet descriptors (e.g. released
-networks) so their identifiers are stable across releases, and SHALL be able
-to construct subnets from user-supplied descriptors, including
-secret-bearing descriptors for private subnets. Static per-subnet seed lists
-SHALL remain usable as a fallback or override for any pinned subnet during
-the migration period.
+- **WHEN** deployments use one pinned descriptor
+- **THEN** they derive one ID and apply the same compatibility checks
 
-#### Scenario: Pinned identifier matches released network
+#### Scenario: Private descriptor is not transmitted
 
-- **WHEN** an app pins a released network's descriptor
-- **THEN** the derived identifier matches the one used by all other pinned
-  deployments, and the subnet is discoverable through the overlay
+- **WHEN** a private subnet is resolved or created
+- **THEN** no overlay message contains its secret
 
-#### Scenario: Static seeds still work
+#### Scenario: Private ID grants no access
 
-- **WHEN** an app is configured with static seeds for a pinned subnet and
-  overlay discovery is unavailable
-- **THEN** the subnet can still be joined via the static seeds
+- **WHEN** a peer knows an ID but fails application authorization
+- **THEN** swarm grants no authorization

openspec/changes/swarm/specs/swarm-overlay/spec.md

@@ -1,202 +1,929 @@
 ## Purpose
 
-Defines the swarm overlay rendezvous protocol: how subnet identifiers are
-derived, how subnet advertisements are formatted, gossiped, stored, and
-queried, the persistent vs transient participant roles, and the binding
-anti-linkability constraints that keep the overlay from becoming a
-cross-subnet correlation point.
+Defines a bounded rendezvous overlay that resolves known subnet descriptors to
+untrusted peer addresses while making bootstrap, replay, resource, persistence,
+and metadata-disclosure boundaries explicit.
 
 ## ADDED Requirements
 
-### Requirement: Subnet identifier derivation
+### Requirement: Versioned subnet identifier derivation
 
-A subnet identifier SHALL be the BLAKE3 hash of a canonical, deterministically
-serialized descriptor binding the application name, network magic bytes, and
-version constraint of the subnet. The descriptor MAY include a secret; when
-present, the resulting identifier SHALL be computationally indistinguishable
-from any other identifier to a party lacking the secret. Two different
-descriptors MUST NOT yield the same identifier.
+A version-1 descriptor SHALL bind the subnet application name, magic bytes,
+and major/minor version pair used by the existing compatibility handshake. Its
+canonical bytes SHALL be, in order:
 
-#### Scenario: Same descriptor, same identifier
+1. ASCII `darkfi-swarm-subnet-v1` followed by one zero byte;
+2. application-name UTF-8 byte length as unsigned 16-bit big-endian, followed
+   by those exact bytes without Unicode normalization;
+3. the four magic bytes;
+4. major and minor versions as unsigned 64-bit big-endian integers;
+5. zero for a public descriptor or one for a private descriptor; and
+6. for a private descriptor only, exactly 32 secret bytes.
 
-- **WHEN** two nodes derive an identifier from byte-identical descriptors
-- **THEN** both obtain the same BLAKE3 identifier
+Application names MUST contain 1 through 32 UTF-8 bytes. Other private-secret
+lengths and flag values MUST be rejected. `SubnetId` SHALL be the 32-byte
+BLAKE3 hash of the canonical bytes. Implementations SHALL rely on collision
+resistance rather than claim collision is impossible. Patch, prerelease, and
+build metadata SHALL NOT affect the ID.
 
-#### Scenario: Private subnet is unguessable
+For public `darkirc`, magic `fb e5 c7 b5`, and compatibility pair `0.5`, the
+canonical bytes SHALL be:
 
-- **WHEN** a party without the secret attempts to name or enumerate a
-  secret-bearing subnet
-- **THEN** they cannot produce its identifier or distinguish it from the
-  identifier space of public subnets
+`6461726b66692d737761726d2d7375626e65742d76310000076461726b697263fbe5c7b50000000000000000000000000000000500`
 
-#### Scenario: Divergent descriptors collide
+and `SubnetId` SHALL be:
 
-- **WHEN** two descriptors differing in any bound field are hashed
-- **THEN** the derived identifiers differ
+`b4c9d83b53cc7473d26bf173a9abd5e3025957141e20779960d587eec88618ed`.
 
-### Requirement: Advertisement format and invariants
+#### Scenario: Golden public identifier
 
-A subnet advertisement SHALL carry exactly one subnet identifier, a list of
-`(address, last_seen)` pairs for peers serving that subnet, and a TTL in
-seconds. Advertisements MUST be unsigned and MUST NOT contain any node
-identity, key, or per-node nonce. Advertised addresses SHALL be restricted to
-publicly shareable transport schemes. An advertisement for one subnet MUST
-NOT embed addresses belonging to another subnet.
+- **WHEN** an implementation derives the specified darkirc identifier
+- **THEN** its canonical bytes and hash equal the normative values
 
-#### Scenario: Advertisement is self-contained per subnet
+#### Scenario: Compatibility-equivalent patch versions
 
-- **WHEN** a node gossips an advertisement for subnet S
-- **THEN** the advertisement contains only S's identifier and addresses
-  attributed to S, and no field links it to any other subnet or to a stable
-  node identity
+- **WHEN** descriptors differ only in patch, prerelease, or build metadata
+- **THEN** they derive the same identifier
 
-#### Scenario: Non-shareable address rejected
+#### Scenario: Bound field differs
 
-- **WHEN** an advertisement contains an address whose scheme is not publicly
-  shareable (e.g. a proxy-internal scheme)
-- **THEN** receiving nodes discard that address rather than storing or
-  relaying it
+- **WHEN** valid descriptors differ in any bound field
+- **THEN** their canonical bytes differ and identifiers are expected to differ
+  under BLAKE3 collision resistance
 
-### Requirement: Gossip propagation with origin ambiguity
+#### Scenario: Malformed private descriptor
 
-Advertisements SHALL propagate by flooding/gossip between overlay peers. A
-node relaying an advertisement MUST relay it unmodified, such that the
-immediate sender of an advertisement is never evidence of its authorship.
-Nodes MUST NOT include provenance, hop counts tied to a node, or relay
-signatures in relayed advertisements. Re-gossip of a node's own served
-subnets SHALL occur on a slow, jittered cadence and MUST NOT be triggered
-immediately upon a subnet starting, an inbound listener appearing, or a new
-overlay connection being established.
+- **WHEN** a private descriptor secret is not exactly 32 bytes
+- **THEN** validation fails before hashing
 
-#### Scenario: Relay preserves ambiguity
+### Requirement: Private identifiers are rendezvous capabilities only
 
-- **WHEN** a node receives an advertisement and relays it to its peers
-- **THEN** the relayed message is byte-equivalent in identifying fields and
-  provides no indicator of whether the relaying node authored it
+A generated private secret MUST use a cryptographically secure random source.
+A caller-supplied secret SHALL be accepted only as exactly 32 bytes and SHALL
+be documented as requiring independent high entropy. It makes an identifier
+difficult to derive only before observation. It MUST NOT be represented as
+authentication, encryption, authorization, or continuing confidentiality.
 
-#### Scenario: No event-triggered advertisement
+Public enumeration MUST omit records from accepted ads marked non-public, but
+visibility is unauthenticated sender data rather than an intrinsic property of
+an ID. Persistent gossip peers necessarily observe IDs they store or relay, and
+an attacker that learns an ID can submit another ad marking it public.
 
-- **WHEN** a node begins serving a subnet or gains a new overlay peer
-- **THEN** it does not immediately emit an advertisement for that subnet;
-  the next emission waits for the jittered cadence
+#### Scenario: Unobserved random private identifier
 
-### Requirement: Subnet queries answered from local state
+- **WHEN** a party neither knows nor observes a uniformly random secret
+- **THEN** deriving its identifier requires guessing the 32-byte secret
 
-The overlay SHALL provide a query for the set of known subnet identifiers and
-a query for the advertised addresses of a specific subnet. Both queries SHALL
-be answered from the answering node's local advertisement state at the time
-of the query, for every connected overlay peer regardless of that peer's
-declared role. Query exchanges MUST NOT require the querier to reveal which
-subnets it participates in beyond the subnet explicitly named in a
-per-subnet query.
+#### Scenario: Private identifier is disclosed
 
-#### Scenario: Subnet list query
+- **WHEN** a private ID is sent in an ad or lookup
+- **THEN** the receiving overlay peer can observe and reuse it
 
-- **WHEN** a node sends a subnet-list query to a connected overlay peer
-- **THEN** it receives the set of subnet identifiers currently known to that
-  peer's local advertisement state
+#### Scenario: Observed private ID is relabeled
 
-#### Scenario: Per-subnet address query
+- **WHEN** an attacker re-advertises an observed private ID with public
+  visibility
+- **THEN** the protocol may catalog the forged public record and makes no
+  intrinsic-privacy claim for the ID
 
-- **WHEN** a node sends a per-subnet query for identifier S
-- **THEN** it receives addresses advertised for S from the answering peer's
-  local state, and the exchange names no other subnet
+### Requirement: Bootstrap uses staged ordinary non-seed overlay connections
 
-### Requirement: Advertisement store with TTL and bounds
+Bootstrap addresses SHALL use ordinary non-seed sessions capable of carrying
+swarm requests and responses, not existing short-lived seed sessions.
+“Ordinary” SHALL describe session type, not a requirement that a transient keep
+the channel indefinitely. A transient with a cache SHALL first construct and
+start an overlay instance using cached peers only. If no ordinary compatible
+channel completes within the stage timeout, it SHALL fully stop and discard
+that instance before constructing a fresh overlay instance using configured
+ordinary bootstrap peers. It MUST NOT depend on runtime manual-session reload.
 
-A persistent overlay node SHALL maintain an advertisement store. Entries
-SHALL expire no later than their TTL after last confirmation. Persistent
-nodes SHALL verify reachability of advertised addresses on a periodic,
-liveness-check schedule and SHALL drop or downgrade entries that fail.
-The store SHALL enforce per-subnet and total entry caps so that
-advertisement floods cannot grow it unboundedly. Transient nodes SHALL NOT
-maintain a persistent store.
+Each stage timeout MUST be finite and no greater than 300 seconds. The cache
+MUST contain no more than 256 endpoint records, 262,144 encoded file bytes, or
+1,024 encoded bytes per URL. A record SHALL contain only the exact canonical
+connect URL and resolved endpoint actually used by a successfully completed
+ordinary channel advertising `("swarm-ad-store", 1)`. Advertised external
+addresses MUST NOT be cached from that feature handshake. The cache MUST NOT
+contain features, subnet IDs, ads, queries, or source/query associations.
 
-#### Scenario: TTL expiry
+#### Scenario: Cached stage succeeds
 
-- **WHEN** an advertisement entry's TTL elapses without re-confirmation
-- **THEN** the store no longer returns that entry in query responses
+- **WHEN** a cached ordinary peer completes before the stage deadline
+- **THEN** configured bootstrap peers are not contacted
 
-#### Scenario: Unreachable advertisement dropped
+#### Scenario: Cached stage fails
 
-- **WHEN** an advertised address repeatedly fails liveness checks
-- **THEN** the store stops serving it and it is eligible for removal
+- **WHEN** no cached peer completes before the stage deadline
+- **THEN** the cached overlay instance is stopped before a fresh configured-peer
+  instance starts
 
-#### Scenario: Flood bounded
+#### Scenario: Bootstrap peer answers lookup
 
-- **WHEN** an attacker floods advertisements exceeding the store caps
-- **THEN** store size stays within its configured bounds
+- **WHEN** a fresh configured-peer stage establishes an ordinary channel
+- **THEN** lookup can run on that channel without a seed session
 
-### Requirement: Overlay node roles — persistent and transient
+#### Scenario: Cache contains no subnet activity
 
-An overlay node SHALL declare itself persistent by advertising a
-swarm-ad-store feature in the connection handshake's features vector; a node
-not advertising it is transient. A transient node SHALL NOT accept inbound
-overlay connections, SHALL NOT emit subnet advertisements, and SHALL NOT
-maintain an advertisement store. A transient node MAY persist a cache of
-overlay peer addresses (an overlay hostlist) so later sessions dial cached
-overlay peers before configured seeds; such a cache MUST NOT record which
-subnets the node queried or joined. A persistent node SHALL maintain a
-TTL-bounded advertisement store and SHALL relay gossip. Both roles SHALL
-handle all overlay protocol messages identically while connected; role
-SHALL NOT alter message formats or per-message processing, only the local
-state available to answer from. No overlay node SHALL refuse
-protocol-correct messages solely because the sender is transient.
+- **WHEN** lookup sessions persist the swarm cache
+- **THEN** it contains successful connect/resolved endpoint pairs only
 
-#### Scenario: Mobile lookup session
+#### Scenario: Advertised external address is not cache authority
 
-- **WHEN** a transient node connects to the overlay, resolves the addresses
-  of a subnet, and disconnects
-- **THEN** it has persisted at most its overlay hostlist cache (overlay
-  peer addresses only, no subnet identifiers), has emitted no
-  advertisements, retains no advertisement store, and appears in no other
-  node's hostlist or advertisement store
+- **WHEN** a connected persistent peer advertises an external address different
+  from the endpoint used by its successful channel
+- **THEN** the different external address is not added to the cache
 
-#### Scenario: Cached bootstrap avoids seeds
+### Requirement: Untrusted dials use validated exact targets
 
-- **WHEN** a transient node starts a session with a populated overlay
-  hostlist cache while its configured seeds are unreachable
-- **THEN** it still establishes overlay connectivity by dialing cached
-  overlay peers
+Before dialing a fresh advertised or configured untrusted target, the swarm
+SHALL resolve clearnet names once and validate the selected socket address. A
+cached target SHALL instead revalidate and reuse its stored successfully
+connected socket without DNS resolution. Unless explicit local-test mode is
+enabled, both paths MUST reject loopback, private, shared, link-local, multicast,
+unspecified, documentation, benchmarking, and otherwise reserved destinations
+for IPv4 and IPv6. Production lilith MUST NOT enable local-test mode.
 
-#### Scenario: Persistent node cold-starts another
+Every validated target SHALL carry the original URL, one mandatory exact socket,
+and a direct or trusted-proxy route kind. For a direct route, the socket is the
+validated destination and connection SHALL use it without a second DNS lookup;
+the original hostname MAY be retained only for TLS identity.
 
-- **WHEN** a fresh node connects to any persistent overlay node and queries
-  a subnet
-- **THEN** it can discover and dial serving peers without any per-subnet
-  seed configuration
+For a trusted-proxy route, the mandatory socket SHALL be the exact endpoint from
+trusted local proxy configuration, not the advertised destination. That proxy
+socket MAY be loopback/private under local trust policy; an advertisement MUST
+NOT select or override it. The untrusted destination SHALL be either a globally
+routable IP literal or a canonical Tor/I2P hidden-service name matching the
+transport. A hidden-service name MUST NOT undergo local DNS and MAY be passed
+only inside proxy negotiation and for TLS identity. Arbitrary clearnet hostnames
+MUST NOT be sent for remote proxy DNS. Missing/malformed proxy configuration and
+transport/scheme mismatch SHALL reject the candidate. One join attempt SHALL
+dial one exact route at most once.
 
-#### Scenario: Role does not split protocol behavior
+A clearnet DNS answer SHALL contain at most 16 socket addresses and every answer
+SHALL consume the join's resolution budget and pass address classification.
+Empty or oversized answers SHALL fail. A bounded nonempty answer SHALL be
+iterated without unchecked indexing; its allowed sockets SHALL be shuffled with
+`OsRng` before selecting at most one exact socket for that URL. A cached trusted-
+proxy route SHALL revalidate that its exact socket still matches current trusted
+local proxy configuration; mismatch fails without DNS or fallback.
 
-- **WHEN** the same query is sent to a persistent and to a transient node
-  that both hold the same local advertisement state
-- **THEN** both answer with the same message types and semantics
+An advertised candidate SHALL remain a typed pair of original URL and validated
+resolved socket through the subnet connection attempt. Before compatibility
+succeeds it MUST NOT be downgraded into a URL-only hostlist, persisted, or sent
+through a connector/refinery path that resolves it again. Failed candidates
+SHALL be dropped. After compatibility succeeds, ordinary host persistence MAY
+record the peer URL, but every future reconnect or refinement attempt MUST
+resolve, validate, budget, and dial one exact socket again.
 
-### Requirement: Metered overlay messages
+#### Scenario: DNS result changes after validation
 
-All overlay protocol messages SHALL be subject to per-message-type metering
-with configured thresholds and penalties, such that a peer flooding any
-overlay message type is throttled or banned under the node's ban policy.
-Message size estimates SHALL be declared for every overlay message type.
+- **WHEN** a hostname changes resolution after an address was validated
+- **THEN** the connection uses the previously validated socket and performs no
+  second DNS lookup
 
-#### Scenario: Query flood throttled
+#### Scenario: Cached hostname would resolve differently
 
-- **WHEN** a peer sends subnet queries beyond the metering threshold
-- **THEN** the node applies the metering penalty for that message type
+- **WHEN** a cached record contains an original hostname and successful socket
+  but DNS now returns another address
+- **THEN** cache reuse revalidates/dials the stored socket and performs no DNS
+  lookup for that cached attempt
 
-### Requirement: No cross-subnet linkability in overlay traffic
+#### Scenario: Private or loopback result
 
-The overlay protocol MUST NOT provide any mechanism that links a single
-node across subnets: advertisements for different subnets served by one
-node MUST NOT be correlatable by their content, and no overlay message
-SHALL carry a stable node identifier, signature key, or address reused
-across subnets. Deployment of a shared inbound endpoint across multiple
-served subnets SHALL be flagged as a known linkability hazard in node
-documentation.
+- **WHEN** an untrusted clearnet target resolves to a prohibited range outside
+  explicit local-test mode
+- **THEN** it is rejected before connection
 
-#### Scenario: Two ads from one operator stay unlinkable
+#### Scenario: Victim endpoint is repeatedly advertised
 
-- **WHEN** one operator serves two subnets through distinct per-subnet
-  addresses and follows the jittered cadence
-- **THEN** an overlay observer cannot bind the two advertisements to each
-  other through any field of the overlay protocol itself
+- **WHEN** many ads name one globally routable victim endpoint
+- **THEN** one join attempt dials it at most once and all dial rate/concurrency/
+  total budgets remain enforced
+
+#### Scenario: Remote proxy DNS target
+
+- **WHEN** an untrusted ad supplies an arbitrary clearnet hostname to a proxy
+  mode that would resolve it remotely
+- **THEN** the target is rejected rather than bypassing egress validation
+
+#### Scenario: Canonical hidden service uses trusted proxy
+
+- **WHEN** an accepted Tor/I2P candidate names a canonical hidden service
+- **THEN** its exact socket is the locally configured trusted proxy, the hidden
+  name is not locally resolved, and the ad cannot alter the proxy endpoint
+
+#### Scenario: Hidden-service proxy is absent
+
+- **WHEN** an otherwise valid hidden-service candidate has no valid configured
+  proxy matching its transport
+- **THEN** it fails before dialer construction without direct-network fallback
+
+#### Scenario: Candidate crosses into subnet connector
+
+- **WHEN** a validated advertised candidate is handed to the subnet connection
+  attempt
+- **THEN** its original URL and exact socket remain typed until compatibility
+  succeeds, with no second DNS lookup or URL-only greylist insertion
+
+#### Scenario: Failed candidate is not persisted
+
+- **WHEN** a validated candidate fails transport or compatibility
+- **THEN** it is dropped without entering persistent hostlist or refinery state
+
+#### Scenario: Compatible peer reconnects later
+
+- **WHEN** an ordinary persisted peer is retried or refined in a later attempt
+- **THEN** that attempt resolves and validates a new exact target under the same
+  egress and dial budgets before connection
+
+### Requirement: Advertised-candidate processing is fully fallible
+
+The complete attacker-selected candidate pipeline SHALL be fallible: URL parse,
+scheme allowlist, host/port extraction, DNS result handling, address
+classification, validated-target construction, proxy selection and negotiation,
+transport/TLS dialing, and compatibility. It MUST NOT use `unwrap`, `expect`,
+explicit panic, unchecked slicing/indexing, or an unimplemented transport branch.
+Unsupported or unaudited schemes MUST be rejected before dialer construction.
+Empty or oversized DNS results, malformed targets, missing/malformed proxies,
+timeouts, cancellation, and transport errors MUST return bounded candidate
+errors. Bounded multiple-address results MUST be iterated and classified without
+unchecked selection. Processing SHALL continue or terminate only according to
+the join budget.
+
+#### Scenario: Resolution returns no addresses
+
+- **WHEN** an advertised clearnet name resolves to an empty set
+- **THEN** candidate processing returns an error without indexing or unwind
+
+#### Scenario: Resolution returns multiple addresses
+
+- **WHEN** a clearnet name resolves to a bounded nonempty address set
+- **THEN** every result is budgeted/classified without unchecked indexing and at
+  most one `OsRng`-shuffled allowed exact socket is selected for that URL
+
+#### Scenario: Every accepted scheme receives hostile input
+
+- **WHEN** arbitrary malformed targets exercise each scheme accepted from ads
+- **THEN** parsing through compatibility returns bounded errors without unwind
+
+#### Scenario: Enabled transport has no audited fallible adapter
+
+- **WHEN** an advertisement selects an enabled but unsupported or unimplemented
+  transport path
+- **THEN** the candidate is rejected before transport construction or dialing
+
+### Requirement: Overlay control channels are never subnet data channels
+
+An overlay channel SHALL remain owned by the overlay `P2p` identified by fixed
+swarm magic bytes, app name, version policy, host state, and protocol registry.
+It MUST NOT carry subnet application messages, be transferred to a subnet
+`P2p`, be re-handshaken in place under a subnet identity, or multiplex traffic
+tagged by subnet ID. Joining SHALL create a separate subnet channel under that
+subnet's own identity and state.
+
+A persistent participant SHALL keep its overlay active while it performs
+durable store or gossip duties. A participant serving any subnet SHALL keep the
+overlay active while advertisement authoring is enabled. The default transient
+policy SHALL retain the overlay for the application session and MUST NOT stop it
+as an automatic reaction to lookup or join completion. The application session
+ends only through explicit overlay stop or full Swarm shutdown. An explicit
+reduced-privacy policy SHALL stop after every caller-visible lookup/join terminal
+outcome—success, empty result, error, timeout, or cancellation—but MUST NOT stop
+after an internal lookup phase within join. Configuration and documentation
+MUST warn that responder/subnet observers can correlate the query, subnet
+connection, and teardown timing. Stopping the overlay MUST NOT stop or transfer
+the independent subnet `P2p`; a later lookup establishes a new overlay session
+only when no active overlay remains.
+
+#### Scenario: Overlay peer also serves requested subnet
+
+- **WHEN** the lookup responder also operates a serving endpoint for the
+  requested subnet
+- **THEN** the client opens a separate subnet connection rather than reusing the
+  overlay channel
+
+#### Scenario: Default transient join does not trigger disconnect
+
+- **WHEN** a default-policy transient completes lookup or subnet join
+- **THEN** join completion itself does not stop the overlay
+
+#### Scenario: Explicit reduced-privacy teardown
+
+- **WHEN** a caller selects immediate teardown and lookup or join reaches any
+  terminal outcome
+- **THEN** the overlay stops without stopping the subnet and configuration/docs
+  flag the timing-correlation risk
+
+#### Scenario: Join's internal lookup completes
+
+- **WHEN** reduced-privacy policy is active and an internal lookup yields
+  candidates while the caller-visible join remains in progress
+- **THEN** the overlay is not stopped until join reaches its terminal outcome
+
+#### Scenario: Persistent duties retain overlay
+
+- **WHEN** persistent store/gossip or serving-advertisement duties remain active
+- **THEN** the participant does not intentionally stop its overlay
+
+### Requirement: Wire messages have fixed correlation and size bounds
+
+Every lookup request SHALL carry a fresh cryptographically random 16-byte
+request ID. Its response or bounded error SHALL echo that ID. A channel SHALL
+have no more than 32 outstanding requests. Responses with unknown, duplicate,
+expired, or mismatched request IDs MUST be rejected. Pending request state MUST
+be removed on response, timeout, or disconnect. Request timeout SHALL default
+to 10 seconds, be configurable no higher than 60 seconds, and produce a local
+timeout error; a late response is unsolicited and receives no wire error.
+
+Protocol hard limits SHALL be:
+
+| Message | Maximum encoded bytes |
+|---|---:|
+| `SubnetAd` | 65,536 |
+| `GetSubnetAddrs` | 128 |
+| `SubnetAddrs` | 65,536 |
+| `GetPublicSubnets` | 128 |
+| `PublicSubnets` | 16,384 |
+| `SwarmError` | 128 |
+
+Every encoded URL MUST be no more than 1,024 bytes. A page cursor SHALL be a
+fixed 65-byte value encoded as version `u8`, last returned key `[32]`, and
+terminal key `[32]`. Size/count validation MUST occur before store, index,
+response, or relay work.
+
+Message command strings and canonical field order SHALL be:
+
+| Command | Fields in encoded order |
+|---|---|
+| `swarm.ad` | `SubnetId[32]`, visibility `u8`, ad ID `[32]`, lifetime `u32`, URL vector |
+| `swarm.geta` | request ID `[16]`, `SubnetId[32]`, optional cursor |
+| `swarm.addrs` | request ID `[16]`, `SubnetId[32]`, URL vector, optional cursor |
+| `swarm.gets` | request ID `[16]`, optional cursor |
+| `swarm.subs` | request ID `[16]`, `SubnetId` vector, optional cursor |
+| `swarm.err` | request ID `[16]`, error code `u8` |
+
+Fields SHALL use existing DarkFi canonical wire encoding. Visibility values
+SHALL be `0 = public` and `1 = non-public`; other values are invalid. Error
+values SHALL be `0 = malformed`, `1 = invalid cursor`, `2 = enumeration
+disabled`, and `3 = busy`; other values are invalid. Lifetime is unsigned
+32-bit and cursor version SHALL be one.
+
+Every attacker-controlled swarm, version, and verack decoder SHALL return a
+fallible error for malformed or truncated input without unwind. Such paths MUST
+NOT use `unwrap`, `expect`, explicit panic, unchecked slicing/indexing, or
+reserve/allocate from an unvalidated declared length/count. Bounds validation
+MUST precede allocation and element decoding.
+
+#### Scenario: Concurrent requests correlate correctly
+
+- **WHEN** multiple lookups are outstanding on one channel
+- **THEN** each response completes only the request whose ID it echoes
+
+#### Scenario: Unsolicited response
+
+- **WHEN** a response carries no live matching request ID
+- **THEN** it is rejected and consumes metering budget
+
+#### Scenario: Message exceeds hard limit
+
+- **WHEN** any message exceeds its encoded maximum
+- **THEN** it is rejected before variable store or relay work
+
+#### Scenario: Payload is truncated at any byte
+
+- **WHEN** a valid swarm, version, or verack payload is truncated at any byte
+  boundary
+- **THEN** decoding returns an error without unwind or excessive allocation
+
+#### Scenario: Declared length is hostile
+
+- **WHEN** an arbitrary payload declares a count or length larger than its
+  validated bound or remaining bytes
+- **THEN** decoding rejects it before reservation, slicing, or element work
+
+### Requirement: Advertisement format is bounded and nonempty
+
+An ad SHALL contain exactly one subnet ID, public/non-public visibility, a
+fresh 32-byte per-ad ID, lifetime seconds, and 1 through 32 serving addresses.
+The ad ID MUST use a cryptographically secure random source and MUST NOT be
+reused for another emission or subnet. It is deduplication data, not identity.
+
+Lifetime MUST be 1 through 86,400 seconds. Ads MUST NOT contain `last_seen`, a
+stable node ID, signing key, author, relay provenance, hop count, or identifier
+shared with another subnet. Every address MUST be valid, publicly shareable,
+and within the URL bound. A receiver SHALL reject the entire ad if any field or
+address is invalid.
+
+#### Scenario: Valid ad is subnet-scoped
+
+- **WHEN** a valid ad for S is accepted
+- **THEN** it contains only S, S's addresses, and a unique ephemeral ad ID
+
+#### Scenario: Empty ad
+
+- **WHEN** an ad contains no address
+- **THEN** it is rejected without store or relay work
+
+#### Scenario: Invalid address
+
+- **WHEN** any ad address is malformed, non-shareable, or overlong
+- **THEN** no part of the ad is stored or relayed
+
+### Requirement: Expiry, replay suppression, and storage remain bounded
+
+A persistent participant SHALL use finite nonzero per-subnet, global-address,
+per-subnet protected-ID, and global protected-ID caps. Configured values MUST NOT
+exceed 1,024 addresses per subnet, 65,536 total addresses, 1,024 protected IDs
+per subnet in the general pool, or 262,144 protected IDs globally. Runtime
+expiry SHALL use a monotonic deadline from local receipt and sender clocks SHALL
+have no effect.
+
+Those caps SHALL default respectively to 256 addresses per subnet, 16,384 total
+addresses, 256 general protected IDs per subnet, and 65,536 protected IDs
+globally.
+
+The receiver SHALL clamp each accepted address lifetime to the lesser of the
+wire lifetime and a local receive cap. That cap SHALL default to 7,200 seconds,
+MUST be nonzero, and MUST NOT exceed 86,400 seconds. Local author lifetime SHALL
+also default to 7,200 seconds and MUST NOT exceed 86,400 seconds.
+
+Receiving a retained ad ID MUST NOT extend expiry or repeat relay work. The
+dedup deadline SHALL be exactly 86,400 seconds after the associated locally
+clamped address expiry, making total protection no greater than 172,800 seconds
+from acceptance. A protected ID MUST NOT be evicted before that deadline. A fresh remote
+ad SHALL be rejected without address mutation or relay when its subnet's
+general-pool quota or the global general pool has no expired slot. Expired IDs
+may be evicted deterministically.
+
+Local-author reserve-subnet partitions SHALL default to 32 and MUST NOT exceed
+256. Each partition contains exactly 256 protected-ID slots and counts within the
+global cap; checked configuration arithmetic SHALL require a nonzero remaining
+general pool. Remote ads MUST NOT consume reserve partitions. A serving
+transition SHALL atomically allocate/reuse one partition for its subnet before
+listener or author activation and fail with a typed capacity error when none is
+available. Stopping service SHALL retain that partition until all its protected
+local IDs expire, then release it atomically when that subnet is not serving, so
+sequential subnet churn cannot overwrite protection.
+
+Startup SHALL validate the partitions independently: general protected IDs MUST
+fit the global general capacity and each general per-subnet quota; local IDs MUST
+fit 256 slots for each distinct reserved subnet and the configured partition
+count. Persisted local IDs already occupy their reserve and MUST NOT be counted
+again as general state. Locally authored IDs MAY NOT evict protected IDs. The
+reserve prevents remote admission from blocking allocated local cadence, but
+does not provide preferential validation or remote role privilege.
+
+Fresh IDs may refresh addresses subject to caps. Address eviction SHALL choose
+expired entries first, then earliest expiry, then lexical key. Stores MUST NOT
+dial advertised addresses or record ad sources, queriers, query history, or
+source-peer/subnet associations. Replay IDs SHALL remain globally keyed; each
+record SHALL bind its advertised subnet only for quota/protection accounting, so
+reuse of one ad ID under another subnet is still a duplicate. A local-author
+reserve record necessarily marks an ID as generated by this process; that local
+fact and reserve occupancy/use/failure/timing MUST NOT enter wire messages,
+RPC, status, metrics, telemetry, or peer-linked state, even as aggregate counters.
+Fresh forged IDs can still poison within bounds; the store provides no authenticity.
+
+Acceptance SHALL atomically commit the global seen-ID record, quota/reserve
+accounting, address records, and public-index mutation before any relay job is
+enqueued. Commit failure SHALL cause no mutation or relay. Rollback of the
+database to a snapshot before that commit can remove the seen ID and permit a
+later replay; this capability makes no non-rollbackable replay guarantee.
+
+Store state SHALL be normalized per `(SubnetId, canonical address)`. Accepting
+a fresh ad updates visibility and expiry for every address present in that ad;
+addresses absent from it retain their current record until independently
+updated, expired, or evicted. An ID is publicly enumerable iff at least one
+live normalized record is marked public. Public-to-non-public and reverse
+updates of the same address take effect atomically. Direct lookup returns all
+live records regardless of visibility.
+
+Persistent replay state SHALL use unsigned 64-bit monotonic epoch ticks in
+seconds. Each seen-ID record stores its checked deadline tick, and one atomic
+metadata checkpoint stores elapsed tick every 300 seconds by default, no less
+often than every 600 seconds, and on clean shutdown. On restart, a record with
+`deadline <= checkpoint` SHALL be expired without subtraction. Otherwise the
+implementation SHALL use checked subtraction, reject a delta greater than
+173,400 seconds as incoherent, clamp valid remaining duration to 172,800
+seconds, and use checked duration conversion and `Instant` deadline addition.
+Overflow, underflow, missing/incoherent epoch metadata, or failed deadline
+construction SHALL return a typed startup error.
+
+If a durable checkpoint cannot complete before the 600-second maximum interval,
+the persistent store SHALL reject fresh ad acceptance and local authoring until
+a checkpoint succeeds or controlled shutdown completes; it MUST NOT continue
+creating deadline deltas outside the validated bound.
+
+All surviving records and new epoch metadata SHALL replace the old epoch in one
+atomic batch; interruption leaves the old epoch loadable. Downtime and the
+uncheckpointed interval are not subtracted, so they MAY extend a record present
+in the loaded database, but restart MUST NOT reset every such ID to a fresh full
+horizon or shorten it. A database rollback before seen-ID commit can remove the
+record entirely and is explicitly outside that guarantee. Address records
+continue to use persisted wall expiry and MAY expire conservatively on clock
+anomalies. If separately validated general/reserve capacities cannot contain
+valid persisted protected IDs, startup SHALL fail without eviction.
+
+Malformed or unverifiable persisted seen-ID, quota/reserve, or epoch state SHALL
+fail startup. Malformed address records MAY be quarantined and a public index MAY
+be rebuilt only when authoritative replay/accounting state remains intact.
+
+#### Scenario: Duplicate does not refresh
+
+- **WHEN** a retained ad ID is replayed
+- **THEN** original expiry remains and no second relay occurs
+
+#### Scenario: Ad ID is reused for another subnet
+
+- **WHEN** a retained global ad ID appears with a different subnet ID
+- **THEN** it remains a duplicate and does not consume that subnet's quota or
+  mutate/relay addresses
+
+#### Scenario: Protected dedup set is full
+
+- **WHEN** a fresh ad arrives while every dedup slot is protected
+- **THEN** the fresh ad is rejected instead of evicting a protected ID
+
+#### Scenario: One subnet fills its protected-ID quota
+
+- **WHEN** fresh remote ads for one subnet consume every unexpired slot in that
+  subnet's general quota
+- **THEN** another fresh ad for that subnet is rejected without consuming slots
+  reserved for other subnets or local authoring
+
+#### Scenario: Remote flood reaches the local-author reserve
+
+- **WHEN** the remote general pool is full while local authoring remains active
+- **THEN** a locally authored ad may use its subnet reserve and no remote ad may
+  consume that slot
+
+#### Scenario: Sequential serving exhausts reserve partitions
+
+- **WHEN** stopped subnets with protected local IDs occupy every configured
+  reserve partition and another subnet requests serving
+- **THEN** transition fails before listener/author activation without evicting
+  or shortening any occupied partition
+
+#### Scenario: TTL expires without probe
+
+- **WHEN** an address reaches local expiry without a fresh accepted ad
+- **THEN** it is no longer returned and no address probe occurred
+
+#### Scenario: Restart preserves address expiry conservatively
+
+- **WHEN** durable state reloads before expiry with a non-rollback wall clock
+- **THEN** only remaining lifetime is restored as a monotonic deadline
+
+#### Scenario: Restart restores checkpointed protection
+
+- **WHEN** a valid persisted seen ID loads after restart
+- **THEN** its checked positive deadline/checkpoint difference is restored
+  conservatively rather than resetting it to the full horizon
+
+#### Scenario: Deadline equals checkpoint
+
+- **WHEN** a persisted deadline tick is equal to or below the checkpoint tick
+- **THEN** the record expires without unsigned subtraction or revival
+
+#### Scenario: Epoch arithmetic is incoherent
+
+- **WHEN** subtraction/addition would underflow/overflow or a stored delta
+  exceeds 173,400 seconds
+- **THEN** startup returns a typed error without clamping wrapped arithmetic
+
+#### Scenario: Crash precedes the next checkpoint
+
+- **WHEN** a process crashes less than 600 seconds after its last checkpoint
+- **THEN** records present in the loaded database may be extended by the
+  uncheckpointed interval and downtime but are not shortened
+
+#### Scenario: Store rolls back before acceptance commit
+
+- **WHEN** an operator restores a database snapshot predating an accepted ad ID
+- **THEN** that ID may be accepted and relayed again, and documentation does not
+  claim rollback-resistant replay suppression
+
+#### Scenario: Reduced capacity cannot hold protected state
+
+- **WHEN** configured dedup capacity is below valid persisted seen-ID count
+- **THEN** startup fails without evicting a protected ID
+
+### Requirement: Gossip is bounded without anonymity overclaim
+
+Persistent participants SHALL relay each newly accepted ad without changing
+identifying contents. Relay fanout MUST be finite and no greater than 64.
+Queued relay work MUST be bounded and duplicates MUST NOT be requeued. Own ads
+SHALL be authored only on a fixed 1,800-second base cadence with independently
+sampled uniform jitter from -600 through +600 seconds. This cadence is not
+configurable in version one. Authored lifetime SHALL default to 7,200 seconds
+and MUST NOT exceed 86,400 seconds. Subnet start, listener start, and new overlay
+channels MUST NOT trigger authoring.
+
+A transient MAY relay newly accepted ads from bounded memory but SHALL NOT
+author one. No author field means an immediate sender is not protocol-level
+proof of authorship; this MUST NOT be described as hiding authorship against
+timing, topology, first-seen, or global observation.
+
+#### Scenario: Relay preserves contents
+
+- **WHEN** an accepted ad is relayed
+- **THEN** subnet ID, visibility, ad ID, lifetime, and addresses are unchanged
+
+#### Scenario: Gossip loop
+
+- **WHEN** the same ad returns during its protected horizon
+- **THEN** no second relay is queued
+
+#### Scenario: Start remains silent
+
+- **WHEN** serving or a new overlay channel starts
+- **THEN** authoring waits for the cadence
+
+### Requirement: Lookup and optional public enumeration are paginated
+
+Direct lookup SHALL name one subnet and return addresses only for it. Each page
+SHALL echo the request ID, contain at most 64 addresses and 65,536 encoded
+bytes, and include at most one fixed cursor. On the first page, the responder
+SHALL capture the current greatest live ordered key as the terminal key. A next
+cursor SHALL contain the last returned key and that fixed terminal key. Later
+pages SHALL return only live keys strictly greater than the last key and no
+greater than the terminal key, advancing the last key monotonically. A cursor
+whose version, length, or key ordering is invalid SHALL return a bounded invalid-
+cursor error. Index mutation MUST NOT invalidate a well-formed cursor, create a
+server snapshot, or force traversal restart; it MAY cause records added/removed
+during traversal to be included or omitted. Page limits still bound completion.
+For every page, the responder SHALL derive canonical keys, return each key at
+most once in strictly ascending order, and keep every key in
+`(previous_last, terminal]` when a previous cursor exists. A next cursor MUST be
+absent on an empty page and otherwise its last key MUST equal the greatest
+returned key with `last < terminal`. The requester SHALL independently derive
+and validate those keys, reject duplicates within/across pages, reject an empty
+page with a next cursor, and reject a response cursor that changes the first
+page's terminal, fails to advance, or disagrees with returned keys. Requester
+dedup state remains bounded by page/item limits.
+
+Public enumeration SHALL be disabled by default and require explicit local
+enablement. If enabled, it SHALL return IDs having at least one live normalized
+address record marked public, at most 256 IDs and 16,384 bytes per page. It SHALL
+be available to every protocol-correct connected peer without role privilege and
+MAY be disabled globally without disabling direct lookup.
+Visibility is not authenticated: an attacker can cause an observed ID to
+appear by submitting a public-marked address record. No response SHALL include
+ad sources or querier data.
+
+#### Scenario: Direct lookup is isolated
+
+- **WHEN** addresses for S are requested
+- **THEN** response pages contain S addresses only within both page bounds
+
+#### Scenario: Non-public record is omitted
+
+- **WHEN** an ID has only accepted non-public records
+- **THEN** it is omitted while direct lookup remains possible to a caller
+  already knowing the ID
+
+#### Scenario: Enumeration setting is omitted
+
+- **WHEN** an operator does not explicitly enable public enumeration
+- **THEN** public-list requests return the bounded disabled error while direct
+  lookup remains available
+
+#### Scenario: Index changes during pagination
+
+- **WHEN** records are inserted, updated, expired, or removed before the next
+  page
+- **THEN** traversal continues strictly after the prior key up to the original
+  terminal key without restart or snapshot state
+
+#### Scenario: Insertions sort after the initial terminal
+
+- **WHEN** new records sort after the terminal key captured on the first page
+- **THEN** they cannot extend that traversal and require a later lookup
+
+#### Scenario: Hostile response does not advance semantically
+
+- **WHEN** a responder returns duplicate/unordered/out-of-window items, changes
+  the terminal, or advances a cursor on an empty page
+- **THEN** the requester rejects the page without adding candidates or
+  continuing from that cursor
+
+### Requirement: Role boundaries and version features are validated
+
+A persistent participant SHALL advertise exactly
+`("swarm-ad-store", 1)`, maintain durable bounded state, and relay ads. A node
+without it SHALL be treated as transient. The feature MUST NOT grant privilege.
+Local and remote version messages SHALL allow at most 10 external addresses and
+10 features; node ID SHALL be at most 64 encoded bytes, app name at most 32,
+semver prerelease and build strings at most 32 each, every URL at most 1,024,
+and every feature name at most 32. Complete outgoing `VersionMessage` and
+`VerackMessage` encoded-size validation MUST succeed before send. Inbound
+decoding of both messages MUST check every declared variable length/count,
+including semver strings, before reservation or allocation while preserving
+existing field order and valid wire encoding byte-for-byte.
+
+A transient SHALL accept no inbound overlay connections, author no ads, and
+persist no swarm-overlay ad/query/history state. It MAY keep bounded in-memory
+ads and the bounded successful-endpoint-only cache. Subnet lifecycle persistence and
+transport-managed state are separate scopes and MUST be documented separately.
+Both roles apply identical decoding, validation, authorization, and work
+bounds.
+
+#### Scenario: Claimed role grants no privilege
+
+- **WHEN** a malicious peer self-declares the persistent feature
+- **THEN** it receives no validation, query, metering, or storage exemption
+
+#### Scenario: Complete version message is oversized
+
+- **WHEN** otherwise valid local fields combine into an oversized outgoing
+  version message
+- **THEN** it is rejected before send
+
+#### Scenario: Remote feature count is oversized
+
+- **WHEN** an inbound payload declares more than 10 features despite fitting
+  the total payload bound
+- **THEN** decoding rejects it before reserving the declared vector capacity
+
+#### Scenario: Remote semver string is oversized
+
+- **WHEN** inbound version or verack declares an overlong prerelease/build
+  string within the total payload bound
+- **THEN** decoding rejects it before allocating the declared string
+
+#### Scenario: Complete verack is oversized
+
+- **WHEN** local app/version fields would exceed `VERACK_MAX_BYTES`
+- **THEN** verack is rejected before send
+
+#### Scenario: Transient overlay persistence
+
+- **WHEN** a transient disconnects
+- **THEN** swarm-overlay state retained by the module is at most its bounded
+  successful connect-URL/resolved-endpoint cache, while separately configured
+  subnet/transport state follows its own documented policy
+
+### Requirement: Resource accounting covers amplification paths
+
+Every message type SHALL have hard per-channel metering. In a 10-second window,
+one channel SHALL accept at most 32 ads, 16 direct lookup requests, 16 direct
+responses, 4 public-list requests, 4 public-list responses, and 16 bounded
+errors before strict-policy delay/penalty. It SHALL also receive at most 32
+store-write and 32 relay-enqueue work tokens per 10 seconds and at most
+1,048,576 response bytes per 60 seconds.
+
+Variable fields SHALL be validated before allocation or work. Configured local
+limits SHALL use these defaults and MUST NOT exceed these maxima:
+
+| Resource | Default | Maximum |
+|---|---:|---:|
+| relay queue jobs | 1,024 | 4,096 |
+| concurrent durable writes | 8 | 32 |
+| concurrent query reads | 16 | 64 |
+| concurrent relay workers | 8 | 32 |
+| pages consumed per direct join lookup | 16 | 16 |
+| pages consumed per public enumeration | 4 | 16 |
+| candidate addresses per join attempt | 64 | 256 |
+| previously compatible retry attempts | 16 | 64 |
+| persisted compatible retry URLs per subnet | 64 | 256 |
+| local-author reserve subnet partitions | 32 | 256 |
+| active subnets | 32 | 256 |
+| concurrent lifecycle attempts | 8 | 32 |
+| shutdown deadline seconds | 120 | 600 |
+| pending-request timeout seconds | 10 | 60 |
+| configured ordinary overlay peers | 8 | 256 |
+| overlay bind/listener addresses | 1 | 16 |
+| serving bind addresses per subnet | 1 | 16 |
+| serving external addresses per subnet | 1 | 32 |
+| overlay inbound channels | 64 | 256 |
+| overlay outbound channels | 8 | 64 |
+| overlay manual channels | 8 | 256 |
+| total established overlay channels | 80 | 512 |
+| untrusted dial concurrency | 4 | 16 |
+| untrusted dial starts per minute | 32 | 128 |
+| DNS resolutions per join attempt | 64 | 256 |
+| dials per resolved destination per attempt | 1 | 1 |
+
+Overlay instances SHALL use strict ban policy. Queue/concurrency/retry settings
+outside these ranges SHALL be rejected at configuration time.
+
+A small request MUST NOT induce an unbounded response, write, relay, allocation,
+or outbound connection. Budget accounting SHALL use ephemeral channel IDs, not
+peer addresses, and SHALL be removed on disconnect.
+
+#### Scenario: Query amplification is bounded
+
+- **WHEN** a channel floods minimal valid requests
+- **THEN** pending state, response bytes, and processing stay bounded and strict
+  penalties apply
+
+#### Scenario: Advertisement cannot induce dialing
+
+- **WHEN** an ad contains an attacker-selected shareable address
+- **THEN** accepting, storing, relaying, expiring, or reporting it opens no
+  connection to that address
+
+### Requirement: Subnet joining alone validates advertised addresses
+
+Lookup results SHALL remain ephemeral typed original-URL/resolved-socket targets
+for the requested subnet until compatibility succeeds; they MUST NOT enter a
+URL-only host/refinery set first. A joining subnet MUST apply ordinary magic,
+application-name, and major/minor checks before treating a peer as compatible.
+Failure MUST drop the candidate, remain fallible, and MUST NOT penalize the
+overlay relay. Passing compatibility does not authenticate an operator or
+authorize application access. Future ordinary reconnect/refinement attempts
+MUST independently resolve and validate a new exact target.
+
+Before resolution or dialing, the joiner SHALL place previously compatibility-
+verified persisted ordinary peers in one tier and fresh overlay URLs in another.
+The Swarm retry index itself SHALL contain at most the configured persisted-
+compatible cap; when full, a newly compatible URL remains usable for its current
+session but MUST NOT evict an existing retry URL merely to enter that index.
+
+A direct join lookup SHALL continue through the first page's terminal until no
+next cursor remains or its fixed 16-page cap is consumed. It SHALL use bounded
+`OsRng` reservoir sampling across every valid URL returned in that traversal to
+select at most the candidate-address cap, rather than truncating an ordered
+prefix when candidate capacity is reached. The full bounded persisted index and fresh
+reservoir SHALL then be independently shuffled with `OsRng`; wire, store, URL,
+hash, DNS-answer, or lexical order MUST NOT choose the attempted prefix.
+
+After both URL tiers are built, candidate preparation SHALL consume no more than
+half the then-remaining overall deadline; verified and fresh resolution/
+validation SHALL each have half that subdeadline. Unused verified time MAY be
+donated to fresh preparation but not conversely. Every persisted peer MUST
+still undergo fresh resolution and egress validation.
+
+Already validated targets SHALL be installed as a two-phase pre-start manual
+plan. At subnet start, the remaining candidate-dial duration SHALL be split at a
+monotonic midpoint. The verified phase MUST stop/cancel by that midpoint and
+consume no more than its configured retry limit or half the total candidate-
+attempt budget. Fresh targets activate for the second half and retain at least
+half the attempt capacity; if the verified phase is empty, fresh dialing MAY
+begin immediately. Every connector uses the exact validated target. No fresh
+candidate is persisted before compatibility.
+
+#### Scenario: Wrong subnet is rejected
+
+- **WHEN** an advertised peer fails a bound compatibility field
+- **THEN** it does not enter the verified ordinary subnet peer set
+
+#### Scenario: Relay is not blamed
+
+- **WHEN** a relayed address fails subnet connection
+- **THEN** the immediate overlay relay is not treated as author
+
+#### Scenario: Attacker grinds lexical address order
+
+- **WHEN** an overlay response contains many addresses chosen to sort before an
+  honest candidate
+- **THEN** the client shuffles the complete bounded fresh tier with `OsRng`
+  before resolution/dial selection, so lexical order does not choose the budget
+
+#### Scenario: Previously compatible tier is large
+
+- **WHEN** persisted compatible peers exceed their retry limit
+- **THEN** the bounded full index is shuffled, a subset consumes at most half
+  the attempt budget and first half of dial time, and fresh overlay candidates
+  retain the remainder
+
+#### Scenario: Ordered lookup exceeds candidate capacity
+
+- **WHEN** terminal-bounded lookup returns more URLs than the candidate cap
+- **THEN** bounded CSPRNG reservoir sampling covers every URL returned through
+  terminal completion or the 16-page cap instead of taking its lexical prefix
+
+### Requirement: Metadata disclosure and identity scoping are explicit
+
+Wire messages MUST NOT intentionally bind different subnets to one stable node
+or signing identity. Durable state MUST NOT contain querier identity or query
+history. An answering peer nevertheless observes the requested ID; requests on
+one channel are linkable; timing, topology, public enumeration, and endpoint
+reuse are metadata surfaces. The capability MUST NOT claim PIR, guaranteed
+origin anonymity, absence of remote traces, or global-observer resistance.
+
+Gossip and store peers necessarily observe and MAY retain every subnet-ID to
+advertised-endpoint mapping they receive. The forbidden provenance association
+is a mapping from overlay source peer to subnet/ad authorship; the rendezvous
+ID-to-endpoint mapping is intentional protocol output and is not confidential.
+
+Every overlay and subnet `P2p` instance SHALL use an independently CSPRNG-
+generated `VersionMessage.node_id`; it MUST NOT be persisted or reused across
+instances, subnets, overlay/subnet roles, or process restart.
+
+Serving documentation SHALL identify endpoint reuse as directly linkable and
+SHALL NOT claim automatic independent Tor/I2P provisioning.
+
+#### Scenario: Query disclosure is documented
+
+- **WHEN** a lookup for S is issued
+- **THEN** documentation states the answering peer observes S
+
+#### Scenario: Overlay and subnet version identities
+
+- **WHEN** one process starts an overlay and one or more subnet `P2p` instances
+- **THEN** their version node IDs are independently generated and unequal
+
+#### Scenario: Shared endpoint is linkable
+
+- **WHEN** one endpoint is advertised for two subnets
+- **THEN** guidance identifies the direct link and makes no contrary claim

openspec/changes/swarm/tasks.md

@@ -1,136 +1,385 @@
-# Tasks: swarm overlay for subnet discovery
-
-## 1. Module foundation
-
-- [ ] 1.1 Create `src/swarm/` module skeleton (`mod.rs`, `settings.rs` with
-  `SwarmSettings` for overlay + role configuration), export it from the
-  `darkfi` crate, and verify `make` compiles with `make clippy` clean.
-
-## 2. SubnetId derivation (D2, spec: swarm-overlay)
-
-- [ ] 2.1 Implement `SubnetDescriptor` (app_name, magic_bytes,
-  version_constraint, optional secret) with canonical serialization and
-  BLAKE3 `SubnetId` derivation; verify unit tests pass: identical
-  descriptors yield equal ids, any field divergence yields different ids,
-  secret-bearing ids are the same length and format as public ones.
-- [ ] 2.2 Add a pinned-descriptor constructor from an existing
-  `{app_name, magic_bytes, version}` network triple; verify a golden-value
-  unit test fixes the darkirc triple's id so all deployments agree.
-
-## 3. Minimal `src/net` additions (security zone: additive only)
-
-- [ ] 3.1 Plumb a features list from `net::Settings` into the
-  `VersionMessage` handshake (currently hardcoded to `vec![]` in
-  `protocol_version.rs`); verify a version-exchange test round-trips the
-  feature and that the diff contains no other `src/net` changes (flag this
-  diff explicitly in the change notes for review).
-- [ ] 3.2 Verify existing net behavior is unchanged:
-  `cargo test -p darkfi --release --all-features net` passes and
-  `make clippy` is clean.
-
-## 4. Overlay wire messages (D4)
-
-- [ ] 4.1 Define `SubnetAd`, `GetSubnets`, `Subnets`, `GetSubnetAddrs`,
-  `SubnetAddrs` via `impl_p2p_message!` with `MAX_BYTES` estimates and
-  metering configurations in the existing style; verify serialization
-  roundtrip and size-estimate unit tests pass.
-
-## 5. Advertisement store (D3, spec: swarm-overlay)
-
-- [ ] 5.1 Implement the ad store: per-subnet and total entry caps, TTL
-  expiry, shareable-scheme filtering, disk persistence, and no recording of
-  querier data; verify unit tests pass for TTL expiry, cap enforcement,
-  restart persistence, and non-shareable-scheme rejection.
-- [ ] 5.2 Implement the ad refinery task: rate-limited liveness checks of
-  advertised addresses with drop/downgrade on failure; verify an integration
-  test shows a dead advertised address is dropped before TTL expiry and
-  checks are not issued in a simultaneous burst.
-
-## 6. ProtocolSwarm: gossip and queries (D4, D8)
-
-- [ ] 6.1 Implement `ProtocolSwarm` gossip handling: validate `SubnetAd`
-  (one subnet, shareable schemes), insert into the ad store, relay
-  unmodified with no provenance or hop fields; register via
-  `ProtocolRegistry` on the overlay's outbound and inbound sessions; verify
-  a test asserts relayed ads are unchanged and non-shareable addresses are
-  dropped rather than relayed.
-- [ ] 6.2 Implement `GetSubnets`/`GetSubnetAddrs` query handling answered
-  from local state for every connected peer; verify a test shows a
-  persistent and a transient node with identical local state answer with
-  the same message types and semantics.
-- [ ] 6.3 Implement the jittered ad re-gossip cadence with no
-  event-triggered emission; verify tests show no ad is sent on subnet
-  start, listener startup, or new overlay connection, and that the cadence
-  tick does emit.
-
-## 7. Swarm lifecycle and roles (D1, D5, D6, D8)
-
-- [ ] 7.1 Implement `Swarm` owning the overlay `P2p` plus the subnet
-  registry, with persistent and transient constructors (persistent:
-  `swarm-store` handshake feature via task 3.1, high inbound; transient:
-  zero inbound, minimal outbound, optional overlay hostlist cache and
-  nothing else on disk); verify a transient integration test writes only
-  the overlay hostlist cache and emits no ads.
-- [ ] 7.2 Implement `join(descriptor)`: overlay resolution, greylist
-  seeding of resolved addrs, subnet `P2p` spawn with `SubnetId`-namespaced
-  datastore/hostlist paths, app protocol registration before start; verify
-  a descriptor-only join succeeds against a local overlay seed with no
-  subnet seed configuration, and unknown subnets fail in bounded time.
-- [ ] 7.3 Implement `serve()`: per-subnet opt-in requiring that subnet's
-  inbound addrs, advertising only those addrs on the 6.3 cadence; verify
-  tests show served-subnet ads carry only that subnet's addresses and
-  join-only mode never emits an ad.
-- [ ] 7.4 Implement `leave()`: stop and deregister the subnet `P2p` with no
-  departure message and state retained for rejoin; verify tests assert no
-  overlay message is emitted on leave and rejoin resumes from the persisted
-  hostlist.
-- [ ] 7.5 Verify runtime lifecycle tests pass: late subnet spawn does not
-  disturb running subnets or the overlay, and concurrent teardown of
-  several subnets terminates cleanly.
-- [ ] 7.6 Implement transient cache-first bootstrap: the overlay hostlist
-  cache is persisted between sessions and dialed before configured seeds,
-  with cache contents limited to overlay peer addresses; verify an
-  integration test where a second session connects through cached peers
-  with all seeds unreachable, and a test asserting the cache contains no
-  subnet identifiers or query records.
-
-## 8. Multi-node integration tests (spec scenarios)
-
-- [ ] 8.1 Cold-start chain test over local transports: overlay seed →
-  persistent node → transient node discovers and joins a subnet with no
-  per-subnet seeds anywhere; verify the transient node connects into the
-  subnet and leaves no trace in any hostlist or ad store.
-- [ ] 8.2 Abuse tests: an ad flood exceeds store caps and triggers
-  metering/ban penalties, and poisoned (unreachable) addrs are dropped by
-  refinement; verify both behaviors in one integration test.
-- [ ] 8.3 Linkability structure test: one node serving two subnets through
-  distinct addresses emits ads that share no field linking them; verify by
-  structural inspection of all emitted messages in a two-subnet test.
-
-## 9. Lilith overlay seed (D7, spec: lilith-overlay-seed)
-
-- [ ] 9.1 Add the `[overlay]` config section with parsing, overlay-only
-  mode, and mixed overlay + legacy per-network operation; verify config
-  parsing unit tests (overlay-only, mixed, malformed) and a mixed-config
-  integration test where both the overlay seed and legacy nets run.
-- [ ] 9.2 Wire the durable ad store and refinery into lilith; verify a
-  restart test shows persisted ads survive a stop/start cycle within TTL
-  bounds and no querier data is stored.
-- [ ] 9.3 Retarget lilith's RPC: listener health, participating subnet ids,
-  ad store stats, with no per-querier data; verify a JSON-RPC test asserts
-  the reported fields and the absence of query-source information.
-
-## 10. Pilot adoption, docs, gates
-
-- [ ] 10.1 darkirc pilot behind a config flag: pinned descriptor via 2.2,
-  `Swarm`-based networking with static-seed fallback preserved; verify
-  tests exercise both the overlay path and the fallback path.
-- [ ] 10.2 Write deployment guidance: onion-per-subnet recommendation,
-  shared-endpoint linkability hazard (required by the no-cross-subnet
-  linkability spec requirement), and transient battery-vs-mixing
-  recommendation to hold the overlay connection for the app session.
-- [ ] 10.3 Full gates green: `make`, `make clippy`, `make test`, and
-  `make fmt` all complete without errors.
-- [ ] 10.4 Invoke `@anon-security-review` on the full diff and record the
-  verdict on the change; a FAIL is blocking — resolve findings or escalate
-  before marking the change ready to apply.
+# Tasks: swarm overlay for subnet rendezvous
+
+## 1. Review checkpoint and module foundation
+
+- [ ] 1.1 Obtain human review of the `src/net/swarm/` design, resolved-target
+  dial work, and Cargo activation of existing `blake3`/`kvdb-overlay`;
+  stop for separate supply-chain review if any new dependency, dependency
+  source, `build.rs`, or proc-macro is needed.
+- [ ] 1.2 Add the gated `swarm` feature and `src/net/swarm/` module structure
+  without enabling existing binaries; verify default, `net`, and `swarm`
+  feature combinations with `make check` and clean `make clippy`.
+- [ ] 1.3 Implement validated role, fixed overlay identity, protocol/local
+  maxima, timeout, source-policy, persistence, and serving settings; add zero,
+  over-maximum, conflicting-identity, and invalid-role tests, then run
+  `make test`.
+
+## 2. Canonical descriptors
+
+- [ ] 2.1 Implement exact version-1 descriptor validation/encoding with manual
+  domain, lengths, big-endian integers, 32-byte app-name bound, and secret shape; add app-name,
+  flag/secret, UTF-8 byte, and excluded-version-field tests, then run
+  `make test`.
+- [ ] 2.2 Implement BLAKE3 `SubnetId` and the normative darkirc byte/hash golden
+  vector plus bound-field divergence and patch-equivalence tests; run
+  `make test`.
+- [ ] 2.3 Add `OsRng` private-secret generation and secret-safe debug/error
+  behavior; verify overlay-facing values contain only derived IDs with
+  `make test`.
+- [ ] 2.4 Prove descriptors and derived IDs are rendezvous inputs only: knowing a
+  private secret/ID and passing network compatibility MUST NOT grant application
+  authorization. Add compatible-but-app-unauthorized and ID-only access tests,
+  then run `make test`.
+
+## 3. Shared networking prerequisites
+
+- [ ] 3.1 Add bounded local version features to `net::Settings`, default empty,
+  enforcing 10 external addresses, 10 features, 64-byte node ID, 32-byte app
+  name, 1,024-byte URL, 32-byte feature name, and 32-byte semver
+  prerelease/build plus duplicate/version checks; add every boundary test and
+  run `make test`.
+- [ ] 3.2 Plumb features into `VersionMessage` without changing compatibility;
+  check complete outgoing version and verack against their maxima and implement
+  bounded inbound decoders for both that check every string/vector—including
+  semver strings—before reservation/allocation while preserving valid wire
+  bytes. Add golden compatibility, huge declared count/string, overlong
+  element, roundtrip, and combined-oversize tests. Generate each P2p node ID with
+  `OsRng`; test overlay/subnet/restart independence and no persistence/reuse,
+  then run `make test`.
+- [ ] 3.3 Record focused human review that shared-net changes alter neither
+  framing, magic, version compatibility, nor manual/seed/inbound reload
+  behavior, and that transport changes are limited to the explicit
+  validated-resolved-target path in 3.4.
+- [ ] 3.4 Add a checked `ValidatedDialTarget` path that resolves clearnet once,
+  rejects prohibited IPv4/IPv6 ranges outside explicit local-test mode, dials
+  the exact socket without DNS re-resolution, preserves hostname only for TLS,
+  and rejects arbitrary proxy-remote clearnet DNS. For Tor/I2P, make the exact
+  socket the trusted locally configured proxy, never ad-selected; pass canonical
+  hidden names only inside proxy negotiation/TLS and never local DNS. Add a
+  pre-start `ManualSession` target API carrying connect URL plus resolved socket
+  plus a two-phase pre-start plan that cancels verified targets at a monotonic
+  switch and activates fresh targets without reload. Add API-state/phase-cancel,
+  IPv4/IPv6 range, cached-no-DNS, rebinding, TLS-name, trusted-proxy socket,
+  hidden-service no-local-DNS, proxy override/bypass, missing proxy, and
+  production-lilith policy tests, then run `make test` and obtain focused
+  transport/privacy review.
+- [ ] 3.5 Audit the full advertised-target path—URL/scheme/host/port, empty,
+  bounded-multiple, and over-16 DNS results, target construction, every accepted
+  transport, proxy negotiation, TLS, timeout/cancellation, and compatibility—and
+  remove or bypass all `unwrap`, `expect`, panic, unchecked indexing, and
+  unimplemented branches. Budget/classify every DNS answer and select at most
+  one exact socket per URL. Reject unaudited schemes before dialer construction;
+  add arbitrary-input and no-unwind tests for every accepted scheme, then run
+  `make test` and `make clippy` and obtain focused transport review.
+
+## 4. Bounded correlated wire protocol
+
+- [ ] 4.1 Implement nonempty `SubnetAd` with exact ID, visibility, ad ID,
+  lifetime, 1..=32 address, 1,024-byte URL, shareable-scheme, and 65,536-byte
+  message limits plus fixed command/field order and visibility values; add
+  serialization/golden-command and every-boundary test, then run `make test`.
+- [ ] 4.2 Implement fresh 16-byte request IDs, fixed 65-byte version/last-key/
+  terminal-key cursors, direct lookup messages, fixed command/field order,
+  defined error values 0..=3, and a 10-second default/60-second maximum local
+  request timeout. Derive canonical response keys and validate strict ordering,
+  uniqueness, `(last, terminal]` windows, exact next-last, unchanged terminal,
+  and no next cursor on empty pages; deduplicate across pages under item/page
+  caps. Add roundtrip, golden-command, malformed/key-order/window/terminal/
+  empty-advance cursor, duplicate-page, invalid error-value, wrong-ID/type,
+  local timeout/late response, and unsolicited-response tests; run `make test`.
+- [ ] 4.3 Implement optional public enumeration messages with request
+  correlation, explicit-enable/default-disabled policy, 256-ID/16,384-byte pages,
+  no role privilege, and live normalized public-record filtering; add omitted-
+  config default-disabled, explicit-enabled, non-public-only omission, mixed
+  records, visibility transitions, attacker public-relabel, pagination, and byte-
+  bound tests, then run `make test`.
+- [ ] 4.4 Implement a 32-entry per-channel pending-request map with timeout and
+  disconnect cleanup; add concurrent identical lookup and saturation tests,
+  then run `make test`.
+- [ ] 4.5 Define hard `MAX_BYTES` and metering for every swarm message, checking
+  encoded bytes/counts before variable allocation/store work; add metering and
+  small-message amplification tests, then run `make test`.
+- [ ] 4.6 Audit every attacker-controlled swarm/version/verack decoder for
+  fallible checked reads and absence of `unwrap`, `expect`, explicit panic,
+  unchecked slicing/indexing, or allocation before declared-size validation;
+  add truncation-at-every-byte, hostile count/length, arbitrary-input,
+  no-unwind, and bounded-allocation tests for every message, then run
+  `make test`.
+
+## 5. Passive advertisement stores
+
+- [ ] 5.1 Implement bounded in-memory address/public indexes, canonical URL
+  keys, one normalized visibility/expiry record per address, public membership
+  iff any live record is public, stateless last/terminal-key traversal, and
+  deterministic eviction; add absent-address retention, same-address visibility
+  transition, mixed record, cap, ordering, expiry, continuous mutation, no-
+  restart progress, insertion-after-terminal, responder terminal-change/non-
+  advancing cursor, and cursor-key validation tests, then run `make test`.
+- [ ] 5.2 Implement monotonic expiry with a 7,200-second receive/author default,
+  86,400-second hard maximum, receiver clamp, unchanged relay wire lifetime, and
+  duplicate handling that never refreshes or requeues; add paused-clock,
+  default/max/clamp, relay-preservation, duplicate/fresh-ID, and expiry tests,
+  then run `make test`.
+- [ ] 5.3 Implement protected seen-ID retention through local address expiry
+  plus 86,400 seconds, a 256-default/1,024-maximum general quota per subnet, and
+  32-default/256-maximum remote-inaccessible local-author subnet partitions of
+  256 slots each within the global cap. Use checked capacity arithmetic and
+  validate persisted general/local partitions separately without double count;
+  retain stopped-subnet partitions until their IDs expire. Reject fresh ads or
+  serving transitions before mutation/network activity when the applicable pool
+  is full. Add one-subnet, distributed-subnet, global, sequential serving churn,
+  local-reserve, stopped-subnet last-ID release/resume race, cadence-window,
+  local-reserve wire/RPC/status/metric/telemetry exclusion, positive restart
+  fixture where reserve IDs fit only when excluded from general accounting,
+  protected-ID, cross-subnet ad-ID reuse, expired-admission, and replay tests,
+  then run `make test`.
+- [ ] 5.4 Implement dual-clock ad-address expiry plus restart-safe dedup:
+  restore bounded remaining address lifetime and persist seen-ID deadlines on a
+  monotonic epoch with a 300-second default/600-second maximum atomic checkpoint.
+  Use `u64` ticks, compare before checked subtraction, expire equality, reject
+  deltas over 173,400, clamp valid remainder to 172,800, and checked-add the new
+  `Instant` deadline. Atomically restore records/new epoch without a full reset.
+  Add clean restart, equality/underflow/maximum/overflow, sub-checkpoint crash
+  extension, downtime, repeated restart, corrupt/missing epoch, interrupted
+  atomic conversion, and paused-clock healthy checkpoint-at-300-seconds tests.
+  Inject checkpoint delay/failure and prove fresh admission/authoring fail closed
+  by 600 seconds while bounded reads remain; then run `make test`.
+- [ ] 5.5 Implement `kvdb-overlay` trees and atomic address/public/dedup/metadata
+  updates; fail startup when configured pool/reserve caps cannot hold valid
+  persisted seen state and rebuild/verify the public index from normalized
+  records. Add restart, mixed visibility, reduced-cap/reserve failure, stateless
+  index, checkpoint, rollback-before-ID-commit, and interrupted-write tests,
+  proving accepted seen/address/index state commits before relay eligibility;
+  then run `make test`.
+- [ ] 5.6 Bound and fallibly decode persisted keys/values. Fail startup on
+  malformed/unverifiable seen-ID, quota/reserve, or epoch state; quarantine/
+  rebuild only address/public-index state when replay/accounting remains intact.
+  Test corrupt lengths, URLs, IDs, each authoritative class, and metadata with
+  `make test`.
+- [ ] 5.7 Keep store interfaces source-free and dialer-free; add a dial spy
+  proving accept, store, relay preparation, query, replay, expiry, saturation,
+  and restart never connect to advertised targets, then run `make test`.
+
+## 6. Protocol roles, relay, and work accounting
+
+- [ ] 6.1 Register `ProtocolSwarm` on `SESSION_DEFAULT` only; verify dispatch on
+  ordinary inbound/outbound/manual/direct channels and absence on seed/refine
+  channels with `make test`.
+- [ ] 6.2 Implement ad validation, store admission, bounded relay queue,
+  duplicate suppression, source exclusion by ephemeral channel ID, and
+  unchanged identifying contents; enqueue relay only after atomic seen/quota/
+  address/index commit. Add commit failure, rollback replay disclosure,
+  malformed, gossip-loop, saturation, fanout, and exclusion tests, then run
+  `make test`.
+- [ ] 6.3 Implement direct/public query handlers over ordered indexes and
+  correlated pending requests; direct lookup ignores visibility and enabled
+  enumeration grants no role privilege. Add continuous-mutation bounded progress,
+  terminal-key, non-snapshot omission/addition, disabled-public, transient
+  requester, no-cross-subnet, and unsolicited-response tests, then run
+  `make test`.
+- [ ] 6.4 Implement the exact per-channel message/work rates and
+  response-byte budget plus configured queue, global semaphore, page, candidate,
+  previously-compatible retry/index, local-author partition, active-subnet,
+  concurrent-attempt, and shutdown defaults/maxima plus request timeout,
+  configured peer, bind/external address, and
+  inbound/outbound/manual/total channel, dial concurrency/rate,
+  resolution-total, and per-destination maxima from `swarm-overlay`; reject
+  over-maximum config, delete channel accounting on disconnect, and add each
+  saturation/strict-penalty test before `make test`.
+- [ ] 6.5 Implement persistent/transient behavior with exact
+  `swarm-ad-store`, no feature privilege, durable versus bounded memory, no
+  transient authoring, and overlay-only persistence scope; add forged-feature
+  and equal-state response tests, then run `make test`.
+
+## 7. Fresh-instance ordinary-peer bootstrap
+
+- [ ] 7.1 Implement bounded endpoint-only cache records containing the exact
+  successful connect URL/resolved endpoint pair, with 256-record,
+  262,144-file-byte, 1,024-URL-byte, shareability, egress, and atomic replacement
+  limits; add malformed/oversized/truncated/content tests, then run `make test`.
+- [ ] 7.2 Cache only a completed ordinary channel's actual validated endpoint
+  after it exposes `swarm-ad-store`; never cache `ext_send_addr` or another
+  advertised address and persist no feature, ID, ad, query, or mapping. Add
+  malicious external-address and file-inspection tests, then run `make test`.
+- [ ] 7.3 Implement cached-peer stage using a fresh overlay `P2p` with
+  empty `Settings.peers`/`Settings.seeds` and install revalidated cached sockets
+  through the pre-start manual-target API; on timeout fully stop/discard it,
+  resolve configured peers once, and install them into a fresh stage. Do not use
+  `P2p::reload()` or DNS-resolve a cached hostname.
+- [ ] 7.4 Add stage-order, cleanup, timeout, no-overlap, no-seed-session, cached
+  success, and configured fallback integration tests; run `make test`.
+- [ ] 7.5 Test that failed cached app/protocol/store startup leaves no task or
+  state before the configured stage and that overall bootstrap remains bounded;
+  run `make test`.
+- [ ] 7.6 Implement transient overlay stop independently from subnet registry
+  lifetime. Default to session-bound retention with no lookup/join-triggered
+  stop; allow immediate teardown only via explicit reduced-privacy policy with
+  timing warning and deterministic stop after every caller-visible success,
+  empty, error, timeout, or cancellation outcome but not join's internal lookup;
+  reject stop while persistent store/gossip or serving-ad duties remain. Add
+  default-no-trigger, all terminal outcomes, internal-phase retention,
+  same-operator correlation, later reconnect, and subnet-survival tests with
+  `make test`.
+
+## 8. Registry-owned subnet lifecycle and source attempts
+
+- [ ] 8.1 Implement per-ID `Initializing`, `Joining`, `Joined`, `Serving`, and
+  `Stopping` ownership with serialized same-ID transitions and concurrent
+  different-ID operation; add legal/duplicate transition tests, then run
+  `make test`.
+- [ ] 8.2 Implement full-ID paths and isolated settings, hosts, refinement,
+  datastores, app state, and shutdown ownership; add cross-subnet isolation and
+  delete tests, then run `make test`.
+- [ ] 8.3 Implement the fallible initializer returning typed app state plus
+  shutdown; retain a type-erased `Arc` and hook in the registry so caller-handle
+  drop cannot end app state. Test pre-start ordering, drop ownership, and
+  initializer failure with `make test`.
+- [ ] 8.4 Keep overlay candidates as ephemeral typed original-URL/resolved-socket
+  targets through the subnet pre-start manual connector; do not insert URL-only
+  host/refinery state before compatibility. Drop failures, persist only after
+  compatible ordinary channel, and re-resolve/revalidate every later outbound/
+  retry/refine attempt. Bound the persisted-compatible index at 64 default/256
+  maximum; reservoir-sample across every URL returned by the terminal traversal
+  through completion or its fixed 16-page cap. Independently shuffle both tiers
+  and DNS answers. Split candidate-preparation resolution time equally, then
+  install a two-phase plan that cancels verified dialing at the midpoint of
+  remaining dial time; verified also consumes at most half the attempts. Add
+  source-cardinality, reservoir, lexical/hash/insertion/DNS-order grinding,
+  resolution/dial-time starvation, phase cancellation, DNS-query-count/exact-
+  socket, local/private/reserved, rebinding, repeated-victim, proxy-DNS, wrong-
+  magic/app/version, temporary-direct, seed/refine-without-peer, unreachable,
+  disconnect-race, and test-only injected-RNG tests; run `make test`.
+- [ ] 8.5 Enforce channel ownership separation: no overlay stream transfer,
+  subnet re-handshake, or subnet-tag multiplexing, even when the answering
+  overlay peer also serves the subnet; add structural and same-operator tests,
+  then run `make test`.
+- [ ] 8.6 Implement overlay-only, static-only, and combined attempts with
+  explicit source activation and per-attempt/overall deadlines; add policy and
+  compatibility tests, then run `make test`.
+- [ ] 8.7 Implement overlay-then-static as complete overlay-attempt rollback
+  followed by a fresh static-configured `P2p` and repeated initializer—without
+  manual/seed reload. Add state-leak, ordering, repeated-initializer,
+  static-success, and aggregate-failure tests, then run `make test`.
+- [ ] 8.8 Add fault injection at construction, initializer, insertion, start,
+  channel wait, timeout, cancellation, and shutdown; verify complete rollback
+  leaves other networks undisturbed with `make test`.
+- [ ] 8.9 Implement silent idempotent leave and explicit retain/delete policy;
+  add repeated leave, join/leave race, retained rejoin, and isolated deletion
+  tests, then run `make test`.
+- [ ] 8.10 Implement swarm shutdown ordering: stop authoring, cancel attempts,
+  stop every subnet despite errors, then overlay; add finite concurrent teardown
+  and failing-hook tests, then run `make test`.
+
+## 9. Initial serving, controlled recreation, and authoring
+
+- [ ] 9.1 Implement separate bind/external serving settings, persistent-role
+  validation, atomic local-author partition allocation before initialization/
+  listener/authoring, and pre-start listener configuration. Release a newly
+  allocated empty partition on pre-author failure; retain nonempty partitions
+  after stop. Add first-server-with-no-peer, reserve exhaustion/churn, forwarded
+  external endpoint, malformed field, and transient rejection tests, then run
+  `make test`.
+- [ ] 9.2 Implement listener-ready serving completion and optional subsequent
+  source discovery without requiring a peer handshake; verify no partial server
+  remains after bind/initializer failure with `make test`.
+- [ ] 9.3 Implement joined-to-serving as serialized full stop/recreate with a
+  serving-configured `P2p` and repeated initializer, never inbound reload; add
+  success, retained-state, bind failure, and no-partial-author tests, then run
+  `make test`.
+- [ ] 9.4 Validate advertised endpoints separately, emit explicit warning on
+  local cross-subnet reuse, and avoid automatic Tor/I2P provisioning claims;
+  add bind/external distinction and reuse tests, then run `make test`.
+- [ ] 9.5 Implement one author task with a fixed non-configurable 30-minute
+  base interval, independent uniform ±10-minute
+  `OsRng` jitter, fresh `OsRng` ad IDs, shuffled subnet order, and bounded
+  two-hour-default/24-hour-maximum lifetime and addresses; add default/clamp,
+  config-rejection, reserve-capacity, and injected clock/RNG tests, then run
+  `make test`.
+- [ ] 9.6 Prove cadence-only behavior: initialize, listener ready, peer connect,
+  recreate, overlay connect, and stop never emit immediately; only cadence
+  ticks author and stop ceases future ads without withdrawal. Run `make test`.
+
+## 10. Lilith persistent overlay
+
+- [ ] 10.1 Add optional lilith overlay config and explicit `swarm` feature with
+  ordinary peers, separate accept/external addresses, production local-test
+  egress disabled, strict policy, and bounded limits; add
+  overlay-only, inbound-only, mixed, prohibited-local-target, malformed, and
+  omitted tests, then run `make test`.
+- [ ] 10.2 Start lilith with the durable passive store and no ad-refinery/dial
+  path; add direct ordinary cold-start, persisted expiry, dedup saturation,
+  per-subnet quota, zero local-author partitions, checked monotonic-epoch
+  remainder restart, equality/overflow/reduced-cap/epoch startup failure, two-
+  hour stale-address clamp, rollback-before-ID-commit replay, forward-clock, and
+  dial-spy tests, then run `make test`.
+- [ ] 10.3 Implement aggregate-only status RPC for listener, aggregate
+  connections, capacities, address/dedup, per-subnet-quota/checkpoint failures,
+  eviction, expiry, and rejection; prove peer/advertised addresses, IDs, sources,
+  per-peer data, and query mappings are absent with `make test`.
+- [ ] 10.4 Keep overlay and legacy instances isolated in settings, policy,
+  registry, paths, failures, and shutdown; add mixed and independent-failure
+  tests, then run `make test`.
+## 11. Multi-node abuse, lifecycle, and privacy verification
+
+- [ ] 11.1 Add local cold-start tests for cached ordinary success, fresh
+  configured fallback, descriptor lookup, app initialization, and ordinary
+  subnet join without per-subnet overlay configuration; run `make test`.
+- [ ] 11.2 Add first-server creation followed by cadence ad, client lookup, and
+  ordinary join, proving a new subnet requires no preexisting peer; run
+  `make test`.
+- [ ] 11.3 Add poisoned/stale tests proving stores/lilith never dial targets,
+  two-hour defaults reduce stale retention, CSPRNG ordering defeats lexical
+  prefix grinding, joining rejects incompatibility fallibly, relays are not
+  blamed, and static fallback remains usable; run `make test`.
+- [ ] 11.4 Add concurrent abuse tests for oversized fields/URLs, query/pending
+  floods, fresh-ID floods, protected-set saturation, replay loops, relay fanout,
+  one-subnet/distributed saturation, local-author reserve, terminal-cursor
+  mutation/duplicate/empty-page churn, durable-write/checkpoint pressure,
+  candidate reservoir/order/time-budget grinding, dial concurrency/rate, per-
+  destination repetition, DNS resolution totals, and victim reflection; verify
+  every bound and strict penalty with `make test`.
+- [ ] 11.5 Add lifecycle tests for initializer failure, caller-handle drop,
+  duplicate joins, seed/refine filtering, cancellation, source reconstruction,
+  serving recreation, concurrent leave, retained rejoin, isolated delete, and
+  failing shutdown hooks; run `make test`.
+- [ ] 11.6 Add artifact/privacy tests proving no protocol-added stable node/
+  signing identity across overlay/subnets (excluding disclosed endpoint reuse),
+  non-public omission, caches/stores free of queries/sources, scoped subnet and
+  transport persistence, explicit ID-to-endpoint visibility but no source-peer/
+  authorship mapping; run `make test`.
+
+## 12. Pilot, documentation, and gates
+
+- [ ] 12.1 Add default-off darkirc pilot using registry-owned app state and
+  explicit overlay-then-static reconstruction while preserving current static
+  behavior; add overlay success, fresh static fallback, repeated initializer,
+  and complete failure tests, then run `make test`.
+- [ ] 12.2 Add bounded aggregate pilot metrics for lookup latency, stale/poisoned
+  failures, per-subnet/global remote dedup pressure, checkpoint failures, page
+  completion/mutation omissions, and bootstrap/source fallback. Expose no local-
+  author reserve occupancy/use/failure timing. Prove schemas contain no peer,
+  query mapping, private ID, local-author fact, or secret with `make test`.
+- [ ] 12.3 Write deployment/migration guidance covering ordinary peers versus
+  seeds, staged reconstruction, initializer reruns, serving recreation downtime,
+  bind versus external endpoints, ID/query disclosure, unsigned poisoning,
+  gossip/store visibility of ID-to-endpoint mappings, egress/DNS/proxy policy,
+  reflection budgets, per-subnet/global dedup saturation and local reserve,
+  monotonic checkpoint extension and rollback-before-commit replay limitation,
+  two-hour TTL defaults, mutation-tolerant non-snapshot pagination, randomized
+  reservoir/time-partitioned candidate tiers, static fallback, endpoint
+  reuse, control/data channel separation, session-bound default versus reduced-
+  privacy immediate teardown and its timing correlation, transient reconnect,
+  transport state, Tor/I2P limitations, rollback, and absent auth/PIR/global-
+  observer guarantees. Record endpoint-reuse enforcement and query-peer privacy
+  budgets as separate follow-up change scopes, not implementation in `swarm`.
+- [ ] 12.4 Run and resolve all required gates without weakening tests/lints:
+  `make fmt`, `make`, `make clippy`, `make test`, and `make check`.
+- [ ] 12.5 Invoke `@anon-security-review` on the complete implementation diff,
+  record the verdict, and treat FAIL as blocking; resolve or escalate findings.
+- [ ] 12.6 Obtain final human patch review of shared `src/net`, swarm, lilith,
+  and darkirc changes, explicitly covering attacker-induced dialing, exact
+  direct/proxy route semantics, complete candidate-pipeline panic
+  removal, reservoir/time-partition fairness, replay checkpoint arithmetic/
+  atomicity/rollback limits, local-author partition metadata, source
+  reconstruction, listener recreation, state ownership, dependency activation,
+  and test evidence. Broader rollout/deprecation is a separate change.