WarpNode
Ties the warp foundation together over a real Seam.
Each peer in the session claims and executes the tasks it owns on the consistent-hash ring — ring.owner(task) == selfId. Results land in a shared, replicated Results board.
Coordinated-path execution guarantee. The CoordinationKind.Coordinated path achieves exactly-once execution under stable leadership, leadership failover, and the tested roster-churn scenarios. The mechanisms:
Coordinated tasks bypass the intent register entirely (#873): consensus — the Raft log plus the leader-side gates below — is the sole arbiter of who executes, so the intent path's ad-hoc lowest-PeerId election never runs for them (and can no longer leak per-task intent entries).
Each ring owner proposes the task to raftNode with a stable
requestId(derived from TaskId) — preventing the same node from double-proposing after a retry.Execution is driven from the committed log, not from the
propose()return. The background onCoordinatedCommit listener fires on every committed entry; only the current Raft leader proceeds, and it must first pass the quorum fence — a RaftNode.readIndex round confirming it still holds a voter quorum at its current term. A deposed-but-unaware leader cannot pass the fence (any quorum it could assemble intersects the majority that elected its successor at a higher term), so the transient dual-leader window (#879 a) closes at the execution decision point.A committed entry consumed while no node held leadership (mid-election — every node skips it on the role check, and
committedis replay=0) is re-driven: on acquiring leadership a node re-proposes every coordinated-queue task it has not executed and that has no replicated result, giving the stranded task a fresh committed entry to fire from (#879 b).A local coordinatedApplied set plus the replicated coordinated-queue membership and Results-board gates deduplicate when two entries for the same task both committed (ring churn double-propose, or a re-drive racing the original entry).
The residual window is execution duration: a leader that passes the fence and is deposed while coordinatedExecutor is still running cannot be distinguished from a crashed one, so the next leader's re-drive may run the task again (at-least-once in that narrow case). Duplicate results are absorbed by the Results LWW ORMap backstop and counted in duplicates; a non-idempotent executor should be tolerant of a rerun that follows its own mid-flight interruption.
Roster source. WarpNode drives the consistent-hash ring from rosterFlow — a Flow of the live peer set. Two pluggable sources are provided:
Seam.rosterSnapshot — derives the roster from Seam.peers; cheap and eventually consistent. Preserves the pre-#826 behavior.
RaftNode.rosterSnapshot — derives the roster from Raft's replicated ClusterConfig; strongly consistent, minimising duplicate executions under stable membership.
The roster source is required — absence would silently choose a consistency model without the caller knowing. Pass the appropriate adapter for your use case.
Liveness-driven failover. WarpNode maintains two failure-detection signals:
Roster departure — when a peer disappears from rosterFlow, the ring is rebuilt immediately and pending tasks re-home to their new owner.
Heartbeat partition — a HeartbeatPartitionDetector runs per admitted peer (peers in rosterFlow that are not selfId). When a peer becomes unresponsive (heartbeat timeout) or lost (reconnect window expired), it is added to partitionedPeers and excluded from the effective ring. On recovery it is removed. The effective ring is
rosterPeers - partitionedPeers; tasks whose former owner is partitioned re-home to the next peer clockwise on the effective ring.
The Results backstop ensures correctness under both signals: duplicate executions during a failover window are absorbed, and the board converges to one entry per task.
Incoming fan-out. WarpNode is the sole collector of seam.incoming (satisfying the kuilt single-collection contract, ADR-034). Every received Swatch is fanned to rawIncoming before the mux channels consume it. The internal MuxSeam subscribes to rawIncoming rather than seam.incoming directly. Per-peer HeartbeatPartitionDetectors subscribe to rawIncoming filtered by sender via PerPeerSeam — no second collection of seam.incoming is ever needed.
Thread-safety. Shared mutable state (ring, claimed, partitionedPeers, detectorJobs, rosterPeers) is guarded by an explicit kotlinx.atomicfu.locks.ReentrantLock so this type is safe under a multi-threaded dispatcher. No limitedParallelism(1) confinement is used — see CLAUDE.md.
Injection contract. scope, rosterFlow, and clock are required parameters. Pass kotlinx.coroutines.test.TestScope.backgroundScope in tests, and a fixed Instant-returning lambda for the clock.
Parameters
This peer's identifier on the seam.
The multi-peer session. WarpNode takes sole ownership of Seam.incoming by collecting it once and fanning every frame to rawIncoming.
A Flow of the current live peer set, used to rebuild the hash ring on membership change. Use Seam.rosterSnapshot for eventual consistency or RaftNode.rosterSnapshot for strong consistency backed by Raft membership.
Coroutine scope for background collection jobs. Required — no default.
QuilterConfig for both internal Quilters. Defaults to the QuilterConfig production defaults. Pass a short-cadence config in tests that need fast anti-entropy.
Provides the current Instant for per-peer HeartbeatPartitionDetectors. Required — never kotlin.time.Clock.System by default. Production callers use { Clock.System.now() }; tests inject a fixed or virtual clock so liveness timeouts are deterministic.
Timing for per-peer heartbeat ping/pong. A tuning parameter — defaults to HeartbeatConfig production defaults (5 s interval, 15 s timeout, 60 s reconnect window). Tests typically inject a short-cadence config.
How owned tasks are claimed. ClaimStrategy.Ring is pure consistent-hash assignment; ClaimStrategy.RingWithIntent adds the intent-register safety net. Defaults to ClaimStrategy.RingWithIntent.
The local op registry that maps symbolic OpIds to their Op implementations. A claiming peer resolves the TaskDescriptor.op here and runs its own registered copy — the function never travels; only the name does. An op absent from the registry leaves the task unclaimed and pending (the "bobbin not loaded yet" state — warp slices C4/C5 will service it via lazy-fetch; see TaskDescriptor).
Suspending function for CoordinationKind.Coordinated tasks. Invoked from the committed-log listener (onCoordinatedCommit) on the current Raft leader, not inline after propose() — and only after the leader passes the RaftNode.readIndex quorum fence, so a deposed-but-unaware leader never fires it. coordinatedApplied plus the replicated queue/results gates prevent re-invocation when two entries for the same task both committed (churn double-propose or a re-drive racing the original entry). The one residual window is a leader deposed while this function is mid-flight — the next leader's re-drive may then run the task again; see the class-level guarantee note. Defaults to an explicit error so a caller who provides a raftNode but forgets to supply an executor surfaces the omission immediately rather than silently doing nothing.
RaftNode backing the CoordinationKind.Coordinated execution path. When supplied, the ring owner proposes the task to this Raft cluster for total-order delivery. Execution then fires from raftNode.committed on the Raft leader's WarpNode, fenced by RaftNode.readIndex and backstopped by leadership-acquisition re-drive of stranded queue entries — exactly-once under leadership failover and warp-roster churn (see the class-level guarantee note for the one mid-execution-deposition residual).
Required for coordinated tasks. If null, calling enqueue with CoordinationKind.Coordinated throws IllegalStateException immediately — fail-loud, never a silent downgrade to the ring path. Pass a us.tractat.kuilt.raft.test.FakeRaftNode from :kuilt-raft-test configured as us.tractat.kuilt.raft.RaftRole.Leader for unit tests, or use us.tractat.kuilt.raft.test.MultiNodeRaftSim with real nodes for consensus-correct cluster tests.
The all-or-nothing lazy-code-mobility bundle (Creel + WasmRuntime + opToBobbin). When null (a symbolic-only node), an op missing from registry leaves the task pending ("bobbin not loaded yet") and anti-entropy re-evaluates — today's behavior. When non-null, an unresolved op is fetched via a node-owned BobbinExchange over a reserved mux channel, loaded via WasmRuntime, registered for reuse, and run. A verified-but-broken kernel (load or run failure) records a terminal-error OpResult rather than retrying forever.
This peer's compilation Target. When non-null and a lazyFetch is present, a bobbin-backed op resolves the best compiled variant for this target per execution and tiers up when one gossips in (counted in executionsCompiled). When null, resolution is exactly the C5b lazy-fetch behaviour (no tiering). Required to be explicit — a platform's target is never guessed.
Opaque gate consulted on the free (CoordinationKind.Free) path immediately before a resolved op runs (see executeViaRegistry). AdmissionControl.admit returning null defers the task — it is unclaimed and retried on a later cycle, never dropped; a returned AdmissionTicket admits it and its AdmissionTicket.settle fires once the task completes. Defaults to AdmissionControl.OPEN, which admits everything with a no-op ticket, so an un-gated node behaves bit-for-bit as before. The :kuilt-warp-heddle satellite supplies a weighted fair-share implementation keyed on TaskDescriptor.lane; warp core references no such type.
How long a peer's advertised CapSet stays live on this node after it was last received, aged out by local receive time (never cross-peer wall-clock). A slot that stops refreshing expires after this window so a crashed peer's stale capability stops steering placement (H8, design §14.6). Re-advertise via advertiseCapabilities periodically to keep a slot live. A pure TTL tuning knob; the empty default view leaves placement over the whole roster (today's behaviour) until the first advertisement.
Constructors
Properties
Cumulative count of task executions whose result was already present on the Results board when recordResult was called — i.e. duplicates absorbed by the LWW ORMap backstop.
Cumulative count of tasks this node drove to completion — including terminal failures recorded via recordTerminalError. A terminal OpResult.failure (broken or malicious kernel) is a completion, not a retry: the task converges to an error result and is not re-attempted.
Cumulative count of task executions this node ran on a compiled variant after tiering up. Goes from 0 to ≥1 the first time a target-matching variant gossips in. The durable tiered-compilation signal — the same counter measures real tiering once D4 lands a real compiler. A GCounter snapshot; forward via recordWarp into a SUM series.
Cumulative count of task executions this node ran by interpreting the raw bobbin — the un-tiered path. A GCounter snapshot; forward via recordWarp into a SUM series.
Functions
Advertise the capabilities this peer serves — GPU, region, held datasets, runtime, memory class — as an opaque CapSet. Publishes into this peer's own slot of the capability board, folds it into the local TTL tracker, and broadcasts it best-effort over the capability channel (H8, design §14.6).
The current live capability view converged on this node — each peer that has a non-expired advertisement mapped to its CapSet. A peer absent from this map has either never advertised or has aged out; it is treated as CapSet.EMPTY for eligibility.
The eligible subset of the current effective roster for a task requiring affinity: the peers whose advertised CapSet satisfies the predicate (a peer with no live advertisement is CapSet.EMPTY, eligible only for Affinity.Anywhere).
Add taskId to the distributed work queue on the CoordinationKind.Coordinated path.
Add a task to the distributed work queue on the CoordinationKind.Free path.
Add a task to the work queue pinned to this peer — the headline pinned-execution affordance.
Publish a compiled bobbin variant through this node's own BobbinExchange so it gossips to the mesh. This is what a compiler node calls after building a variant (in the spike, via the fake compiler). Requires a lazyFetch capability — throws IllegalStateException otherwise, fail-loud rather than silently dropping the variant.
Register this node as a compiler node: installs the well-known CompileOp.ID op in registry so compile(sourceHash, target, optLevel) requests reach this node as ordinary ring-dispatched tasks rather than requiring a direct imperative publishVariant call.