CoordinationCost
The G4 coordination-cost score for a Draft pipeline.
A planner minimises this cost; an executor pays it. Three dimensions are tracked:
rounds — the count of coordinated nodes in the planned DAG. Each DraftStage.Embroider contributes 1 round; each DraftStage.BatchedEmbroider also contributes 1 round — a batched proposal carries an entire level's worth of agreements in one Raft round-trip, regardless of how many ops it batches. So:
A fully-monotone Draft has 0 rounds.
K independent DraftStage.Embroider nodes (not consolidated) → rounds = K.
Those same K nodes consolidated into one DraftStage.BatchedEmbroider (one dependency level) → rounds = 1.
A dependency chain with L levels → rounds = L (the DAG depth), one node per level.
rounds is therefore an active lever: calling Draft.plan (which includes consolidation) drives coordinationCost(plan(draft)).rounds ≤ coordinationCost(draft).rounds, with strict < whenever independent agreements exist to batch. This is the improvement E-3 could not show (E-3 was pinned at rounds ≤ 1 on a single-embroider linear pipeline).
coupling — the blast-radius term: the maximum batch size (DraftStage.BatchedEmbroider.opIds size) across all coordinated nodes. A DraftStage.Embroider contributes 1; a fully-monotone draft contributes 0. Batching K agreements into one round is efficient, but it couples their failure domains: a single rejection forces a retry of all K. The planner minimises rounds first; coupling is the secondary objective, reflecting the honest tradeoff — fewer rounds at the cost of a larger retry unit. A future planner might cap batch size explicitly.
coordinatedVolume — the estimated number of elements entering all coordinated stages combined. When rounds == 0, this is 0. For each coordinated node the estimate derives from WarpStats HyperLogLog sketches via a predecessor walk; unknown filter selectivities are treated conservatively. The estimate is produced by Draft.coordinationCost.
Ordering. Costs are compared lexicographically — minimise rounds first, then coupling, then coordinatedVolume. A draft with fewer rounds always beats one with more, regardless of the other two dimensions. When rounds are tied, the smaller-batch option wins (less blast radius); volume is the final tiebreaker.
See also
Samples
val src = OpId("source.docs")
val mapScore = OpId("map.score")
val filterThreshold = OpId("filter.above-threshold")
val embroider = OpId("embroider.rank")
// Programmer places embroider early (before the filter).
val unplanned: Draft<ByteArray> = Warp.shuttle(src)
.map(mapScore)
.embroider(embroider)
.filter(filterThreshold)
// Build stats: 1 000 source docs, 50 pass the filter.
var stats = WarpStats.empty()
for (i in 1..1_000) stats = stats.piece(stats.observe(src, "doc_$i"))
for (i in 1..50) stats = stats.piece(stats.observe(filterThreshold, "doc_${i * 20}"))
// Unplanned: embroider before filter → full source cardinality.
val unplannedCost = unplanned.coordinationCost(stats)
check(unplannedCost.rounds == 1)
check(unplannedCost.coordinatedVolume >= 900L) { "should see ~1000 docs" }
// Planned: embroider deferred past filter → only ~50 docs reach consensus.
val planned = unplanned.plan(stats)
val plannedCost = planned.coordinationCost(stats)
check(plannedCost.rounds == 1)
check(plannedCost.coordinatedVolume < 100L) { "should see only ~50 docs after filter" }
check(plannedCost < unplannedCost)
check(unplanned.isEquivalentTo(planned)) { "plan must preserve equivalence" }// Branch C chains two embroideries: embroider(C) must commit before embroider(D).
val branchA = Warp.shuttle(OpId("source.a")).embroider(OpId("embroider.a"))
val branchB = Warp.shuttle(OpId("source.b")).embroider(OpId("embroider.b"))
val branchC = Warp.shuttle(OpId("source.c"))
.embroider(OpId("embroider.c"))
.map(OpId("map.m"))
.embroider(OpId("embroider.d"))
val unplanned: Draft<Unit> = branchA.combine(branchB).combine(branchC)
val planned: Draft<Unit> = unplanned.plan(WarpStats.empty())
val stats = WarpStats.empty()
// Unplanned: 4 separate Embroider nodes → 4 rounds (one per node).
check(unplanned.coordinationCost(stats).rounds == 4)
// Planned: BatchedEmbroider(A,B,C) at level 0 + Embroider(D) at level 1 → 2 rounds.
check(planned.coordinationCost(stats).rounds == 2)
// rounds is a real lever — the planner measurably cuts round count.
check(planned.coordinationCost(stats) < unplanned.coordinationCost(stats))
// Coupling = 3: the level-0 batch bundles three agreements (blast-radius = 3).
check(planned.coordinationCost(stats).coupling == 3)Properties
Estimated elements entering all coordinated stage(s); 0 when rounds == 0.
Maximum batch size across all coordinated nodes — 0 when rounds is 0, 1 for a lone DraftStage.Embroider, and DraftStage.BatchedEmbroider.opIds.size for a batched node.