Scalability Measurements

Performance characteristics of Weft DST framework under increasing workload: shim overhead, broker latency, node scaling, recording size, and shrinking efficiency.

A. Shim Overhead (Synthesis Examples)

Runtime overhead of libweft_shim.so relative to native execution (best of 5 runs, wall time):

Interpretation:

• chrono: Not an artifact — chrono spends ~2.8s in real sleep/nanosleep/usleep calls natively. Under the shim, time is virtual: sleeps advance the simulated clock and return immediately, so the whole program finishes in 3ms. This row measures time acceleration (a core DST feature — sleep-heavy tests run ~1000× faster), not overhead.
• montecarlo: ~70% overhead for CPU-bound simulation under the shim (LD_PRELOAD interception on its syscalls).
• entropy: ~100% overhead, but on a 1ms baseline — dominated by shim initialization, not steady-state cost.

Observation: For CPU-bound workloads, shim overhead is moderate (~70% on montecarlo). For sleep- or timer-driven workloads (most distributed protocols), virtual time makes simulated runs dramatically faster than real time.

B. Broker Datagram RTT (Median Latency)

Round-trip time for 5000 paired send-recv operations (10,000 broker ops total):

Mean (Weft): 125 µs/datagram
Variance: 112–138 µs (±10%)

Interpretation:

• Native loopback: 0.4 µs/op (kernel UDP stack, no LD_PRELOAD overhead).
• Weft broker: 125 µs/op = ~300× slower than native loopback.
• This is expected: shim intercepts every syscall, marshals to broker, waits for response, and descheduled guest thread.

Limitation: Per-operation latency percentiles (p50, p99) cannot be measured in-guest; guest clocks are virtual (seeded, not wall-clock). Broker-side instrumentation is recommended (Phase 8 optimization).

C. Node Scaling + Broker Memory

Chord workload, 1 seed, 45 ticks, latency=uniform:1000-8000 (measured with GNU time; the first run of this section silently failed because /usr/bin/time is absent from the rust:1.84-bookworm image — the bench script now installs it):

Interpretation: Broker memory is flat (~2.3 MB) from 7 to 14 nodes — the broker holds only in-flight messages and per-node queues, not history (records stream to the gzip log). Wall time is dominated by fixed startup cost at this scale; log size grows with message volume (~1.4× from 7→14 nodes). Nothing here suggests a scaling wall below tens of nodes; the practical ceiling is per-node process overhead on the host, not the broker.

D. Recording Size vs. Run Length

Deterministic Chord workload (7 nodes, 1 seed, varying simulation ticks):

Linear Scaling: Log size grows roughly linearly with simulation ticks (3× ticks ≈ 6× log size, accounting for compression variance).

Extrapolation to 5000-seed Campaign:

• Average log size per seed: ~0.65 MB (from Chord and Raft prior runs: 0.64 MB Chord, 0.47 MB Raft).
• 500-seed campaign: ~325 MB.
• 5000-seed campaign: ~3.25 GB.

Compression: Weft-log v1 uses gzip. Typical compression ratio ~1:8–1:10 for broker event streams (high repetition of message formats).

E. Shrinking Efficiency (Fuzz)

Delta-debugging on ~10k-event violation corpus (3 nodes, 3300 sends, 8 seeds, fifo+dup invariants, latency=uniform:0-8000ms, loss=2%):

Summary:

• Shrinking time: 80–600 executions per violation (highly variable, depends on structure).
• Reduction rate: 1:800× to 1:2000× (14043 → 7–17 ops).
• Total fuzz+shrink time: 132 ms (for 8 seeds, 6 distinct violations found).

Interpretation: Delta-debugging is effective; 10k-event fuzz takes ~16ms per seed on average. Shrinking is the expensive phase (multiple re-executions to verify minimality), but still completes in <1s per violation on modest hardware.

F. Live-Run Verdict Reproducibility

Chord protocol seed 0, 10 consecutive live runs (latency=uniform:1000-60000ms, network non-determinism):

Results:

• Violations: 1 (seed 0 reached violation in 1 of 10 runs)
• OK (clean runs): 8
• Discard (uninformative): 1

Interpretation: Cross-process message delivery order is re-rolled on each live execution. Same seed can reach different verdicts. Recording replay of any single run is byte-for-byte identical (Phase 5), but live-run campaigns are statistical, not deterministic. This is expected and irreducible without kernel-level scheduling determinism.

Summary: Practical Scalability

Conclusion

Weft is practical for 500–5000 seed campaigns on commodity hardware:

• Shim overhead is moderate and acceptable.
• Broker latency is high in absolute terms but predictable for deterministic replay.
• Recording size is manageable (GB per campaign, not TB).
• Shrinking is efficient (minimal repros in seconds).
• Live-run drift is expected and documented as a Phase 3 limitation.

Next optimization targets (Phase 8): See SCALABILITY_RECOMMENDATIONS.md.