CE Performance Benchmark — 2026-05-25

Host: local dev machine (4 cores, 15 GiB RAM, Debian Linux)
Node: height 10,064 — chain file 1.7 MB (bincode + zstd-3)
Binary: release build from main @ dcb2a9f
Config: ce start --port 4001 --api-port 8080 --bootstrap/relay 178.105.145.170:4001

---

Setup

cargo build --release            # 3 min 16 s (cold, 4 cores)
cp target/release/ce ~/.local/bin/ce
ce start --port 4001 --api-port 8080 \
  --bootstrap /ip4/178.105.145.170/tcp/4001/p2p/<relay-id> \
  --relay     /ip4/178.105.145.170/tcp/4001/p2p/<relay-id>

ce is installed at ~/.local/bin/ce (already in PATH).

---

Results

1. HTTP API Latency (100 sequential requests, loopback)

Endpoint	p50	p95	p99	max
GET /health	0.32 ms	0.43 ms	10.92 ms	10.92 ms
GET /status	0.32 ms	0.39 ms	0.49 ms	0.49 ms
GET /jobs	0.32 ms	0.38 ms	0.46 ms	0.46 ms
GET /signals	0.32 ms	0.46 ms	2.28 ms	2.28 ms
GET /atlas	0.32 ms	0.46 ms	0.57 ms	0.57 ms
GET /bootstrap	0.34 ms	0.40 ms	0.49 ms	0.49 ms

Axum on Tokio is fast — median 0.32 ms, well within budget for all read paths.

The occasional p99 spike on /health (10.92 ms) is OS scheduling jitter, not application code.

2. Mining Rate

Metric	Value
Blocks in 30 s	3
Rate	0.100 blocks/s
Interval	10.0 s/block

Exactly on the 10-second mining ticker target. PoW difficulty is 0 (no leading-zero requirement) so every hash seals instantly — block rate is entirely timer-driven.

3. CLI Cold-Read Performance (ce status)

Metric	Value
p50	30.9 ms
p95	32.5 ms
min	30.2 ms
max	32.5 ms
Chain height	10,041
Bytes/block	~170 B compressed (~1,358 B uncompressed est.)
Deserialize rate	325,055 blocks/s

ce status reads and fully deserializes the chain from disk on every invocation. At 10K blocks this costs ~30 ms. The decompress + deserialize path is fast — most of the time is process startup + zstd decompression of the 1.7 MB file.

4. API Throughput (concurrent requests)

Concurrency	GET /status RPS	GET /signals RPS
1	1,018	2,056
2	4,091	3,431
4	4,128	3,735
8	2,974	3,497
16	4,556	3,690
32	4,835	4,479
64	6,596	5,236

Throughput is good and scales with concurrency. The variance between runs reflects Python threading overhead in the test harness rather than Axum behavior.

5. Lock Contention Under Load

Sequential vs. concurrent (16 reader threads):

Path	Sequential p50	Concurrent p50
GET /health	0.32 ms	3.59 ms
GET /status	0.32 ms	3.59 ms

Under 16 concurrent readers, latency grows from 0.32 ms to 3.59 ms — a 11x increase — because both /status and /signals acquire an exclusive tokio::sync::Mutex, serializing all concurrent reads.

6. Signal Submission

Path	p50	p95	p99	max
POST /signals/send	0.37 ms	0.69 ms	—	11.75 ms

Fast for single-threaded use; subject to same mutex contention under load.

7. Process Resources (after 4 min uptime)

Metric	Value
RSS	70.8 MB
VSZ	388 MB
CPU	0.6%
MEM	0.4%

Very lean. The node is mostly idle between 10-second mining ticks.

---

Performance Gain Areas

P1 — Replace Mutex<Chain> with RwLock<Chain>

Impact: high
Every read endpoint (/status, /signals, /jobs, /atlas, /bootstrap) acquires an exclusive tokio::sync::Mutex<Chain> lock. Only write operations (block append, tx pool update) need exclusivity.

Switching to tokio::sync::RwLock lets read requests run concurrently with zero contention, eliminating the 11x latency spike at c=16 (3.59 ms → ~0.32 ms).

Affected file: crates/ce-node/src/lib.rs:169 — chain: Arc<Mutex<Chain>> and all usage sites.

The same applies to SignalRing and Atlas — both are read-heavy.

P2 — Status sidecar file for ce status CLI

Impact: medium
ce status fully deserializes the chain every time (30 ms at 10K blocks, ~300 ms at 100K). The CLI only needs height, difficulty, and balance — all maintained in Chain's O(1) caches.

Write a small ~/.local/share/ce/status.json (height, balance, difficulty, timestamp) on every block seal. ce status reads this 200-byte file in <1 ms instead of deserializing the full chain.

Relevant save path: crates/ce-node/src/lib.rs:257 — chain_path2 in the mining loop.

P3 — Peer observability: INFO log + /peers endpoint

Impact: medium (ops/debugging)
The mesh swarm has no INFO-level log for connection established/closed. In production, there is no way to verify peer connectivity without enabling RUST_LOG=ce_mesh=debug. This makes diagnosing relay/NAT issues extremely hard.

Two changes needed:

1. Add info!("connected to peer {peer_id}") in ce-mesh::handle_event on ConnectionEstablished.
2. Add a GET /peers API endpoint that returns connected peer IDs and their announce heights.

Relevant file: crates/ce-mesh/src/lib.rs:566 — handle_event.

P4 — gVisor availability for container isolation

Impact: medium (security)
Every startup logs WARN: gVisor not available, falling back to runc. Without gVisor, container isolation relies on runc's Linux namespace stack instead of the gVisor VM boundary. This is the only persistent WARN on startup and directly weakens the adversarial compute isolation model.

Install gVisor (runsc) and configure Docker/containerd to use it as the ce runtime:

sudo apt-get install runsc
sudo runsc install

Relevant file: crates/ce-container/src/lib.rs (detection logic).

P5 — Gossip deduplication / message size

Impact: low-medium, mesh-scale dependent
At current chain height (10K) the block gossip payload includes the full block header + all transactions encoded with bincode. The chain uses blocks: Vec<Block> — sync responses send up to 500 blocks per response. At ~1,358 bytes/block uncompressed, a 500-block sync response is ~680 KB uncompressed. Adding zstd to gossip wire encoding (already used on disk) would cut mesh bandwidth.

Currently gossip messages are bincode-only. Blocks could be gossiped as bincode+zstd to reduce relay bandwidth.

P6 — Chain rebuild_caches on reorg

Impact: low, correctness path
try_reorg calls rebuild_caches() twice (once for the reorg candidate, once for the winner) — full O(n) scan both times. At scale (100K+ blocks), reorgs will be expensive. The incremental apply_block_to_cache already exists; a targeted undo + re-apply would be O(fork-depth) instead.

---

Mesh Connectivity Note

The relay at 178.105.145.170:4001 is reachable (TCP confirmed). The libp2p swarm binds port 4001 on all interfaces and dials the relay. No INFO-level connection event is emitted when the relay handshake succeeds (see P3 above), so connectivity can only be verified by watching block height progression against known network state.

---

Quick Wins Summary

Priority	Change	Effort	Latency impact
P1	Mutex<Chain> → RwLock<Chain>	~1 day	11x under load
P2	Status sidecar file	~2 h	30 ms → <1 ms (CLI)
P3	Peer INFO logs + /peers endpoint	~2 h	Ops quality
P4	Install gVisor	~30 min	Security
P5	Gossip zstd compression	~4 h	Bandwidth
P6	Incremental reorg undo	~1 day	Reorg path only