[Benchmark] mimalloc vs jemalloc

[eval-exec]

What problem does this PR solve?

https://github.com/microsoft/mimalloc
Problem Summary:

• CKB hard-codes jemalloc in the main binary today, which makes allocator experiments invasive.
• The existing benchmark suite measures throughput, but it does not provide a simple side-by-side allocator memory check.
• A quick allocator-memory run in this environment did not show a memory win for mimalloc yet, so the comparison needs to be explicit and easy to repeat.

What is changed and how it works?

What's Changed:

• makes mimalloc the default allocator for ckb
• keeps jemalloc available behind --features jemalloc
• makes profiling imply jemalloc, so jemalloc-only profiling still builds correctly
• gates jemalloc-specific memory tracker hooks behind a dedicated jemalloc feature
• adds cargo bench -p ckb-benches --bench allocator_memory to compare allocator RSS / peak RSS on the existing overall block-processing workload
• documents the allocator benchmark commands in the benches crate docs

Related changes

• None.

Check List

Tests

• Manual test (add detailed scripts or steps below)

cargo test -p ckb --no-run
cargo test -p ckb --no-run --no-default-features --features jemalloc
cargo test -p ckb --no-run --features profiling
cargo bench -p ckb-benches --bench allocator_memory --no-run --features ci
cargo bench -p ckb-benches --bench allocator_memory --no-run --no-default-features --features "ci jemalloc"
LD_LIBRARY_PATH=/nix/store/xp989kyfg52803fmkzbz5py35jphcpgd-gcc-14.3.0-lib/lib cargo bench -p ckb-benches --bench allocator_memory --features ci -- 16 1
LD_LIBRARY_PATH=/nix/store/xp989kyfg52803fmkzbz5py35jphcpgd-gcc-14.3.0-lib/lib cargo bench -p ckb-benches --bench allocator_memory --no-default-features --features "ci jemalloc" -- 16 1

Observed allocator bench output from the last two commands:

• mimalloc: after_workload rss_bytes=173170688, after_drop rss_bytes=174518272
• jemalloc: after_workload rss_bytes=125931520, after_drop rss_bytes=115007488

Side effects

• Performance regression: possible. In the small manual memory run above, jemalloc used less RSS than mimalloc, so this PR is best treated as allocator-comparison plumbing plus a default swap that still needs broader benchmarking on realistic workloads.

[Copilot]

Pull request overview

This PR makes allocator experiments less invasive by switching CKB’s default allocator to mimalloc (while keeping jemalloc available via a feature flag), and adds a dedicated benchmark to compare allocator RSS/peak RSS on the existing “overall” block-processing workload.

Changes:

• Switch default allocator to mimalloc, keep jemalloc behind --features jemalloc, and make profiling imply jemalloc.
• Gate jemalloc-specific memory-tracker hooks behind a dedicated jemalloc feature.
• Add allocator_memory benchmark and document how to run it.

Reviewed changes

Copilot reviewed 10 out of 11 changed files in this pull request and generated 4 comments.

Show a summary per file

File	Description
util/memory-tracker/src/process.rs	Gates jemalloc ctl usage behind feature = "jemalloc" and adjusts tracking loop logic.
util/memory-tracker/src/lib.rs	Updates docs/feature semantics for jemalloc profiling support.
util/memory-tracker/Cargo.toml	Makes jemalloc deps optional; adds jemalloc feature; profiling implies jemalloc.
src/main.rs	Switches global allocator selection to feature-based jemalloc/mimalloc.
ckb-bin/Cargo.toml	Adds jemalloc feature forwarding to memory-tracker.
benches/src/lib.rs	Documents allocator-memory comparison commands.
benches/benches/benchmarks/overall.rs	Extracts reusable run_overall_rounds helper used by new benchmark.
benches/benches/allocator_memory.rs	Adds new benchmark binary reading RSS/peak RSS and running the overall workload.
benches/Cargo.toml	Adds optional allocator deps + bench target; defaults benches to mimalloc.
Cargo.toml	Adds mimalloc dep/feature, makes tikv-jemallocator optional, updates feature wiring and defaults.
Cargo.lock	Adds mimalloc packages and updates transitive dependency resolution.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

ckb_metrics::handle() is called multiple times per loop iteration (for process metrics, jemalloc metrics, and the new is_some() check). Consider caching the handle once at the start of the loop iteration (e.g., let metrics = ckb_metrics::handle();) and reusing it to avoid repeated lookups and ensure consistent behavior within an iteration.

[Copilot]

tikv-jemallocator is only declared for cfg(all(not(target_env = "msvc"), not(target_os = "macos"))) in Cargo.toml, but the #[cfg(feature = "jemalloc")] global allocator here is not target-gated. Enabling the jemalloc feature on Windows/MSVC or macOS will compile this block and fail due to the missing dependency. Consider matching the cfg to the dependency target (or emitting a compile_error! on unsupported targets when feature="jemalloc" is enabled).

[Copilot]

This benchmark hard-depends on Linux /proc (/proc/self/status) and will panic at runtime on non-Linux targets. Consider gating the binary with #[cfg(target_os = "linux")] (and providing a stub main that prints a clear message on other OSes) or using a cross-platform crate (e.g., sysinfo) to read RSS/peak RSS.

[Copilot]

The new benchmark commands are ambiguous about where they should be run from. Earlier in this doc you use cd benches && cargo bench ..., and the PR description uses cargo bench -p ckb-benches ...; running cargo bench --bench allocator_memory from the workspace root likely won’t find this bench target. Consider updating these lines to consistently include -p ckb-benches (or cd benches && ...).

Suggested change

	//! cargo bench --bench allocator_memory --features ci
	//! cargo bench --bench allocator_memory --no-default-features --features "ci jemalloc"
	//! cargo bench -p ckb-benches --bench allocator_memory --features ci
	//! cargo bench -p ckb-benches --bench allocator_memory --no-default-features --features "ci jemalloc"