Skip to content

Benchmarks

This repository ships Criterion microbenchmarks for ingestion (in-process, no HTTP) and an optional HTTP soak example for columnar writes against a live server. Throughput is reported as logical points per second for batches of 1000 rows (Criterion’s Throughput::Elements(1000)), unless noted.

What exists

Artifact Kind Feature / notes
benches/ingestion_line_protocol.rs Criterion Always built — line protocol only
benches/ingestion_columnar.rs Criterion Requires --features columnar-ingest
examples/bench_columnar_http.rs Binary (not Criterion) Requires columnar-ingest; needs a running hyperbytedb

Cargo.toml registers the two Criterion benches (harness = false). The HTTP tool is an [[example]], not a [[bench]].

Prerequisites

  • Release profile: Criterion uses [profile.bench] from Cargo.toml (inherits release with debug symbols for line numbers in reports).
  • Columnar: pass --features columnar-ingest for ingestion_columnar and for bench_columnar_http.
  • Batch size: both Criterion sources use const BATCH: u64 = 1000. Change it in the bench file to experiment; all group names embed the numeric suffix (e.g. parse_1000).

Criterion: line protocol (ingestion_line_protocol)

Measures three stages of the server-side ingest path on temporary RocksDB directories: parse only, parse + metadata registration, and parse + metadata + WAL append.

Run

cargo bench -p hyperbytedb --bench ingestion_line_protocol

Benchmark groups

Group Function What it measures
line_protocol_parse parse_1000 parse_line_body_to_points on a synthetic line body (precision ms). No I/O.
line_protocol_metadata parse_plus_metadata_1000 Parse + prepare_batch_metadata against a temp metadata store (benchdb created once).
line_protocol_wal metadata_plus_wal_append_1000 Parse + metadata + build WalEntry + wal.append. line_protocol_wal uses sample_size(20) for steadier WAL timings.

Synthetic lines look like bench,host=bench v={i} {ts}\n with millisecond timestamps.

Scope

These benches do not include Axum, HTTP parsing, gzip, auth, replication, flush-to-Parquet, or chDB.

Criterion: columnar MessagePack (ingestion_columnar)

Same overall story as line protocol, plus fast paths that avoid full Point materialization where possible. Body bytes are produced with rmp_serde::to_vec_named from a ColumnarMsgpackBatch (see Columnar MessagePack write format).

Run

cargo bench -p hyperbytedb --features columnar-ingest --bench ingestion_columnar

Point-expansion path (mirrors line protocol structure)

Group Function What it measures
columnar_decode parse_1000 parse_columnar_msgpack_to_points (msgpack → Vec<Point>).
columnar_metadata prepare_batch_metadata_1000 Parse to points + prepare_batch_metadata on temp RocksDB.
columnar_wal metadata_plus_wal_append_1000 Full path: points + metadata + WalEntry + append. sample_size(20).

Fast path (decoded wire batch)

Group Function What it measures
columnar_decode_fast decode_only_1000 decode_columnar_batch only.
columnar_decode_fast decode_to_points_1000 Decode + columnar_batch_to_points.
columnar_decode_fast decode_to_parquet_1000 Decode + columnar_batch_to_parquet (Arrow/Parquet-shaped without WAL).
columnar_metadata_fast prepare_columnar_metadata_1000 prepare_columnar_metadata on a structured ColumnarMsgpackBatch (no msgpack parse in the timed section).
columnar_wal_fast fast_metadata_plus_wal_append_1000 Decode → prepare_columnar_metadata → columnar_batch_to_points → WalEntry + wal.append. sample_size(20).

Columnar MessagePack write format (v1)

Optional ingest encoding for high-throughput workloads, enabled with the columnar-ingest Cargo feature.

HTTP

  • Method / path: POST /write (same as line protocol and row-wise MessagePack).
  • Query: db (required), precision optional (ns default, ms, us, s, …) — timestamp arrays use the same units as row-wise msgpack.
  • Content-Type: application/vnd.hyperbytedb.columnar-msgpack.v1 (case-insensitive; parameters after ; are ignored). Defined as CONTENT_TYPE in columnar_msgpack.rs.

MessagePack body

One map (not an array of rows) with keys:

Key Type Required Description
measurement string yes Shared measurement for all rows.
tags map string → string no (default empty) Constant tags on every point.
field string yes Name of the single float field.
values array of float64 yes One sample per row; length N is the batch size.
timestamps array of int64 no Length must be N if present. If omitted, the server may use wall-clock nanoseconds (same constraints as missing timestamps in line protocol; can fail with WallClockTimestampUnavailable).

The server expands this into N internal points and then runs the normal metadata + WAL path.

Design notes

  • One float column keeps the wire format small and matches common throughput scenarios.
  • For cluster replication, the coordinator re-encodes to Influx line protocol for peers (same as row-wise msgpack).

HTTP soak: bench_columnar_http

Not a Criterion bench: it spawns concurrent reqwest workers that POST columnar bodies to a running hyperbytedb and prints aggregate latency (p50/p90/p95/p99) and throughput after a warmup window.

Run

  1. Start hyperbytedb (e.g. cargo run --release or Docker).
  2. Create the target database (the example defaults to db=server):
curl -sS "http://127.0.0.1:8086/query" --data-urlencode "q=CREATE DATABASE server"
  1. Run the example:
cargo run --release -p hyperbytedb --example bench_columnar_http --features columnar-ingest

Or use the helper script (same as above, forwards extra args to Cargo):

./scripts/bench-columnar-http.sh

Environment variables

Variable Default Description
HYPERBYTEDB_URL http://127.0.0.1:8086 Base URL of the server
WORKERS 12 Concurrent POST tasks
BATCH_SIZE 1000 Rows per request body
DURATION_SECS 60 How long to measure after warmup
WARMUP_SECS 5 Warmup period excluded from stats

The example posts to /write?db=server&precision=ms with the columnar Content-Type and reports points/sec and latency summaries at the end.

Reference numbers (illustrative only)

The tables below are from one Linux x86_64 run (roughly 2026, recent stable rustc). Re-run on your machine before comparing hardware or publishing.

Line protocol (approx. median)

Benchmark Time (approx.) Throughput (approx.)
parse_1000 ~544 µs ~1.8 M points/s
parse_plus_metadata_1000 ~640 µs ~1.6 M points/s
metadata_plus_wal_append_1000 ~950 µs ~1.0 M points/s

Columnar (approx. median)

Benchmark Time (approx.) Throughput (approx.)
metadata_plus_wal_append_1000 ~509 µs ~2.0 M points/s
fast_metadata_plus_wal_append_1000 ~378 µs ~2.6 M points/s
decode_only_1000 ~5.2 µs ~192 M elements/s
decode_to_points_1000 ~130 µs ~7.7 M points/s
decode_to_parquet_1000 ~101 µs ~9.9 M points/s
prepare_columnar_metadata_1000 ~195 ns metadata hot path (1000-row batch)

prepare_columnar_metadata_1000 is dominated by a very small per-iteration path; treat as relative, not a headline “G points/s” for end-to-end ingest.

Reporting environment (for fair comparisons)

When publishing numbers, record:

  • Git: git rev-parse HEAD
  • Rust: rustc -V
  • CPU: lscpu or model in /proc/cpuinfo
  • RAM: grep MemTotal /proc/meminfo
  • Disk (NVMe vs SSD) if WAL/Parquet I/O is part of the scenario

Expectations and limits

  • Criterion benches isolate in-process costs (parse → metadata → WAL on temp RocksDB). They exclude HTTP, auth, cluster replication, and query engine work.
  • They are not comparable to third-party “maximum ingress” figures that use different storage engines or full pipelines.
  • Criterion may emit HTML reports under target/criterion/ for detailed plots.

See also