Skip to content

Advanced features

This page covers HyperbyteDB features beyond basic read/write operations: clustering, continuous queries, TLS, and S3 storage backends. Authentication is documented in its own guide.

Clustering

HyperbyteDB supports master-master replication where every node in the cluster accepts both reads and writes. Data writes are replicated asynchronously to all peers. Schema mutations (CREATE/DROP DATABASE, DELETE, etc.) use Raft consensus for consistent ordering.

How it works

  1. A client writes to any node.
  2. That node persists to its local WAL and returns 204 to the client.
  3. The node fans out the write via HTTP to all peers.
  4. A header (X-Hyperbytedb-Replicated: true) prevents infinite replication loops.

Configuration

Each node needs a unique node_id and must list all other nodes as peers:

Node 1: 

[cluster]
enabled = true
node_id = 1
cluster_addr = "node1.example.com:8086"
peers = "node2.example.com:8086,node3.example.com:8086"

Node 2: 

[cluster]
enabled = true
node_id = 2
cluster_addr = "node2.example.com:8086"
peers = "node1.example.com:8086,node3.example.com:8086"

No initialization step is required. Nodes begin replicating as soon as they start.

Cluster endpoints

Endpoint Method Description
/cluster/metrics GET Cluster mode, node address, peer list
/cluster/nodes GET All nodes with a self flag

Important considerations

  • Replication is asynchronous and best-effort. If a peer is down, the write succeeds locally but the peer misses it until anti-entropy or hinted handoff delivers it.
  • There is no distributed query fan-out. Each node queries only its own Parquet files.
  • Schema mutations (CREATE DATABASE, DROP DATABASE, DELETE, etc.) are replicated via Raft for consistent ordering.
  • Hinted handoff stores writes for unreachable peers and replays them on reconnection.

Continuous Queries

Continuous queries (CQs) automatically downsample data on a schedule, writing aggregated results to a target measurement.

Create a continuous query

CREATE CONTINUOUS QUERY "cq_cpu_1h" ON "mydb"
BEGIN
  SELECT mean("usage_idle") AS "mean_usage_idle"
  INTO "cpu_1h"
  FROM "cpu"
  GROUP BY time(1h), *
END

With resample control

CREATE CONTINUOUS QUERY "cq_cpu_1h" ON "mydb"
RESAMPLE EVERY 30m FOR 2h
BEGIN
  SELECT mean("usage_idle") INTO "cpu_1h" FROM "cpu" GROUP BY time(1h), *
END
  • RESAMPLE EVERY 30m — execute every 30 minutes (instead of every resample_every_secs default).
  • FOR 2h — look back 2 hours on each execution.

Manage continuous queries

SHOW CONTINUOUS QUERIES
DROP CONTINUOUS QUERY "cq_cpu_1h" ON "mydb"

CQ definitions are stored in metadata and survive restarts. The background scheduler evaluates CQs every 10 seconds.

User authentication

Enable [auth] enabled = true to require credentials on /write and /query. How credentials are sent, which HTTP paths stay public, and when admin is required for cluster/internal APIs are all covered in Authentication.

TLS / HTTPS

Enable encrypted connections:

[server]
tls_enabled = true
tls_cert_path = "/etc/hyperbytedb/cert.pem"
tls_key_path = "/etc/hyperbytedb/key.pem"

HyperbyteDB uses rustls (no OpenSSL dependency). Both files must be PEM-encoded. When TLS is enabled, the server only accepts HTTPS connections — there is no mixed HTTP+HTTPS mode.

Generate a self-signed certificate (testing only)

openssl req -x509 -newkey rsa:4096 -keyout key.pem -out cert.pem \
  -days 365 -nodes -subj '/CN=hyperbytedb'

S3 Storage Backend

HyperbyteDB can store Parquet files on S3-compatible storage instead of (or in addition to) the local filesystem.

Configuration

[storage]
backend = "s3"

[storage.s3]
bucket = "my-hyperbytedb-bucket"
prefix = "data/"
region = "us-east-1"
# For MinIO, Cloudflare R2, etc.:
# endpoint = "http://minio:9000"
# access_key_id = "minioadmin"
# secret_access_key = "minioadmin"

How it works

  • The WAL and metadata store remain on local disk (RocksDB).
  • Only Parquet data files are stored in S3.
  • Reads download Parquet files to temporary local storage for chDB to query.
  • The StoragePort trait abstracts the difference, so flush, compaction, and query paths work identically.

Supported services

Any S3-compatible API works: AWS S3, MinIO, Cloudflare R2, DigitalOcean Spaces, Backblaze B2, etc. Set the endpoint field for non-AWS services.

Columnar MessagePack Ingest

An optional high-performance ingest format is available behind the columnar-ingest feature flag. It accepts columnar data encoded as MessagePack, reducing parse overhead compared to line protocol for bulk imports.

Build with the feature enabled:

cargo build --release --features columnar-ingest

This adds support for Content-Type: application/x-msgpack on the /write endpoint.

Statement Summary

When enabled, HyperbyteDB tracks recently executed InfluxQL statements for debugging and observability.

[statement_summary]
enabled = true
max_entries = 1000

View recent statements:

curl -sS 'http://localhost:8086/api/v1/statements'

Each entry includes the normalized query text, digest, execution time, and error status.

See Also