Benchmarks — the full method and every number

One realistic workload — a billing SaaS's invoice ledger — run as two separate apples-to-apples comparisons: the in-process, in-memory engines (HoloDb embedded and DuckDB), and the networked services (HoloDb server and a durable on-disk SQL Server). Every result — single-table rollups and cross-table joins alike — is verified byte-for-byte identical across every engine before it is timed, including the queries where HoloDb trails.

The workload

CREATE TABLE customers (
  customer_id  INT  PRIMARY KEY,   -- 200,000 customers
  name         TEXT,
  segment      TEXT,              -- smb / mid / enterprise / strategic
  tier         TEXT,              -- gold / silver / bronze
  signup_month INT
);
CREATE TABLE invoices (
  invoice_id   INT  PRIMARY KEY,   -- 10,000,000 invoices
  customer_id  INT,               -- FK → customers, ~50 invoices each
  month        INT,               -- yyyymm, 202301 … 202412 (24 months)
  region       TEXT,              -- US 40% / EU 30% / APAC 20% / LATAM 10%
  status       TEXT,              -- paid 78 / pending 10 / overdue 7 / refunded 3 / failed 2 (%)
  amount_cents INT                -- money as integer minor units (cents), long-tail $20–$50,000
);

How it's run

Same data, same SQL. The customers and invoices tables above, generated once with a fixed seed and loaded into every engine through its own idiomatic bulk path.
Real billing queries. The sixteen queries are the ones a subscription business actually runs: revenue by status / month / region, the running total and invoice count, a single-invoice lookup, overdue outstanding, big-ticket count, largest invoices, top customers, a region×status breakdown — and five cross-table JOINs to the customer dimension: revenue by segment, enterprise revenue, revenue by signup cohort, top named customers, and segment×status.
Verified identical. Before timing, each query's full result set is normalized and compared across all engines — if any engine disagrees, the run is flagged, not published. Every number below passed.
Timed fairly. Median of many runs of execute + consume-all-rows through each engine's own typed client API (no boxing), on a 16-core machine.
DuckDB runs in-process, in memory (its fastest mode).
SQL Server is the durable, on-disk peer: SQL Server 2022 (LocalDB), reading its on-disk table and paying a client/server round-trip the embedded engines don't.
HoloDb runs both embedded (in-process) and as a networked TLS server (the fair peer to SQL Server's round-trip).
Why cents. Money is stored as integer minor units — the model every ledger uses. Integer SUM/COUNT are exact and constant-time from HoloDb's maintained accumulators; the queries it loses are the ones it has to scan for (top-K, filtered sums, high-cardinality and multi-column groups).

In-process, in memory — HoloDb (embedded) vs DuckDB (in-process), 10M invoices ⋈ 200k customers, median ms

billing query	HoloDb	DuckDB	faster
revenue by region	0.004	48.6	HoloDb ~12,000×
invoices by status	0.006	59.0	HoloDb ~9,800×
total invoiced	0.002	2.95	HoloDb ~1,500×
invoice count	0.001	0.98	HoloDb ~980×
revenue by month	0.011	7.11	HoloDb ~650×
look up one invoice	0.005	0.61	HoloDb ~120×
big-ticket invoices (>$5k)	6.40	3.85	DuckDB ~1.7×
revenue by region & status	129.7	76.8	DuckDB ~1.7×
top customers by revenue	321.7	162.0	DuckDB ~2.0×
largest invoices	11.8	4.48	DuckDB ~2.6×
overdue outstanding	82.7	14.1	DuckDB ~5.8×
revenue by segment (JOIN)	56.7	65.6	HoloDb ~1.2×
revenue by segment & status (JOIN)	135	94.9	DuckDB ~1.4×
enterprise revenue (JOIN)	34.3	10.3	DuckDB ~3.3×
revenue by signup cohort (JOIN)	44.0	15.3	DuckDB ~2.9×
top named customers (JOIN)	808	322	DuckDB ~2.5×

The purple cell is the faster engine. Bulk load, rows/sec: DuckDB 916k · HoloDb 359k. The six rollups and the lookup resolve from maintained accumulators and the PK index (no scan, hence microseconds); DuckDB's columnar engine keeps a modest lead on the single-table scans. The five cross-table JOINs now fan out in parallel across all cores: HoloDb wins revenue by segment and lands within ~1.4–3.3× of DuckDB on the rest (down from 15–92× before), all verified identical, and beats SQLite and SQL Server outright.

Networked service — HoloDb server vs SQL Server 2022 (LocalDB), 10M invoices ⋈ 200k customers, median ms

billing query	HoloDb server	SQL Server	faster
invoices by status	0.124	4,032	HoloDb ~32,000×
revenue by month	0.153	2,965	HoloDb ~19,000×
revenue by region	0.134	2,575	HoloDb ~19,000×
total invoiced	0.129	1,156	HoloDb ~9,000×
invoice count	0.138	589	HoloDb ~4,300×
largest invoices	7.42	3,001	HoloDb ~405×
big-ticket invoices (>$5k)	4.20	689	HoloDb ~164×
revenue by region & status	131.7	3,260	HoloDb ~25×
overdue outstanding	83.2	1,004	HoloDb ~12×
top customers by revenue	285.8	3,277	HoloDb ~11×
enterprise revenue (JOIN)	39.7	1,792	HoloDb ~45×
revenue by segment & status (JOIN)	140	6,556	HoloDb ~47×
revenue by segment (JOIN)	57.5	2,946	HoloDb ~51×
revenue by signup cohort (JOIN)	67.6	2,853	HoloDb ~42×
top named customers (JOIN)	806	4,173	HoloDb ~5.2×
look up one invoice	0.20	0.20	tie (~0.2 ms)

Both are networked services paying a client/server round-trip. HoloDb serves the working set from RAM with the same maintained accumulators; SQL Server 2022 (LocalDB) reads its on-disk table. Bulk load, rows/sec: HoloDb server 697k · SQL Server 349k. HoloDb wins 15 of the 16, including all five JOINs by 5× to 51× now that they fan out in parallel across all cores (the heaviest, the name-grouped join, lands at 806 ms versus SQL Server's 4,173 ms) — with the 16th, a sub-millisecond point lookup, a tie.

What the numbers say

The maintained aggregates — the whole-ledger total and count, and the single-column GROUP BY rollups (revenue by status, month, region) — resolve in microseconds and are constant-time: a kept tally is read instead of scanning, so response holds as the ledger grows. The single-invoice lookup resolves from the primary-key index. In-process, HoloDb leads on the six rollups and the lookup; as a networked service against SQL Server it leads on 15 of the 16, including all five joins by 5× to 51× (the name-grouped join finishes in 806 ms, well under SQL Server's 4.2 s).

The five cross-table JOINs fan out in parallel across every core (through the same EvalApp-gated pool the scans use): HoloDb wins revenue by segment outright and lands within ~1.4–3.3× of DuckDB on the rest — down from 15–92× before — all verified identical. DuckDB's pure-columnar engine keeps a modest lead on the single-table scans (a filtered sum, a top-K, a multi-column group). HoloDb targets the mixed transactional, analytics and vector workload, durable and larger-than-memory, in a single embeddable dependency.

← Back to HoloDb · Run it in the browser →