Stratum: SQL that branches
A few years ago I hit a problem I suspect many data engineers know. I had a million-row analytical dataset and I wanted to run an experiment: modify a few pricing assumptions, re-run a set of aggregation queries, compare the results against the original. Simple enough, except that in a mutable database “compare against the original” means either keeping a copy of the data or hoping nothing changed. Neither approach scales.
Datahike solves this for entity-level data. Its storage is EAVT-indexed like Datomic’s, tuned for entity traversal and point lookups. That’s the right structure for a system-of-record, but not for scanning 10M rows to compute a GROUP BY with SIMD. Stratum explores the columnar alternative: the same CoW branching semantics, but over column-oriented storage optimized for analytical scans. SQL is the natural interface for this access pattern, and Datahike didn’t have one yet. The longer-term plan is integration: Stratum’s columnar engine and SQL support as a query path within Datahike’s Datalog planner.
A columnar dataset can be a value. Make it immutable with structural sharing and git-like semantics follow: fork a dataset in O(1), modify branches independently, time-travel to any snapshot, persist named commits to storage. Then add SIMD execution via the Java Vector API, and it turns out you can beat DuckDB on most single-threaded analytical queries from pure JVM code, without native compilation or JNI.
The SQL interface
Stratum speaks the PostgreSQL wire protocol. The quickest entry point is the standalone server:
java --add-modules jdk.incubator.vector \
--enable-native-access=ALL-UNNAMED \
-jar stratum-standalone.jar \
--index orders:/data/orders.csv
Any PostgreSQL client connects immediately, whether psql, DBeaver, JDBC or psycopg2:
psql -h localhost -p 5432 -U stratum
-- Standard analytical SQL
SELECT region,
SUM(amount * discount) AS revenue,
COUNT(*) AS orders
FROM orders
WHERE ship_date BETWEEN '2024-01-01' AND '2024-12-31'
GROUP BY region
ORDER BY revenue DESC;
-- Query CSV and Parquet files inline, auto-indexed on first access
SELECT payment_type,
AVG(tip_amount),
PERCENTILE_CONT(0.95) WITHIN GROUP (ORDER BY tip_amount)
FROM read_csv('/data/taxi.csv')
GROUP BY payment_type;
Full DML: SELECT, INSERT, UPDATE, DELETE, UPSERT (INSERT ON CONFLICT). CTEs, correlated subqueries, window functions (ROW_NUMBER, RANK, LAG, LEAD, running aggregates), joins (INNER/LEFT/RIGHT/FULL with multi-column keys), set operations (UNION/INTERSECT/EXCEPT). Aggregates: SUM, COUNT, AVG, MIN, MAX, STDDEV, VARIANCE, CORR, MEDIAN, PERCENTILE_CONT, APPROX_QUANTILE, COUNT(DISTINCT). CASE WHEN, COALESCE, date functions, LIKE/ILIKE, FILTER clause. Full SQL reference →
How the engine works
Every column is split into fixed-size chunks. Each chunk carries pre-computed statistics: minimum, maximum, sum, count. This enables two optimizations.
Zone-map pruning. Each chunk carries pre-computed min, max, sum, and count statistics. DuckDB stores only min and max per segment, used for predicate filter pushdown, skipping segments that can’t contain rows matching a WHERE clause. Both engines do this. What DuckDB doesn’t pre-compute is per-segment SUM or COUNT, so unfiltered aggregates like COUNT(*), SUM(price), or AVG(price) require a full data scan in DuckDB. In Stratum, these are answered by traversing the pre-computed metadata at tree nodes without touching row data. SELECT AVG(price) FROM orders on 10M rows: Stratum 0.1ms, DuckDB 7.1ms.
Fused SIMD execution. Most columnar engines evaluate predicates in one pass, then apply the result mask during a separate aggregation pass. Stratum fuses these into a single loop: predicates and accumulation run simultaneously via Java Vector API VectorMask chains, processing four doubles or longs per SIMD instruction on the benchmark machine. The fused loop needs neither intermediate arrays nor a second pass, so it allocates nothing extra.
The Vector API (JDK 21+) provides DoubleVector and LongVector operations, and Stratum uses the widest vector shape the CPU supports: AVX-512 (eight doubles) on x86 processors that have it, AVX2 (four) on those that don’t, and NEON or SVE on ARM. The benchmark CPU below, an Intel Lunar Lake laptop chip, has no AVX-512, so every number in this note ran on 256-bit AVX2. The bet was that the JVM incubator API had matured enough to compete with native code on analytical workloads without the deployment complexity of a native library. The benchmarks suggest that bet paid off.
Performance
Single-threaded comparison vs DuckDB v1.4.4 (JDBC in-process) on 10M rows, Intel Core Ultra 7 258V, JVM 25. Median of 10 iterations, 5 warmup. The first 11 rows are a selection of queries where Stratum was faster; the remaining 12 are every query where DuckDB was faster:
| Query | Stratum | DuckDB | Ratio |
|---|---|---|---|
| TPC-H Q6 (filter + sum-product) | 13ms | 28ms | 2.2x faster |
| Filtered COUNT (NEQ pred) | 3ms | 12ms | 4.0x faster |
| TPC-H Q1 (7 aggs, 4 groups) | 75ms | 93ms | 1.2x faster |
| H2O Q3 (100K string groups) | 71ms | 362ms | 5.1x faster |
| H2O Q10 (10M groups, 6 cols) | 832ms | 7056ms | 8.5x faster |
| LIKE '%search%' | 47ms | 240ms | 5.1x faster |
| AVG(LENGTH(URL)) | 38ms | 170ms | 4.5x faster |
| H2O Q6 (STDDEV group-by) | 30ms | 81ms | 2.7x faster |
| H2O Q9 (CORR) | 61ms | 134ms | 2.2x faster |
| MEDIAN(price) | 68ms | 158ms | 2.3x faster |
| ROW_NUMBER window | 316ms | 426ms | 1.3x faster |
| H2O Q8 (top-2 per group, ROW_NUMBER) | 1645ms | 1208ms | DuckDB 1.4x faster |
| H2O J1 (join to 100-row table) | 41ms | 27ms | DuckDB 1.5x faster |
| ClickBench Q1 (sparse NEQ filter, COUNT) | 22ms | 4.5ms | DuckDB 4.9x faster |
| ClickBench Q7 (COUNT, 2 predicates) | 37ms | 8.7ms | DuckDB 4.2x faster |
| ClickBench Q15 (1.1M groups, COUNT) | 115ms | 74ms | DuckDB 1.6x faster |
| ClickBench Q43 (DATE_TRUNC group-by) | 74ms | 46ms | DuckDB 1.6x faster |
| ClickBench Q12 (string group-by) | 274ms | 105ms | DuckDB 2.6x faster |
| COUNT(DISTINCT), 1.1M distinct | 109ms | 61ms | DuckDB 1.8x faster |
| COUNT(DISTINCT) GROUP BY region | 31ms | 27ms | DuckDB 1.1x faster |
| NYC taxi Q2 (AVG tip by passengers) | 45ms | 23ms | DuckDB 2.0x faster |
| NYC taxi Q4 (filtered SUM by month) | 42ms | 31ms | DuckDB 1.4x faster |
| TPC-DS Q1 (GROUP BY store) | 23ms | 13ms | DuckDB 1.8x faster |
The full results for this run list 57 single-threaded comparisons at 10M rows: DuckDB was faster on the 12 above, and Stratum on the other 45, three of them by a ratio that rounds to 1.0x. DuckDB wins on sparse-selectivity filters, window-based top-N, high-cardinality hash group-by at scale (1M+ unique groups where hash tables become DRAM-bound), and COUNT(DISTINCT). Results for later Stratum versions are in the current benchmark docs.
DuckDB is an excellent system. These results show that pure JVM code can compete with a mature native engine on many common analytical queries while adding branching, which DuckDB doesn’t offer.
Branching: where it diverges
Other systems branch data. Apache Iceberg and Delta Lake keep table snapshots, lakeFS branches the files in an object store, Dolt versions a SQL database like a Git repository, and Snowflake creates zero-copy clones. Stratum’s combination is narrower: an in-process columnar dataset is an ordinary Clojure value that forks in O(1), the same SIMD engine queries every branch, and Yggdrasil can branch it together with a Datahike database and search indexes.
Each column is backed by a chunked B-tree (PersistentColumnIndex) that implements Clojure’s IPersistentCollection and IEditableCollection protocols. When you call (st/fork ds), you get a new dataset that shares all unchanged chunks with the original. No data is copied, only a new root pointer into a shared tree. Mutations through the transient protocol only write-copy the chunks they touch. A billion-row dataset costs almost nothing to fork.
require('[stratum.api :as st],
'[konserve.file-store :as fs],
'[clojure.core.async :refer [<!!]])
;; Open storage, load the orders dataset (10M rows)
def store: <!!(fs/new-fs-store("/data/stratum"))
def orders: <!!(st/load(store, "orders"))
;; Fork in O(1): structural sharing, zero data copied
def experiment: st/fork(orders)
;; Persist the fork as a named branch
<!!(st/sync!(experiment, store, "experiment"))
;; Query both branches via SQL, passing column data as a table map
st/q("SELECT SUM(price * qty) FROM t", {"t" st/columns(orders)})
;; => {:SUM(price * qty) 4821903.40} ← main branch
st/q("SELECT SUM(price * qty) FROM t", {"t" st/columns(experiment)})
;; => {:SUM(price * qty) 4401238.66} ← experiment branch
;; Time-travel: load any historical branch by name
def baseline: <!!(st/load(store, "orders-baseline"))
(require '[stratum.api :as st]
'[konserve.file-store :as fs]
'[clojure.core.async :refer [<!!]])
;; Open storage, load the orders dataset (10M rows)
(def store (<!! (fs/new-fs-store "/data/stratum")))
(def orders (<!! (st/load store "orders")))
;; Fork in O(1): structural sharing, zero data copied
(def experiment (st/fork orders))
;; Persist the fork as a named branch
(<!! (st/sync! experiment store "experiment"))
;; Query both branches via SQL, passing column data as a table map
(st/q "SELECT SUM(price * qty) FROM t" {"t" (st/columns orders)})
;; => {:SUM(price * qty) 4821903.40} ← main branch
(st/q "SELECT SUM(price * qty) FROM t" {"t" (st/columns experiment)})
;; => {:SUM(price * qty) 4401238.66} ← experiment branch
;; Time-travel: load any historical branch by name
(def baseline (<!! (st/load store "orders-baseline")))
From the server side, register-live-table! lets you expose named branches as separate SQL tables and query them with plain SQL over the PostgreSQL connection without touching the Clojure API.
Practical uses:
- Reproducible experiments: fork a dataset, run your pipeline on the fork, compare results against the original without managing separate data copies or locking the source
- Audit trails: every query result is tied to a specific database state, so you can always recover the exact snapshot that produced a given answer
- What-if analysis: branch before a bulk UPDATE, run your scenario, inspect the diff and discard it, leaving the original untouched
- Zero-ETL: Datahike is the system-of-record. Stratum queries the same versioned snapshots directly, without an extraction pipeline
For Clojure developers
If you’re coming from the Clojure ecosystem, Stratum datasets behave like ordinary Clojure values. They implement IPersistentCollection, ILookup, IEditableCollection, so tablecloth and tech.ml.dataset work with them directly as column maps. You can query with SQL strings or a Clojure DSL that composes programmatically:
require('[stratum.api :as st])
;; DSL: composable, no string manipulation
st/q(
{:from {:price prices, :qty quantities, :region regions},
:where [[:> :price 100]],
:group [:region],
:agg [[:sum [:* :price :qty]] [:count]]})
;; SQL string: same engine underneath
st/q(
"SELECT region, SUM(price * qty), COUNT(*)\n FROM orders WHERE price > 100 GROUP BY region",
{"orders" {:price prices, :qty quantities, :region regions}})
(require '[stratum.api :as st])
;; DSL: composable, no string manipulation
(st/q {:from {:price prices :qty quantities :region regions}
:where [[:> :price 100]]
:group [:region]
:agg [[:sum [:* :price :qty]]
[:count]]})
;; SQL string: same engine underneath
(st/q "SELECT region, SUM(price * qty), COUNT(*)
FROM orders WHERE price > 100 GROUP BY region"
{"orders" {:price prices :qty quantities :region regions}})
The DSL is useful when building queries programmatically, because it avoids string interpolation and the injection risk that comes with it, and results are plain Clojure maps.
The origin
This work started with Votorola, a collaborative democracy project that needed distributed state. The limitations of imperative systems led to Clojure, then to replikativ for distributed replication, then to Datahike for immutable entity-level storage. Each step sharpened the same conviction that mutability is the core problem. When data changes in place, you lose history, auditability, and the ability to reason about what a system knew at any point in time.
My PhD work on simulator-based inference at UBC’s PLAI lab reinforced this. Probabilistic systems need to fork hypotheses, accumulate evidence, and explain their reasoning, tracking not only the current state but the path that led to it. Stratum is the analytics piece of the infrastructure we’re building for that.
The ecosystem
- Datahike: immutable Datalog database, the system-of-record for structured entity data
- Stratum: SIMD-accelerated columnar SQL for analytics and scans over those same snapshots
- Proximum: version-controlled vector search (HNSW)
- Scriptum: git-like branching for full-text search (Lucene)
- Yggdrasil: unified branching across all of the above
Via Yggdrasil you can fork a Datahike database, a Stratum dataset, and a Proximum index together, with consistent snapshots across SQL, Datalog, and vector search at the same point in time.
Getting started
;; deps.edn: check https://clojars.org/org.replikativ/stratum for latest
{:deps {org.replikativ/stratum {:mvn/version "0.1.7"}}}
:jvm-opts
["--add-modules=jdk.incubator.vector" "--enable-native-access=ALL-UNNAMED"]
;; deps.edn: check https://clojars.org/org.replikativ/stratum for latest
{:deps {org.replikativ/stratum {:mvn/version "0.1.7"}}}
:jvm-opts ["--add-modules=jdk.incubator.vector"
"--enable-native-access=ALL-UNNAMED"]
# Standalone server with built-in demo tables (lineitem, taxi: 100K rows each)
java --add-modules jdk.incubator.vector \
--enable-native-access=ALL-UNNAMED \
-jar stratum-standalone.jar --demo
Requirements: JDK 21+ for 0.1.7; Stratum releases since May 2026 need JDK 22+ for the foreign-memory API.
Source: github.com/replikativ/stratum
Documentation: Query DSL · SQL Interface · Dataset API · Anomaly Detection · Benchmarks · Architecture
Source and documentation: github.com/replikativ/stratum. Feedback welcome on the Clojurians #datahike channel.
Revised in November 2026 for precision.