Read-only copy. This public copy of trans-bench is read-only: it shows the published campaigns. Clone the repository to run benchmarks or to edit systems. github.com/Sirneij/trans-bench, branch verified-rerun-2026
Campaign · with memory

verified_2026_v2

Verified campaign with memory measurements, September 2026. This campaign holds the measurements of the paper Database System Performance on Recursive Queries (Section 4). It is a complete re-run of the first 2026 campaign (verified_2026) with the present harness, in which every run is a separate process with a 600 s limit and every result is checked. Unlike the first campaign, it also records the memory each query used and the kind of every failure.

Runs executed
12,289
9 series, plus 528 skipped
Completed and verified
12,215
99.40% of executed runs
Failed
74
23 timeout, 13 out of memory, 36 unsupported, 2 iteration limit
Wall-clock span
—
2026-09-29 02:26 → 2026-09-30 17:01 UTC

Outcome by series

completedtimeoutout of memory / errorunsupported / limitskipped
CockroachDB 1,450
DuckDB 2,582
MariaDB 1,802
MariaDB (4 GB tmp) 1,200
MongoDB 606
Neo4j 667
PostgreSQL 1,478
SingleStore 1,054
XSB 1,978

Verification

Incorrect results (row count or hash differs from the reference closure):
  • duckdb/double_recursion: binary_tree, cycle, grid, multi_path, path, reverse_binary_tree, y
  • mariadb/right_recursion: scale_free
Scale-free and Barabási–Albert graphs (no reference closure): the systems agree on 17 of 19 graph sizes.

Environment

date
2026-09-29T02:26:35Z
machine
Apple M3 Pro, 11 cores, 18 GB
os
macOS 27.0 (26A428)
python
Python 3.12.14
xsb
not captured
postgresql
PostgreSQL 17.11 (Homebrew) on aarch64-apple-darwin25.6.0, compiled by Apple clang version 21.0.0...
mariadb
/opt/homebrew/opt/mariadb/bin/mariadbd Ver 13.0.2-MariaDB for osx10.23 on arm64 (Homebrew)
cockroachdb
v26.3.2
mongodb
db version v8.3.11
neo4j
2026.09.0, APOC: apoc-2026.09.0-core.jar
java
openjdk version "21.0.12.1" 2026-08-18
trans-bench commit
bc0149ff17ddc440f7395b09b553ae8da20452b5 + uncommitted changes (code.patch)
singlestore image
ghcr.io/singlestore-labs/singlestoredb-dev:0.2.62...