Thunderduck vs Dremio
Both query the data where it already lives. Dremio wraps that in a lakehouse platform — a semantic layer, reflections and engines you run. Thunderduck is just the engine: a container per query, priced before it starts and gone when it ends.
| Thunderduck | Dremio | |
|---|---|---|
| Compute model | Serverless: a container per query, sized from the query's own plan, gone when it finishes. | Coordinator plus executor engines. Engines can be sized and routed per workload, and stopped when idle on their cloud. |
| What you operate | Nothing. There is no engine to size, tune or keep alive. | Engine sizing and routing rules, reflection refreshes, and a semantic layer somebody has to curate. |
| Cost when idle | Zero. No instance exists between queries. | Nothing while an engine is stopped — but a reflection refresh is compute that runs on its own schedule, whether or not anyone asks a question. |
| Making it fast | The plan picks the machine. Each query gets the CPU, memory and DuckDB limits it actually needs — and an exact repeat is served from the last result at zero cost, with no schedule to maintain. | Reflections: materialized derivations the optimizer substitutes automatically. Very effective, and a copy with a refresh schedule. |
| Modelling layer | Catalog discovery plus column meanings, exposed to humans and agents. Modelling is your dbt project, in your repo. | A first-class semantic layer — virtual datasets, spaces, lineage and governance — that analysts and BI tools browse as the product. |
| Sources | S3, GCS, Azure, PostgreSQL, MySQL, Iceberg (Glue and Nessie), BigQuery. | A broader connector list, including relational, NoSQL and SaaS sources. |
| Writing to tables | Read-first. CREATE TABLE AS and dbt models when you want something materialized. | Full Iceberg DML from the same engine, plus table maintenance — OPTIMIZE, VACUUM, snapshot expiry. |
| Cost visibility | Planned and priced before it runs; a budget can refuse the query. The estimate is a first-class answer, not a log line. | Job history, engine-level attribution and queueing rules — the accounting lands after the work. |
| AI agents | A native MCP server: Claude and ChatGPT connect directly, with schema discovery, column meanings and a per-query cost ceiling. | MCP, Arrow Flight SQL and JDBC are all available. The budget that stops an agent mid-exploration is yours to build. |
We agree about the lakehouse and disagree about the platform
Dremio and Thunderduck start from the same conviction: the data belongs in open formats in your own object storage, and loading it into a warehouse to ask a question is a tax. Neither of us wants your Iceberg tables copied somewhere proprietary.
The difference is how much product sits on top. Dremio builds a platform around the engine — a semantic layer of curated virtual datasets, a catalog, governance, engines you assign to workloads. For an organisation that wants one modelled surface for every analyst and every BI tool, that layer is the reason to buy it, and we do not have an equivalent. Thunderduck is the engine on its own: connect a source, discover the catalog, run the query, pay for the seconds it took.
A reflection is a copy with a schedule
Reflections are the clever part of Dremio. The optimizer quietly rewrites your query to hit a materialized derivation instead of the raw files, and a dashboard that would have scanned a partition set answers immediately. When the same queries run all day over data that changes on a known cadence, that trade is excellent.
It is still a copy. It costs storage, it costs the compute that refreshes it, and it introduces a staleness window someone has to reason about — and every new question that the reflections do not cover runs at raw speed until somebody defines one. Thunderduck has nothing to define or maintain: an exact repeat of a query, from the same user within a short freshness window, is served from the prior result automatically — no schedule, no storage to provision. It only ever helps that one exact question again; a related question with a different filter still plans and runs fresh, the way it always has. When you genuinely do want a materialization, it is a CREATE TABLE AS or a dbt model that lands as an Iceberg table you own — visible in your repo, not implicit in the engine.
Priced before it runs, not attributed afterwards
Both engines will happily read more than you meant. The difference is when you find out. Thunderduck plans every query, prices it, and can refuse it against a budget before a single byte is scanned — per agent, per user, per workspace.
That is also what makes it safe to point an LLM at your data. An agent's first instinct is rarely a selective filter, and it never sees the bill. Connecting one to a SQL endpoint is easy anywhere; the part that stops it full-scanning 400 GB out of curiosity is the part we ship.
Choose Dremio when
- You want a curated semantic layer that analysts and BI tools browse as the product.
- Repeated queries over stable data, where a maintained reflection pays for itself many times a day.
- You need full Iceberg DML and table maintenance from the engine that also serves the queries.
- A platform team is ready to own engines, refreshes and the modelling layer.
Choose Thunderduck when
- You want SQL across scattered sources with nothing to size, tune, refresh or keep alive.
- Load is spiky or occasional, and an engine sized for the busy hour is hard to justify.
- AI agents will be querying, and every query should be planned, priced and bounded first.
- Credentials cannot leave your account — so the engine has to run in your cloud or on-prem.
Query in place, minus the platform.
Nothing to size, nothing to refresh, and a price on every query before it runs.