Awesome

DuckDB Clickhouse Native File reader

This experimental rust extension allows reading ClickHouse Native Format database files.

Experimental: USE AT YOUR OWN RISK!

Status

• Basic Fomat Reading
• Column Extraction
• Blocks Parser & Iterator
• Type Mapping for complex fields
• Compression support

Input

Generate some native files with clickhouse-local or clickhouse-server

--- simple w/ one row, two columns
SELECT version(), number FROM numbers(1) INTO OUTFILE '/tmp/numbers.clickhouse' FORMAT Native;
--- simple w/ one column, 100 rows
SELECT number FROM numbers(100) INTO OUTFILE '/tmp/100.clickhouse' FORMAT Native;
--- complex w/ multiple types
SELECT * FROM system.functions LIMIT 10 INTO OUTFILE '/tmp/functions.clickhouse' FORMAT Native;

Usage

Read ClickHouse Native files with DuckDB. Unoptimized full-file reading.

D SELECT * FROM clickhouse_native('/tmp/numbers.clickhouse');
┌──────────────┬─────────┐
│  version()   │ number  │
│   varchar    │  int32  │
├──────────────┼─────────┤
│ 24.12.1.1273 │ 0       │
└──────────────┴─────────┘

D SELECT count(*), max(number) FROM clickhouse_native('/tmp/100000.clickhouse');
┌──────────────┬─────────────┐
│ count_star() │ max(number) │
│    int64     │    int32    │
├──────────────┼─────────────┤
│       100000 │       99999 │
└──────────────┴─────────────┘