Iceberg v3: tracking

[subkanthi]

Tracking: Iceberg v3 Feature Support in Altinity/ClickHouse

Tracks every Iceberg v3 feature area in the Antalya fork: Altinity PRs, features inherited from upstream ClickHouse via sync into antalya-26.6, and remaining work.

Legend: merged = [x], open/WIP = [~], not started = [ ]

1. Schema-level support for new v3 types

• timestamp_ns / timestamptz_ns — inherited from upstream #97132 (in antalya-26.6 via upstream sync). Altinity fix open: #2239 (nanosecond partition pruning + reject DateTime64(9) on format-version < 3).
• [~] unknown — #2363 (open; branch iceberg_unknown_data_type).
• variant — no Altinity PR; upstream open #117198.
• geometry / geography — inherited from upstream #103113 (in antalya-26.6; read behind allow_experimental_geo_types_in_iceberg). Altinity backport #2037 was closed unmerged, superseded by the upstream sync.

2. Column default values (schema evolution)

• initial-default / write-default — not started; schema evolution currently fills ClickHouse type defaults instead.

3. Multi-argument transforms

• [~] #2318 (open; read source_ids without throwing). Upstream does not plan full support (not in the Java implementation).

4. Binary deletion vectors

• Puffin deletion-vector-v1 read — #2183 (earlier attempt #2119 closed).
• Databricks/Delta .bin deletion vectors — #2271.
• Puffin file reader (foundation) — #2179 (backport of upstream #103936).
• [~] Port to Antalya 26.8 — #2379 (open).
• DV write — not implemented; v3 mutations fail closed in Mutations.cpp.

5. Table-level encryption metadata

• Not started; manifest compaction currently rejects files with key_metadata instead of preserving it.

6. Puffin-based table/partition statistics

• Not started; Puffin reader exists but stats are not consumed for scan planning.

7. Row lineage

• Tolerate v3 reserved field IDs in native Parquet reader — #2198 (backport of upstream #107377).
• [~] _row_id / _last_updated_sequence_number virtual columns for CDC — #2308 (open backport of upstream #115603).
• CDC write path (first-row-id, next-row-id, added-rows on commit) — no Altinity PR yet; upstream #116171.

8. Format-version 3 general handling

• Create v3 tables via iceberg_format_version = 3; minimal v3 test coverage — inherited from upstream #104849 (in antalya-26.6).
• Tolerate external format-version upgrades — inherited from upstream #100407 (in antalya-26.6).
• [] Fail-closed guards: DELETE/UPDATE and manifest compaction rejected on v3 — inherited from upstream #111617.

Known gaps / fail-closed behavior

• Mutations and DV writes throw on v3; manifest compaction disabled on v3; v3 manifest writes reuse the v2 Avro schema (no DV/row-id fields); timestamp_ns write path missing.