Format support matrix

undatum supports 140+ formats via iterabledata (catalog size depends on the installed iterabledata version and optional extras). Capabilities differ by format: many are read-only, some require extras, and streaming support varies.

Recommended engine: iterabledata ≥ 1.0.18 on Python 3.10+ for the full open-data pack, lakehouse writes, codec profiles, and Avro write support. Run undatum formats list to see what your install actually exposes.

How to inspect live capabilities

# Human-readable catalog
undatum formats list

# Capability matrix (read / write / streaming / maturity / native bulk)
undatum formats list --capabilities

# Named tables or sheets
undatum formats tables workbook.xlsx

# One format: memory model, selection pushdown, codecs, native bulk, example args
undatum formats describe parquet
undatum formats describe xml

# Machine-readable
undatum formats list --capabilities --json

Core formats (typical CLI workflows)

Format	Read	Write	Streaming	Notes
CSV / TSV	yes	yes	yes	Delimiter auto-detected (,;\\t|`)
JSON Lines / NDJSON	yes	yes	yes	Preferred for nested records
JSON (array/object)	yes	yes	limited	Prefer JSONL for large files
Parquet	yes	yes	batched	Requires pyarrow (default dependency); codec profiles fast/balanced/max
Arrow / Feather	yes	yes	batched	Native batch convert path in iterabledata 1.0.17+
Excel XLSX / XLS	yes	limited	sheet-based	--table / --sheet for a named sheet; --start-page for a 0-based index
XML	yes	limited	yes	Use --tagname for record elements
BSON	yes	yes	yes
AVRO	yes	yes	batched	Writable since iterabledata 1.0.14 (lazy schema inference)
ORC	yes	yes	batched	Robust schema inference for unusual column names (1.0.14+)
YAML	yes	yes	limited
DuckDB / SQLite files	yes	yes	table-based	Default table name derived from output filename when omitted (1.0.14+)

Compression codecs commonly used with the above: gz/gzip, zst, bz2, xz, lz4, zip, 7z. Codec performance profiles (fast, balanced, max) are available through iterabledata; undatum repack uses maximum compression by default.

Use undatum repack to recompress container-coded files at maximum compression, or to rewrite Parquet/ORC/AVRO with format-native compression. See repack.

Format families (non-exhaustive)

Family	Examples	Notes
Tabular text	csv, tsv, jsonl/ndjson, annotatedcsv, csvw, fwf, ssv	Core CLI path
Columnar / analytics	parquet, orc, avro, arrow, geoparquet, zarr, vortex, ddb	Many need optional deps
Lakehouse / warehouse	delta, iceberg, lance, hudi (RO), ducklake, paimon, paimon_row, paimon_mosaic	Delta/Iceberg writable in iterabledata 1.0.18+; install via pip install "undatum[lakehouse]" (or iterabledata[lakehouse])
Geospatial	geojson, geojsonseq, gml, gpx, shp, gpkg, kml, fgdb/gdb, mif, asc, e00, las, bag, czml	Open-data GIS pack largely experimental (1.0.18)
Scientific / geophysical	h5, nc, mat, segy, grib2, mseed, cif, pdb, xyz, edi	Often read-only; extras such as mat, geophysical, lidar
Business / legacy	mdb/accdb, lotus123 (123/wk1), xlsb	Access needs iterabledata[access]
Containers	zip, tar (RO multi-member), WebDataset	TAR tags records with _member (1.0.16+)
Genomic / bio	genomic_vcf, bam, sam, bed*, gff3/gtf, cram, fasta/fastq	VCF vs vCard .vcf disambiguated by content
Graph / RDF	graphml, gexf, jsonld, nt, ttl, trig, hdt	HDT via open-data pack
Logs / feeds / OTLP	log, gelf, cef, rss, kafka, OTLP JSON/Protobuf
Niche open data	iati, fst, webdataset	Experimental; optional extras

Important caveats

1. Read-only formats exist. Do not assume every format listed by formats list can be a conversion target. undatum fails fast with writable-format suggestions when you pick a read-only output. Hudi remains read-only pending Python SDK write support.
2. Extras. Some connectors need optional installs (undatum[cloud], undatum[api], undatum[extract], database drivers, or undatum extras that forward iterabledata packs: undatum[lakehouse], undatum[gis], undatum[scientific], undatum[access], undatum[compression]). You can still install iterabledata extras directly if you prefer.
3. Large files. Prefer --low-memory on convert / sort / dedup, and Parquet or JSONL over in-memory JSON arrays. Parquet/Arrow writers buffer bounded batches (iterabledata 1.0.17+).
4. Python version. Recent iterabledata releases (1.0.11+) require Python ≥ 3.10. undatum still declares requires-python >= 3.9; use Python 3.10+ to pull the latest iterabledata catalog from PyPI.
5. Security / error policy (iterabledata 1.0.16+): XML parsers disable external entities; pickle reads warn unless trust=True (undatum convert --trust, and the same flag on other read commands); malformed rows raise FormatParseError unless you pass --on-error skip or --on-error warn. Pair with --error-log errors.jsonl to keep a JSONL record of skipped rows.

For the authoritative list, always trust undatum formats list --capabilities for your installed version. Upstream release notes: iterabledata CHANGELOG.

Catalog notes

undatum supports 140+ formats through iterabledata (exact catalog depends on the iterabledata version and optional extras). Format detection is automatic from file extensions and content; override with --format-in / --format-out when needed. Run undatum formats list for the authoritative catalog on your installation. Prefer iterabledata ≥ 1.0.18 on Python 3.10+ for lakehouse writes and the open-data format pack — see the format support matrix.

Core tabular formats

Format	Extensions / ids	Notes
CSV / TSV	.csv, .tsv (csv, alias tsv)	Delimiter and encoding auto-detected
JSON Lines	.jsonl, .ndjson (jsonl, alias ndjson)	One JSON object per line; ideal for streaming
JSON	.json	Array or object documents
Parquet / ORC / Avro	.parquet, .orc, .avro	Columnar and binary row formats; Avro is writable (iterabledata 1.0.14+)
Arrow / Feather	.arrow, .feather	Bounded batch I/O; native batch convert path available
Excel	.xls, .xlsx, .xlsb, .ods	Named sheet via --table / --sheet; --start-page for a 0-based index
BSON	.bson	Binary JSON (MongoDB)
DuckDB / SQLite	.ddb, .duckdb, .sqlite, .db	Table name defaults from output filename when omitted

Structured, geospatial, scientific, and lakehouse

• XML — convert with --tagname to specify the record element (XXE-hardened parsers in iterabledata 1.0.16+)
• YAML / TOML / INI — config and metadata formats (yml, toml, ini)
• Geospatial — geojson, geojsonseq, geoparquet, fgb, gpx, shp, gpkg, kml, FileGDB (fgdb), MapInfo MIF, LAS, …
• Lakehouse — Delta and Iceberg support bounded writes (1.0.18+); also Lance, DuckLake, Paimon; Hudi remains read-only. Install with pip install "undatum[lakehouse]"
• Scientific / statistical — h5, nc, mat, segy, grib2, sas, sav, dta, and others (many read-only)
• Containers — ZIP; read-only TAR multi-member archives (tar / .tgz); WebDataset
• Graph / RDF — graphml, gexf, jsonld, nt, ttl, trig, hdt, …

Compression

Read and write through compressed containers without manual decompression: GZ, XZ, BZ2, ZIP, ZSTD, LZ4, 7Z, and other codecs supported by iterabledata. Codec profiles fast / balanced / max are available in iterabledata 1.0.17+; undatum repack defaults to maximum compression.

# Process JSONL inside a ZIP or XZ archive
undatum headers --format-in jsonl data.zip
undatum count data.jsonl.xz

Choosing a format

Use case	Recommended formats
Streaming ETL / logs	JSON Lines, CSV
Analytics / data lakes	Parquet, ORC, Avro, Delta, Iceberg
API interchange	JSON, JSON Schema
Packaging / catalogs	Frictionless Data Package (undatum package)
Geospatial pipelines	GeoJSON / GeoJSONSeq → GeoParquet

Inspect read/write capabilities before converting:

undatum formats describe parquet
undatum formats list --writable --capabilities

Per-format reader/writer details live in the upstream iterabledata formats docs.