Skip to main content

Paimon Table Format

Description​

Apache Paimon tables (catalog / warehouse) via pypaimon. This is distinct from the standalone Paimon Row and Paimon Mosaic file formats.

Maturity: experimental. Append-only tables (no primary keys) are supported for writes.

Dependencies​

pip install iterabledata[paimon-table]
# or
pip install iterabledata[paimon]

Usage​

from iterable import open_iterable

warehouse = "/path/to/warehouse"

with open_iterable(warehouse, mode="w", iterableargs={
    "format": "paimon",
    "database": "demo",
    "table": "people",
    "create_table": True,
}) as dest:
    dest.write_bulk([
        {"id": 1, "name": "Alice"},
        {"id": 2, "name": "Bob"},
    ])

with open_iterable(warehouse, iterableargs={
    "format": "paimon",
    "database": "demo",
    "table": "people",
}) as source:
    print(source.list_tables())  # ['demo.people', ...]
    for row in source:
        print(row)

File vs table​

Format idWhat it opens
paimonWarehouse/catalog table
paimon_row / .rowStandalone Row file
paimon_mosaic / .mosaicStandalone Mosaic file

Parameters​

ParameterDescription
formatpaimon
databaseDatabase name (default default)
tableTable name
columnsOptional projection
create_tableCreate append-only table on write
batch_sizeWriter flush threshold

Error Handling​

  • Missing dependency: optional libraries raise ImportError with an install hint (pip install 'iterabledata[<extra>]' when an extra exists).
  • Write mode: read-only formats raise WriteNotSupportedError or ValueError when opened with mode="w".
  • Bad or unsupported input: may raise ValueError, OSError, or library-specific errors.
  • See Troubleshooting for decoding, detection, and engine issues.

See Also​