Skip to main content

Quality and packaging

Assess quality, encode reusable rules, and produce evidence before data is released.

Gate a dataset release​

undatum validate data.csv --rules rules.yml --format-out json \
  --violation-report violations.json --fail-on-warnings

Example rule files live in the examples/validation-rules directory; the rule library lists the built-in formats (dates, phone numbers, ISO codes, IBAN, ...), unique and references.

Report quality against thresholds​

undatum quality data.csv --rules rules.yml -o report.html

--thresholds turns the report into a data contract: the command exits with 1 when a threshold fails. See quality.

Catch schema drift​

undatum diff --schema data.csv data.jsonl
undatum schema-drift data.csv data.jsonl --fail-on removed,type

Detect unintended changes​

undatum diff previous.parquet current.parquet --key id --ignore-order \
  --max-changed-rows 0 --summary-only

Publish a Frictionless package​

undatum package create data.csv --package-dir release --output release/datapackage.json
undatum package validate release/datapackage.json

Prepare a safe public extract​

undatum mask source.csv --fields email,phone --method hash --salt "$SALT" --output public.csv
undatum doc public.csv --pii-detect --pii-mask-samples --output DATASET.md

See quality, validate, schema-drift, package, mask, and doc.