Dataset onboarding

Bring your dataset

Have CSV, JSON, or database rows? This workspace walks you through converting your data into valid DisclosureOS records: find the right fields, validate your output, and see exactly what is missing.

30 target fields|226 completeness paths|Migration guide

Audit your source data

Understand which fields are always present, how dates are encoded, whether coordinates are real GPS fixes, and what the source's own id scheme is.

Migration guide

Write a row mapper

One function from your source row to an Observation. Guard every enum boundary, use stable source-prefixed ids, and carry the raw row in extensions.

Mapper pattern

Validate your output

Run validation on the generated JSON files. Use --json for scripted workflows and CI.

npx @disclosureos/cli validate ./out --recursive

Check completeness

See which fields are missing and what percentage of the standard your records cover. The missing-fields list tells you what to target next.

npx @disclosureos/cli completeness ./out --recursive

Fix and iterate

Update your mapper, re-export, re-validate. Quarantine failures instead of dropping them. Do not fabricate observable or origin claims from keywords.

Fix loop

Runnable examples

Three complete, type-checked examples you can run locally:

pnpm --filter @disclosureos/examples golden-path

Author one record through every layer (Nimitz case).

pnpm --filter @disclosureos/examples migration-path

Convert three source rows, quarantine failures, validate, and score completeness.

pnpm --filter @disclosureos/examples migration-csv-path

Parse CSV rows, guard empty strings, map fields, and run the same validation loop.