Building on the Standard

End to end: ingest real data, structure it, claim what's anomalous, score it, ship it

This guide builds a small but real research tool on the standard: ingest a dataset, structure it as observations, add evaluations, score the corpus, and expose it. It's the golden path at project scale.

What we're building

A "case triage" service: it ingests reports, validates them, and surfaces the most compelling cases — with their disagreements visible. The same skeleton extends to FOIA trackers, archive search, map apps, or analysis pipelines.

Terminal

mkdir case-triage && cd case-triage && npm init -y && npm pkg set type=module
npm install @disclosureos/records @disclosureos/observables @disclosureos/origins @disclosureos/scoring @disclosureos/schema
npm install -D typescript @types/node @disclosureos/cli

1. Ingest: source rows → observations

Write one mapper per source. Keep raw source data in extensions, mark everything draft, and let the factory validate:

src/ingest.ts

import { createObservation } from '@disclosureos/records/factories';
import type { Observation } from '@disclosureos/records';

export function fromReport(report: SourceReport): Observation {
  return createObservation(
    {
      temporal: { date: report.date, dateCertainty: 'approximate' },
      location: {
        name: report.place,
        country: report.country,
        latitude: report.lat,
        longitude: report.lng,
        siteType: 'unknown',
      },
      summary: report.narrative.slice(0, 280),
      description: report.narrative,
      dataSourceId: report.sourceName,
      extensions: { ingest: { raw: report.id, batch: process.env.BATCH_ID } },
    },
    { id: `${report.sourceName}-${report.id}`, status: 'draft' },
  );
}

The full honest-mapping playbook — fuzzy dates, enum guards, quarantining failures — is in Data Migration.

2. Evaluate: claims, attributed and cited

Evaluation is a separate pass from ingestion, done by people (or pipelines) who sign their work:

src/evaluate.ts

import { createObservableClaim } from '@disclosureos/observables';
import { createOriginClaim } from '@disclosureos/origins';
import { evidenceRef } from '@disclosureos/records/shared';
import type { EnrichedObservation } from '@disclosureos/schema';

export function addRadarAssessment(obs: EnrichedObservation, sensorId: string): void {
  const REF = evidenceRef('sensor', sensorId);

  obs.observableAssessments ??= {};
  obs.observableAssessments.technology ??= {};
  (obs.observableAssessments.technology.instantaneous_acceleration ??= []).push(
    createObservableClaim('measured', {
      confidence: 0.7,
      rationale: 'Track shows velocity change inconsistent with known aerospace capability.',
      evidenceRefs: [REF],
      evaluatedBy: 'case-triage-team',
    }),
  );

  (obs.origin ??= []).push(
    createOriginClaim('1.1', 0.5, {
      rationale: 'Physical craft indicated by sensor track; insufficient data for deeper classification.',
      evidenceRefs: [REF],
      evaluatedBy: 'case-triage-team',
    }),
  );
}

The discipline that keeps this credible is covered in Claiming Observables and Claiming Origins.

3. Validate: one gate, everywhere

Every record entering storage passes the full contract:

src/store.ts

import { parseEnrichedObservation } from '@disclosureos/schema';

export function assertValid(record: unknown) {
  const result = parseEnrichedObservation(record);
  if (!result.success) {
    throw new Error(result.issues.map((i) => `${i.path}: ${i.message}`).join('\n'));
  }
  return result.data!;
}

And the data directory is checked in CI:

Terminal

npx disclosureos validate ./data --recursive --strict

Storage layer: see Supabase Integration for the JSONB-with-promoted-columns pattern.

4. Triage: scoring with disagreement visible

src/triage.ts

import { rankByCompellingness, getCompleteness } from '@disclosureos/scoring';
import type { EnrichedObservation } from '@disclosureos/schema';

export function triage(corpus: EnrichedObservation[]) {
  return rankByCompellingness(corpus).map(({ observation, result }) => ({
    id: observation.id,
    summary: observation.summary,
    score: result.score,
    range: result.range,            // always ship the range...
    contested: result.contested,    // ...and the contested flag
    completeness: getCompleteness(observation).percentage,
    scoringVersion: result.scoringVersion,
  }));
}

A UI built on this shows: the strongest cases first, a visible band of disagreement, and a completeness meter telling researchers where work is needed. That's the case-file dashboard pattern — score, claims, and evidence, each one click apart.

5. Expose: the API is the contract

Because the wire format is the standard, your API needs no bespoke DTOs:

src/api.ts

export async function GET() {
  const corpus = await loadPublished();          // EnrichedObservation[]
  return Response.json(triage(corpus));
}

export async function POST(request: Request) {
  const obs = assertValid(await request.json()); // 400 on contract violations
  await save(obs);
  return Response.json({ id: obs.id }, { status: 201 });
}

Any other tool in the ecosystem can consume your GET and contribute via your POST — including tools in other languages validating against the JSON Schema artifact. That interoperability is the entire reason the standard exists.

Where this goes

Add institutional attribution — claims from verified institutions, surfaced with badges; the evaluatorWeight hook in compellingness is the seam.
Embed a map — observations carry coordinates, so the same records render geospatially with any mapping library.
Publish your dataset — a directory of validated observation JSON is a publishable, interoperable dataset. Ship it with disclosureos validate in CI and a schemaVersion stamp.

Supabase Integration

Store and query Observation records in Postgres — JSONB document with promoted columns

Contributing

How to contribute to the DisclosureOS standard: setup, conventions, schema gates, and pull requests

Building on the Standard

End to end: ingest real data, structure it, claim what's anomalous, score it, ship it

What we're building

Terminal

mkdir case-triage && cd case-triage && npm init -y && npm pkg set type=module
npm install @disclosureos/records @disclosureos/observables @disclosureos/origins @disclosureos/scoring @disclosureos/schema
npm install -D typescript @types/node @disclosureos/cli

1. Ingest: source rows → observations

Write one mapper per source. Keep raw source data in extensions, mark everything draft, and let the factory validate:

src/ingest.ts

import { createObservation } from '@disclosureos/records/factories';
import type { Observation } from '@disclosureos/records';

export function fromReport(report: SourceReport): Observation {
  return createObservation(
    {
      temporal: { date: report.date, dateCertainty: 'approximate' },
      location: {
        name: report.place,
        country: report.country,
        latitude: report.lat,
        longitude: report.lng,
        siteType: 'unknown',
      },
      summary: report.narrative.slice(0, 280),
      description: report.narrative,
      dataSourceId: report.sourceName,
      extensions: { ingest: { raw: report.id, batch: process.env.BATCH_ID } },
    },
    { id: `${report.sourceName}-${report.id}`, status: 'draft' },
  );
}

The full honest-mapping playbook — fuzzy dates, enum guards, quarantining failures — is in Data Migration.

2. Evaluate: claims, attributed and cited

Evaluation is a separate pass from ingestion, done by people (or pipelines) who sign their work:

src/evaluate.ts

import { createObservableClaim } from '@disclosureos/observables';
import { createOriginClaim } from '@disclosureos/origins';
import { evidenceRef } from '@disclosureos/records/shared';
import type { EnrichedObservation } from '@disclosureos/schema';

export function addRadarAssessment(obs: EnrichedObservation, sensorId: string): void {
  const REF = evidenceRef('sensor', sensorId);

  obs.observableAssessments ??= {};
  obs.observableAssessments.technology ??= {};
  (obs.observableAssessments.technology.instantaneous_acceleration ??= []).push(
    createObservableClaim('measured', {
      confidence: 0.7,
      rationale: 'Track shows velocity change inconsistent with known aerospace capability.',
      evidenceRefs: [REF],
      evaluatedBy: 'case-triage-team',
    }),
  );

  (obs.origin ??= []).push(
    createOriginClaim('1.1', 0.5, {
      rationale: 'Physical craft indicated by sensor track; insufficient data for deeper classification.',
      evidenceRefs: [REF],
      evaluatedBy: 'case-triage-team',
    }),
  );
}

The discipline that keeps this credible is covered in Claiming Observables and Claiming Origins.

3. Validate: one gate, everywhere

Every record entering storage passes the full contract:

src/store.ts

import { parseEnrichedObservation } from '@disclosureos/schema';

export function assertValid(record: unknown) {
  const result = parseEnrichedObservation(record);
  if (!result.success) {
    throw new Error(result.issues.map((i) => `${i.path}: ${i.message}`).join('\n'));
  }
  return result.data!;
}

And the data directory is checked in CI:

Terminal

npx disclosureos validate ./data --recursive --strict

Storage layer: see Supabase Integration for the JSONB-with-promoted-columns pattern.

4. Triage: scoring with disagreement visible

src/triage.ts

import { rankByCompellingness, getCompleteness } from '@disclosureos/scoring';
import type { EnrichedObservation } from '@disclosureos/schema';

export function triage(corpus: EnrichedObservation[]) {
  return rankByCompellingness(corpus).map(({ observation, result }) => ({
    id: observation.id,
    summary: observation.summary,
    score: result.score,
    range: result.range,            // always ship the range...
    contested: result.contested,    // ...and the contested flag
    completeness: getCompleteness(observation).percentage,
    scoringVersion: result.scoringVersion,
  }));
}

5. Expose: the API is the contract

Because the wire format is the standard, your API needs no bespoke DTOs:

src/api.ts

export async function GET() {
  const corpus = await loadPublished();          // EnrichedObservation[]
  return Response.json(triage(corpus));
}

export async function POST(request: Request) {
  const obs = assertValid(await request.json()); // 400 on contract violations
  await save(obs);
  return Response.json({ id: obs.id }, { status: 201 });
}

Where this goes

Add institutional attribution — claims from verified institutions, surfaced with badges; the evaluatorWeight hook in compellingness is the seam.
Embed a map — observations carry coordinates, so the same records render geospatially with any mapping library.
Publish your dataset — a directory of validated observation JSON is a publishable, interoperable dataset. Ship it with disclosureos validate in CI and a schemaVersion stamp.

Supabase Integration

Store and query Observation records in Postgres — JSONB document with promoted columns

Contributing

How to contribute to the DisclosureOS standard: setup, conventions, schema gates, and pull requests

Building on the Standard

What we're building

1. Ingest: source rows → observations

2. Evaluate: claims, attributed and cited

3. Validate: one gate, everywhere

4. Triage: scoring with disagreement visible

5. Expose: the API is the contract

Where this goes

On this page

Building on the Standard

What we're building

1. Ingest: source rows → observations

2. Evaluate: claims, attributed and cited

3. Validate: one gate, everywhere

4. Triage: scoring with disagreement visible

5. Expose: the API is the contract

Where this goes

On this page