Evaluation you can inspect
Scoring
Make the basis of an evaluation clear.
Scoring makes an evaluation inspectable: who made it, what they examined and how they reached the result. Different providers can publish their own methods while keeping those choices visible.
Explore how scoring worksHow scoring works
Start with the question being evaluated
An evaluation needs a defined purpose. Checking a source file, reviewing documentation and setting research priorities require different work. Each result needs rules suited to its purpose.
- 01 / Method
What does it measure?
Define the question, the information needed and how the result will be expressed. Name the method and retain its version.
- 02 / Evaluation
What was examined?
Identify the reviewer and the exact records supplied. Keep their reasoning available alongside any calculation.
- 03 / Result
What does it mean?
Explain the result on its own scale. Preserve missing work and disagreements so a score does not conceal either.
DisclosureOS provides a shared format for provider scores and tools for checking records. It does not assign a universal scientific grade or choose a winning interpretation.
Compare evaluations
Different reviews, visible reasons
Two organizations can examine the same record and reach different scores. Read the question, method and reasons before comparing the numbers.
DisclosureOS example case · Fictional providers
Same method, different judgments
Both teams reviewed the same seven files under the same method. Their judgment about method detail differs.
Review team A (fictional)
How well is this sample record documented for independent review?
The source explains collection, splitting, subsampling and partial extraction, and retains three readings. I count this as partial documentation even though the full analytical procedure and uncertainty budget are missing.
See the reasoning
- Source traceability25 / 25
- The reviewed observation identifies these source digests, and this example retains the selected source bytes. This does not authenticate the original sources.
- Specimen identity25 / 25
- The collection sheet and preparation protocol distinguish M-014, portions A and B, and analytical portion A1. The laboratory report assigns the copper result to A1.
- Method detail15 / 30
- The source explains collection, splitting, subsampling and partial extraction, and retains three readings. I count this as partial documentation even though the full analytical procedure and uncertainty budget are missing.
- Stated limits20 / 20
- The sources identify the unrecorded collection time, nominal masses, separate blank matrix, missing uncertainty budget and lack of a demonstrated link to the light.
Review team B (fictional)
How well is this sample record documented for independent review?
I rate the method detail as insufficient for this independent review. The narrative names a partial extraction but omits the operating procedure, recovery study and uncertainty budget needed to reconstruct the reported analysis.
See the reasoning
- Source traceability25 / 25
- The reviewed observation identifies these source digests, and this example retains the selected source bytes. This does not authenticate the original sources.
- Specimen identity25 / 25
- The collection sheet and preparation protocol distinguish M-014, portions A and B, and analytical portion A1. The laboratory report assigns the copper result to A1.
- Method detail0 / 30
- I rate the method detail as insufficient for this independent review. The narrative names a partial extraction but omits the operating procedure, recovery study and uncertainty budget needed to reconstruct the reported analysis.
- Stated limits20 / 20
- The sources identify the unrecorded collection time, nominal masses, separate blank matrix, missing uncertainty budget and lack of a demonstrated link to the light.
Both scores use the same method, subject and exact input files. Their recorded judgments remain separate.
These are authored teaching examples. Their arithmetic and file references are checked when this page is prepared. The reviewers’ decisions remain open to challenge.
Inspect the methods and example filesLevels, grades & rankings
Know what the label describes
A support level describes the information behind a characteristic. A grade summarizes a result under a named method. A ranking orders a collection using that method.
Support for a characteristic
What supports the description?
- Not indicated
- The record does not indicate the characteristic. This is not proof that it was absent.
- Reported
- An account describes the characteristic.
- Documented
- Supporting records, images or other documentation are available.
- Measured
- Instrument measurements support the characteristic.
- Confirmed
- Peer-reviewed analysis and independent verification support the characteristic.
These retained support labels do not measure how anomalous a characteristic is. They are separate from an assessment’s outcome.
Read the support-level definitionsGrades under one method
How does this method group results?
The shared method in the example defines these bands. Other providers can use a different scale or no grades at all.
- D · Substantial gaps
- 0 to under 50
- C · Partial documentation
- 50 to under 75
- B · Substantial documentation
- 75 to under 90
- A · Extensive documentation
- 90 to 100
Grade boundaries belong to the method. DisclosureOS does not prescribe a universal A–F grade or a probability that an explanation is true.
A rank also needs context
“First” means first within a specified collection, under a specified method. Changing either can change the order.
Read a check result
A pass has a specific meaning
The example case includes reviews of a soil sample and its possible connection to the field observation. Choose a saved run to see which parts were checked.
Saved software check · Fictional inputs
The sample and the field observation
Check that the review connects the intended soil sample and field observation, using the exact records supplied.
- Document structurePassed
- Required fields and permitted values in the supplied document.
- Internal consistencyPassed
- References, identities and revision relationships within the checked records.
- Linked record snapshotsPassed
- Exact supplied record versions and their cross-record references. Source material mentioned inside them is outside this check.
- Documentation profileNot checked
- Additional requirements for a specific task. No documentation profile was applied to these saved runs.
No scoring policy or documentation profile was applied to these runs. The source files retain the example’s original name, Meridian.
Missing or changed files
Unknown and failed are different results
Try three conditions for one temperature reading in the example case. The record stays the same; only the file supplied to the check changes.
Controlled example · One selected reading
The supplied file matches
The bytes match the recorded fingerprint. The reader can locate the selected temperature value in the file.
- Document structure
- Passed
- Internal references
- Passed
- Supplied file identity
- Passed
- Selected source value
- Passed
First temperature reading
weather.csv
Evaluation tools
Keep enough detail to review the result
The standard supports more than a pass or fail. Its tools preserve the scope, reasoning and remaining work behind an evaluation.
Provider scores and their methodsKeep each result attached to its author and rules
- A provider identifies what its score measures, the exact method and source records, and the reasons for its result.
- A method defines its scale, optional grades and how to handle missing information. An incomplete review can remain unscored.
- Shared rules and source material help readers compare results. They do not make a review correct or authenticate its author.
Requirements for a specific taskKnow what a profile asks for
- A named, versioned profile states the requirements being checked.
- Completion reports distinguish satisfied, missing, failed, unchecked and inapplicable requirements, with reasons and next actions.
- Research profiles can check declared calibration, timing and measurement context. They do not establish that the equipment performed as declared.
Current assessments and shared sourcesSummarize reviews without turning them into votes
- Group assessments by the question and subject they address, while preserving who made each declaration.
- Keep replaced reviews in history and collapse exact copies in the current summary.
- Show differing outcomes and shared inputs. Several reviews using the same source do not establish independent support.
The basis of a numerical scoreKeep the rules and denominator visible
- State exactly what a number measures, which inputs it includes and how any weights are applied.
- A count of populated fields measures field presence. It does not measure the usefulness or accuracy of their contents.
- The current research workflow does not calculate an overall anomaly score. Support for an observable describes a characteristic, not how extraordinary it is.
A result someone else can inspectPreserve the inputs, rules and output
- Retain the exact input versions, the evaluator and policy versions, and the individual results.
- File fingerprints help identify the same bytes when the work is repeated.
- A reproducible receipt does not authenticate a source or prove that a declared laboratory method was performed correctly.
What a result means
Reproducing a number is one part of review
The tested soil portion has three fictional copper readings. A fixed method reproduces their mean from the retained source file.
(1.21 + 1.25 + 1.29) ÷ 3
1.25 mg/kgWhat that establishes
The retained calculation reproduces the stated mean. It does not establish unusual composition, laboratory accuracy or a connection to the reported light.
Measurement uncertainty was not calculated by this method.
Keep the question behind the number
More populated fields do not necessarily make a record more useful. More reviews do not necessarily mean more independent sources. A well-documented observation can still have an unresolved explanation.
Explore or build
Put the standard to work
From the original record to an evaluation someone else can inspect.