Data quality · 8 min read

Data quality considerations in cohort discovery

Units, windows, code systems and the difference between “no record” and “no condition” — and why missing data should never become a silent no.

Unknown is a result

When a record has no value in the required window, the honest answer is “unknown”. Counting it as “does not meet” removes people who might be eligible; counting it as “meets” admits people who might not be. Keep Unknown visible and route it to a person.

Absence of a record is not absence of a condition

For exclusion criteria, a dataset can’t prove that something never happened. Showing “no record found” — distinct from a confirmed negative — tells the screener what still needs checking.

Units and code systems

The same analyte can arrive in different units (HbA1c in % or mmol/mol) and the same diagnosis under different code systems. Convert where a defined conversion exists; otherwise mark the value as Unknown and surface it at import.

Check coverage before you run

Before a cohort run, look at how many records have a value for each criterion in its window. Low coverage predicts a large “data gaps” group.