How the eligibility engine works, and what each layer is forbidden from doing.

Scope and intended use

Inclusion Health reads the eligibility criteria that sponsors publish on ClinicalTrials.gov and, for one person at a time, separates the studies that have already ruled that person out from the studies that have not. It runs in a browser tab on the reader's own device.

The intended use is triage before a conversation. The output is a shortened list and a set of quoted criteria a person can take to a clinician or a study coordinator. It is not a determination of eligibility and does not attempt to be one.

The problem being solved

Eligibility criteria are written for trial coordinators, and they are published as free text. A single record commonly contains forty to two hundred separate criteria, expressed in prose, with no structure beyond an inclusion and exclusion split that sponsors apply inconsistently.

Three properties of that text defeat naive parsing:

Negation

"Patients not currently receiving anticoagulation" and "patients currently receiving anticoagulation" differ by one word and invert the meaning. A system that matches concepts without resolving scope will place a patient in the wrong arm of the criterion.

Thresholds and units

"eGFR > 45 mL/min/1.73m²", "creatinine clearance of at least 45", and "renal function adequate per investigator" are three different kinds of statement. The first is decidable from a number, the second requires a unit conversion, and the third is not decidable by any machine.

Delegated judgement

A large fraction of criteria explicitly hand the decision to the investigator: "in the opinion of the investigator", "deemed clinically significant", "otherwise suitable". These read as criteria but contain no decidable content.

The design consequence is that the interesting engineering problem is not reading the criteria. It is knowing which ones must not be answered.

Data source and retrieval

Studies come from the ClinicalTrials.gov version 2 REST API, queried by condition, filtered to recruiting statuses. The client requests only the fields it uses: identifier, brief title, overall status, eligibility criteria text, sex, minimum and maximum age, healthy volunteer flag, site country and city, phase, study type and lead sponsor.

Retrieval follows the API's page token up to a cap of four pages of one hundred records. When that cap is reached the interface says so rather than presenting a truncated list as complete.

Architecture

The system is deliberately hybrid. A deterministic symbolic layer produces every verdict. Learned components exist only to route and surface criteria that the symbolic layer could not compile. No learned component is permitted to rule a study in or out.

This ordering is the central design claim of the system. It means a model error can cause a criterion to be shown to a reader under the wrong heading, but cannot cause a study to be wrongly excluded from that reader's list.

Layer one: structured gates

Before any text is read, four comparisons run against structured registry fields: sex, minimum age, maximum age, and whether any listed site is in the reader's country.

These are the only fields the registry publishes in a machine-readable form, and they are the only comparisons in the system that carry no interpretive risk. Where a field is absent, or the reader has not supplied the corresponding value, the gate reports "not given" rather than assuming a default.

Layer two: the rule compiler

Eligibility text is split into individual criteria, then each criterion is compiled against a hand-authored lexicon of 35 clinical concepts, each with surface forms, negation cues, and where applicable a numeric comparator with unit handling.

Negation and scope

Negation is resolved at compile time by cue detection with a bounded scope window, so that a negated concept produces a predicate with an inverted polarity rather than a missed match. The compiler records the exact character span that triggered each predicate.

Hedge detection

Before a criterion is compiled, it is tested against a set of hedge patterns that mark delegated judgement. A criterion carrying a hedge is flagged investigator and is never machine-decided, regardless of what else it contains. This is a hard gate, not a confidence penalty.

Abstention

Each compiled predicate carries a confidence. Below a threshold of 0.70 the compiler abstains: the criterion is displayed verbatim and marked as undecided rather than resolved. The threshold is fixed and applies uniformly.

Provenance

Every decision the compiler makes is displayed with the concept matched, the polarity, the numeric comparison where present, the confidence, and the literal cue text from the criterion. A reader can check any verdict against the source sentence without leaving the page.

Layer three: calibrated placement

The compiler leaves most criteria unread. A multi-label linear classifier places some of those into concept classes, so a reader can see what a criterion is about even when the system will not decide it.

Supervision

Labels come from the rule engine itself, applied across the corpus: distant, or weak, supervision over 32,628 criteria. The classifier is therefore an approximation of the compiler that generalises to surface forms the lexicon does not list. It is not trained on human eligibility judgements, and it is not a second opinion about the compiler.

Representation and model

Features

Word TF-IDF, 1 to 2 grams, 9,000 terms, plus character TF-IDF over word boundaries, 3 to 5 grams, 12,000 terms. Each block is L2-normalised separately before concatenation, so that 21,000 features do not let one representation dominate the other.

Classifier

One-vs-rest L1-penalised logistic regression over 32 classes. The L1 penalty is load-bearing: the fitted model retains 1,143 non-zero coefficients, which keeps the model small enough to ship to a browser and sparse enough to inspect by hand.

Calibration

Isotonic regression per class. Expected calibration error falls from 0.0065 to 0.0008.

Uncertainty

Split conformal prediction at alpha = 0.10, producing a per-class threshold and a prediction set rather than a point label. Classes whose conformal threshold is degenerate are dropped at training time rather than shipped with a vacuous guarantee.

The output is a set of candidate concepts with a coverage guarantee attached. It is used to group and label undecided criteria in the interface. It never changes a verdict.

Layer four: semantic recall

An optional fourth layer, switched off by default, embeds criteria that the first three layers left entirely unplaced and classifies them by nearest centroid over concept prototypes. A MiniLM sentence encoder, quantised to int8, runs in the same browser tab through ONNX Runtime on WebAssembly. Nothing is uploaded.

It is restricted to its single nearest concept. Widening it to the top two candidates was measured and rejected: on the internal set it moved precision from roughly 96% to roughly 65%, which is exactly the kind of trade this system is built to refuse.

The layer is off by default because it costs a one-time model download and because its contribution is recall, not correctness. A reader who turns it on is told what it is doing.

Question selection

After the first pass, many studies remain undecided pending facts the reader has not given. Rather than asking for a full history, the interface asks the smallest number of questions that resolves the most studies, chosen by a greedy maximum-coverage strategy over the open set.

Each question displays how many still-open studies it would settle, so the reader can stop at any point and see what stopping cost them. Answers are held in the page and are never transmitted.

Evaluation

This section is the one most likely to be over-read, so it states the measurement and its limits together.

Reported precision of 100.0% on 46 items should be read as "no error was observed in a small sample", with a confidence interval wide enough to be uninformative, not as a claim of perfection.

Coverage is reported, never assumed

Every search displays what fraction of criteria compiled, how many were decidable, how many were delegated to investigator judgement, and how many were left unread. That number moves substantially with the condition searched. It is shown because a tool that conceals how much it could not read is misrepresenting itself.

Known limitations

Registry quality

Sponsors author and update their own records. A study listed as recruiting may have closed, and published criteria may differ from the operative protocol. The tool does not verify or correct registry content.

Protocol depth

The full protocol contains criteria the registry never publishes. A study the tool cannot rule out may still exclude a person for reasons that were never public.

Lexicon coverage

35 concepts cover common eligibility patterns and miss specialised ones. Rare-disease criteria in particular are frequently left unread.

Single-rater evaluation

See section 10. This is the most important limitation and the reason this document exists.

Language

English only. Criteria published in other languages are not handled.

No outcome evidence

There is no evidence that using the tool changes enrolment, time to enrolment, screen-failure rate, or any patient-facing outcome.

Data handling

The architecture makes most privacy questions moot by never collecting the data in the first place.

Computation

Entirely client-side. The model, the lexicon and the solver all execute in the reader's browser.

Transmitted

The condition name, to the public ClinicalTrials.gov API, exactly as a search box would send it.

Never transmitted

Age, sex, country, and every answer given to the solver. These do not leave the tab.

Stored

Nothing about a person. There is no account, no analytics, no tracking, and no field anywhere on the site for entering health details.

Consequence

There is no patient database to breach, subpoena, or de-identify, because one was never created.

A content security policy served with every page restricts outbound connections to the registry and the site's own origin, so the claim above is enforced by the browser rather than resting on good intentions.

Reproducibility

The rule engine, the training pipeline, the evaluation harness and the browser inference code are all readable, and the browser implementation is verified to reproduce the Python implementation's outputs. The shipped model is a versioned artefact: the same input produces the same output on any device, with no server call and no model drift between sessions.

This matters for a study. A result obtained against a given build stays reproducible against that build, which is not true of systems that call a hosted model.

What a validation study would add

The measurement in section 10 answers "does the compiler still do what it did last week". The questions that matter clinically are untouched:

• How does the compiler perform against criteria labelled independently by multiple clinicians, on a set large enough to give a usable confidence interval?
• Does the abstention behaviour match where clinicians agree the text is undecidable?
• Does a patient using the tool arrive at a coordinator better prepared, and does that change screen-failure rates?
• Where the tool is wrong, is it wrong in ways that are dangerous or merely useless?

Designing that study is the current priority, and it is the thing this project needs from a clinical partner.