Data quality and stewardship are the two halves of governance. AWS supplies a capable managed service for the first and nothing for the second. This document covers what Glue Data Quality measures, precisely where it stops, and the shape of the steward application you must build.

AWS Glue Data Quality

AWS describes it as a service that "allows you to measure and monitor the quality of your data so that you can make good business decisions. Built on top of the open-source DeeQu [sic] framework, AWS Glue Data Quality provides a managed, serverless experience." Rules are written in DQDL, "a domain specific language that you use to define data quality rules", with "25+ out-of-the-box DQ rules to start from".

The quality score is defined simply: "The percentage of data quality rules that pass (result in true) when you evaluate a ruleset." That gives a single trackable number per source per load, which is the metric a steward actually manages.

The two entry points differ, and the difference decides your design

"There are two entry points for AWS Glue Data Quality: the AWS Glue Data Catalog and AWS Glue ETL jobs." They are not equivalent, and the asymmetry is the single most practical thing to know:

Feature	Data Catalog entry point	ETL jobs entry point
Data Quality rule recommendations	Supported	Not supported
Author and run DQDL rules	Supported	Supported
Identifying records that failed checks	Not supported	Supported
Auto scaling	Not supported	Supported
ML-based anomaly detection	Supported	Supported
EventBridge and CloudWatch integration	Supported	Supported
Incremental data quality	Pushdown predicates	Glue bookmarks

Read rows one and three together. Recommendations come from the catalog; the identity of failing rows comes from ETL. So the sensible pattern is to use the catalog entry point once, at design time, to have AWS propose a starting ruleset — "AWS Glue Data Quality quickly analyzes your data and creates data quality rules for you" — then move those rules into the ETL job, where you can quarantine the rows that failed. AWS's own framing supports this: the ETL entry point "lets you perform proactive data quality tasks. Proactive tasks help you identify and filter out bad data before you load a data set into your data lake."

Rules that matter for a healthcare master

DQDL rule types are the enforcement point for everything matching depends on. The ones that earn their place here:

Concern	Rule shape	Why it protects matching
Identifier completeness	Completeness on MRN and assigning-system columns	A null system URI gives false links
Identifier format	ColumnValues matching an NPI or MRN pattern	Malformed identifiers match nothing
Uniqueness within source	Uniqueness on the source primary key	Duplicate Unique IDs are rejected
Birth date plausibility	ColumnValues within a sane date range	A default date becomes a mass false match
Referential integrity	Referential integrity between two datasets	Orphan provider references
Volume sanity	RowCount, or a dynamic rule on a trailing average	A truncated feed shrinks the master
Freshness	FileFreshness on the landed objects	A stalled feed looks like deletions

Two capabilities worth naming. Anomaly detection is ML-based and generally available — the release notes record "Anomaly Detection and Dynamic Rules are now generally available" in August 2024 — and dynamic rules allow thresholds relative to history, such as a row count compared with an average of the last ten runs. That is the right tool for "this feed looks wrong today" without hand-maintained thresholds.

The limits, stated

"Data quality rules can't evaluate nested or list-type data sources." FHIR-shaped data is deeply

nested, so flatten before evaluating. This is a real pipeline stage, not a footnote.

2,000 rules per ruleset; 65 KB per ruleset. Split by entity and source.
"there is a limit of 100,000 statistics per account, and these statistics will be retained for a

maximum of two years." Long-horizon quality trending needs your own store.

Billing is per DPU-hour at $0.44, with recommendation and evaluation tasks "provisioned with ... 2

DPUs with a 1 minute minimum billing duration", and anomaly detection at "1 DPU per statistic". Quality checks inside an ETL job carry no separate charge beyond the job's runtime.

What Glue Data Quality does not do

It measures and reports. It does not remediate, it does not route an exception to a human, and it does not track whether anyone acted. It emits to EventBridge and CloudWatch, which is the hook — but the queue, the assignment, the resolution and the audit of resolution are yours.

The steward application

This is the largest single gap in the pattern and the honest answer to why licensed MDM products still sell. No service cited in this set provides a steward workbench. Built properly it is months of work, and it is a PHI-handling application with its own authentication, authorisation and audit obligations.

Why a human is unavoidable

Both matchers produce assertions that need adjudication. Entity Resolution's ML matching returns match groups with "an associated confidence score of 0.0-1.0", which is precisely an invitation to a threshold and a review band. HealthLake's resource matching is explicit that review is the intended workflow: "Because matching is non-destructive, you review the resulting links and decide how to act on them." And it "does not apply priority ordering or conflict resolution between identifiers: any shared, non-placeholder identifier is enough to link two resources" — so a shared placeholder-like identifier that slipped past filtering produces links only a human will catch.

In healthcare the asymmetry of error is severe. A false merge joins two patients' clinical histories and is a patient-safety event. A false split is a care-coordination failure. Neither should be decided by a threshold alone.

The screens

Screen	Purpose
Review queue	Candidate matches in the uncertainty band, prioritised by risk
Side-by-side compare	Two or more candidates, attribute by attribute, with source and date
Attribute override	Pick a winning value per attribute, with a mandatory rationale
Merge and unmerge	Join or split masters; unmerge must be a first-class operation
Search	Find a master or source record by identifier, name or demographic
Quality exceptions	Rows quarantined by Glue Data Quality, with assignment and resolution
Decision history	Every decision on this master, by whom, when and why
Match tuning	Effect of a proposed rule change against a labelled set, before deployment

Unmerge deserves emphasis because it is routinely under-built. A merge is easy; reversing one after six months of downstream propagation requires that you kept the pre-merge state and that every consumer can handle a master identifier splitting in two. Design the split event into the publish contract from the start.

Amazon OpenSearch Service is the right engine behind the search and queue screens: steward search is fuzzy, multi-field and latency-sensitive, which is not what Aurora is for. Treat it as what it is — a second full copy of patient demographics, in scope for the BAA, with its own encryption, fine-grained access control and snapshot obligations, and a requirement that merges, splits and amendments propagate into the index rather than leaving a stale name searchable. Lake Formation governs what a steward sees in the curated zone, with permissions "enforced using granular controls at the column, row, and cell-levels"; the console enforces its own row scoping on the master.

The decision store, and why it is an input

The critical design property: a steward decision must be an input to the next pipeline run, not a log of the last one. Bulk processing "re-processes all data for any new matches and updates to existing matches", and a steward's verdict is not part of that job's state. Without replay, the next run silently reverses human judgement — and in a clinical setting, re-merging two patients a clinician separated is an incident.

So the resolve stage runs in a fixed order:

  match output (S3)        steward_decision (Aurora)      survivorship rules (Aurora)
         |                          |                              |
         v                          v                              v
  +-------------------------------------------------------------------------+
  | 1. load match groups and match IDs                                      |
  | 2. resolve match IDs to master_id via the crosswalk                     |
  | 3. APPLY OVERRIDES: reject_match, merge, split   <- human wins          |
  | 4. evaluate survivorship per attribute on what remains                  |
  | 5. write master_attribute versions; bump master_entity.version          |
  | 6. emit change events to EventBridge                                    |
  +-------------------------------------------------------------------------+

Step 3 before step 4, always. The whole stage must be deterministic and idempotent: the same match output plus the same decision store must produce the same master, so that reprocessing is safe and a disputed value can be explained from stored inputs rather than from a log.

Measuring match quality

You cannot tune what you do not measure, and no service measures this for you. Build a labelled evaluation set — a few thousand record pairs adjudicated by humans — and run it on every rule change, reporting precision, recall and the volume landing in the review band. Be clear about what that set is: real patient pairs, so the evaluation harness is a PHI workload. It and the match tuning screen belong in the Data OU under production controls, not in a lower environment; only synthetic data goes to non-production. This is the artefact that makes a rule change a decision rather than a guess, and in healthcare it is also the evidence that the matching configuration was validated. Keep it versioned next to the rules.

Two levers the documentation hands you for the tuning itself. Rule order matters, because the default waterfall means "records matched at a higher rule level are excluded from subsequent rules", and AWS advises you "order your rules to find more precise matches first". And rule-based output carries the rule number, so "the rule number can demonstrate the precision of each match group such that rule one is more precise than rule two" — which lets you set a review threshold per rule rather than one threshold for everything.

The reference data gap

One further absence. A healthcare master depends on code sets that change on a published schedule: ICD-10-CM, SNOMED CT, RxNorm, NPI registry extracts, state licence files, postal reference data. Comprehend Medical links to some of these — and pins a version, naming "the 2021 version" of ICD-10-CM on its page — but nothing cited here manages the lifecycle of your own reference data: versioning, effective dating, approval and the re-evaluation of golden records when a code is retired. That is a modest but real build, and it is the kind of thing that is forgotten until the first annual code update breaks a survivorship rule.