--- cuppy_hub: aws cuppy_path: examples/healthcare-master-data/01-reference-architecture.md cuppy_section: docs cuppy_doc: Y4DQ-4KEH cuppy_doc_uuid: 5945d4de-cf6c-483e-b828-789433bc63d6 cuppy_permalink: https://cuppy.io/aws/d/Y4DQ-4KEH cuppy_version: 1 cuppy_updated: 2026-10-23T16:16:52.575Z cuppy_canonical: https://cuppy.io/aws/docs/examples/healthcare-master-data/01-reference-architecture cuppy_mcp: https://cuppy.io/mcp cuppy_body_sha256: 99aa7cc5fe36abf82c7f7d8be13136ef9847647e18316a44a4b04f259e8ddf0b # cuppy_body_sha256 is over the document body ALONE, excluding both generated blocks. --- # Reference architecture for a HIPAA healthcare master ## The scale this set is sized against Every estimate in this set assumes one stated scale, named here so that the arithmetic elsewhere is checkable. Substitute your own figures before reusing any total. | Dimension | Assumed value | | --- | --- | | Source systems | 6 — two EHRs, registration, claims, provider roster, patient portal | | Patient source records | 12 million, resolving to roughly 8 million distinct patients | | Provider records | 400,000, resolving to roughly 300,000 distinct providers | | Organisations and locations | 15,000 and 25,000 | | Monthly changed records | 2 million across all entity types | | Synchronous lookups | 200,000 per month from registration screens | | Region | US East (N. Virginia), single Region, DR to a second Region | | Clinical data volume in FHIR | 2 TB growing 100 GB per month | These are assumptions, not measurements. They exist so the cost model in document 07 has something to multiply. ## The zone model Four zones, each with a different retention and access posture. The zone model is what makes the architecture auditable: every master value can be traced back to an immutable object. | Zone | Store | Purpose | Mutability | | --- | --- | --- | --- | | Raw | S3 with Object Lock | Every inbound payload, exactly as received | Immutable | | Curated | S3, Glue Data Catalog | Standardised, quality-checked, matchable | Rewritable | | Master | Aurora PostgreSQL | Golden records, crosswalk, steward decisions | Versioned | | Interoperability | HealthLake | FHIR R4 clinical and demographic view | Transactional | Raw uses S3 Object Lock, which "uses a write-once-read-many (WORM) model to store objects" and "works only in buckets that have S3 Versioning enabled". Governance mode is the right default for raw: it protects objects "against being deleted by most users" while allowing a holder of `s3:BypassGovernanceRetention` to act, and that bypass requires an explicit `x-amz-bypass-governance-retention:true` header. Compliance mode is stronger — an object "can't be overwritten or deleted by any user, including the root user in your AWS account", and "the only way to delete an object under the compliance mode before its retention date expires is to delete the associated AWS account". Reserve it for the audit log, and understand the irreversibility before choosing it. ## End-to-end flow ``` SOURCES INGEST / LAND CURATE + MEASURE ------- ------------- ---------------- EHR (HL7v2, FHIR) --\ API Gateway + Lambda Glue ETL (Spark) Registration -------+-> Kinesis Data Streams --> - standardise, flatten Claims / payer -----+ or Amazon MSK - Glue Data Quality (DQDL) Provider roster ----+ | - quarantine failing rows Portal / CRM -------/ v | S3 RAW (Object Lock, v versioned, KMS CMK) S3 CURATED + Glue Data Catalog | (Lake Formation governed) | | | v | MATCH (Step Functions, serialised) | +--------------------------------+ | | AWS Entity Resolution | | | rule-based workflow (primary) | | | ML workflow (second pass) | | | GenerateMatchId (synchronous) | | +--------------------------------+ | | match groups + match IDs (S3) | v | RESOLVE (CUSTOM CODE) | +--------------------------------+ +-------------------->| crosswalk: matchId -> master id| lineage back to | survivorship rule evaluation | the raw object | steward override replay | | version + provenance per attr | +--------------------------------+ | +------------------+------------------+ v v MASTER (Aurora PostgreSQL) HealthLake (FHIR R4) golden records, crosswalk, resource matching -> steward decisions, audit Linkage (preview) | | v v PUBLISH (IDs only) FHIR R4 APIs, outbox -> EventBridge Bulk Data export, -> SNS / SQS per consumer SMART on FHIR -> S3 extract for analytics | | v +-----------> CONSUMERS <-------------+ Athena, Redshift, OpenSearch, downstream apps, steward console ``` ## Every service's role | Service | Role in this architecture | | --- | --- | | Amazon S3 | Raw and curated zones; match output; extracts. Object Lock for immutability | | AWS KMS | Customer managed keys per zone; key policy is the second access control | | AWS Glue (ETL) | Standardise, flatten FHIR-shaped data, write curated tables | | AWS Glue Data Catalog | The table definitions Entity Resolution reads from | | AWS Glue Data Quality | DQDL rulesets, scores, and quarantine of failing rows | | AWS Lake Formation | Column, row and cell-level permissions over curated data | | AWS Entity Resolution | Matching; produces match groups and match IDs | | AWS HealthLake | FHIR R4 store, FHIR APIs, Bulk Data export, integrated NLP | | Amazon Aurora PostgreSQL | The master: golden records, crosswalk, steward decisions | | Aurora reader, second AZ | Failover target for the synchronous registration path | | AWS Step Functions | Orchestrates the pipeline and serialises matching jobs | | AWS Lambda | Transformations, survivorship evaluation, synchronous lookups | | Amazon Kinesis / MSK | Streaming ingestion of ADT and registration events | | Amazon EventBridge | Publishes master-change events; also a HealthLake event target | | Amazon SNS / SQS | Per-consumer fan-out with retry and a dead-letter queue | | Amazon Athena | SQL over curated data and over HealthLake data | | Amazon Redshift | Analytical consumption where a warehouse already exists | | Amazon OpenSearch Service | Steward search over candidate matches | | Amazon Comprehend Medical | PHI and clinical entity detection in free text | | Amazon Macie | Sensitive-data discovery over the S3 zones | | AWS CloudTrail | API audit, including data events on the PHI buckets | | AWS Config | Conformance pack `Operational-Best-Practices-for-HIPAA-Security` | | AWS Security Hub CSPM | Aggregated findings across accounts | | IAM Identity Center | Human access; Cognito where patients or partners authenticate | | Amazon VPC and PrivateLink | Private connectivity; no PHI path over the public internet | | AWS Backup | Backup and Vault Lock for Aurora, S3, DynamoDB | | AWS Control Tower | Landing zone, organisational units, guardrails | HealthLake's own documentation names IAM, S3, CloudTrail, CloudWatch, CloudFormation, AWS PrivateLink, EventBridge, AWS Lake Formation and Amazon Athena among its related services, so the integration points above are documented rather than inferred. ## Account boundaries A single account is the wrong answer. Use AWS Control Tower to establish organisational units, and separate accounts by blast radius and by whether PHI is present. ``` Organization (AWS Organizations, Control Tower landing zone) | +-- Security OU | +-- Log archive CloudTrail org trail, Config history. No PHI. Object Lock. | +-- Security tooling Security Hub CSPM, Macie delegated admin, GuardDuty | +-- Infrastructure OU | +-- Network Transit gateway, egress, PrivateLink endpoints | +-- Shared services CI/CD, artefact store. No PHI. | +-- Data OU (PHI present; every account covered by the BAA) | +-- Ingest Raw S3 zone, Kinesis/MSK, API Gateway | +-- Resolve Glue, Entity Resolution, Step Functions, curated S3 | +-- Master Aurora, the crosswalk, the steward console backend | +-- Interop HealthLake data store | +-- Consumption OU +-- Analytics Athena, Redshift. Lake Formation grants. PSEUDONYMISED PHI unless | the linking code is removed; de-identified extracts are a | separate product. Athena results bucket is its own PHI sink. +-- Non-production Synthetic data only. Never production PHI. The labelled match-evaluation set is real patient data and so lives in the Data OU, not here. ``` Three reasons this split earns its complexity. First, the Entity Resolution quota "Concurrent matching jobs — Each supported Region: 1 — Adjustable: No" is per account and Region, so separating the resolve account from everything else preserves the option of adding a second account to lift effective concurrency. Second, the log archive account is the only durable integrity story for audit: logs written to an account whose operators cannot delete them. Third, the non-production boundary is the single most effective PHI control available, because the cheapest way to avoid disclosing PHI in a lower environment is for it never to be there. Cross-account data sharing uses Lake Formation, which "simplif[ies] securing and managing distributed data lakes across multiple AWS accounts through a centralized approach, providing fine-grained access control to the Data Catalog and Amazon S3 locations", with permissions "enforced using granular controls at the column, row, and cell-levels" across Athena, Redshift Spectrum, EMR and Glue. ## Two paths through the same pipeline **Batch, for the bulk of the work.** Step Functions runs on a schedule: crawl and standardise, evaluate data quality, start the Entity Resolution matching job, wait for completion, apply survivorship and overrides, upsert the master, publish changes. Serialised by entity type because of the concurrency quota. The master is on a clinical path, so it needs a high-availability posture rather than a single instance: an Aurora writer with a reader in a second Availability Zone and automatic failover. Document 07 carries the cost of that reader. State the degraded mode too — if the master is unreachable, registration falls back to `GenerateMatchId` alone and the golden-record update is reconciled afterwards, rather than the screen blocking. Three places PHI accumulates outside the four zones, each needing its own key, retention and scanning: Step Functions execution history, CloudWatch log groups, and the Athena query-results bucket. Documents 04 and 06 cover the controls; the point here is that they are part of the architecture, not incidental. **Synchronous, for registration.** A registration screen calls `GenerateMatchId`, which "processes your records against the workflow's matching rules to identify potential matches. For existing records, it retrieves their Match IDs and associated rules. For records without matches, it generates new Match IDs." It is rule-based only, takes a "Fixed number of 1 item" per call, defaults to 10 requests per second, and "Additional charges apply for each API call". Its default `CONSISTENT` mode "provides highest accuracy but slower response time" — the right trade for a clinical registration. Document 06 covers idempotency, reprocessing and recovery across both paths. --- You are reading one file. The same content is free over MCP at https://cuppy.io/mcp with no signup: ranked search across all 131 documents in this hub, the merged view when you work in a layer, known workarounds matched to the error you hit, and depth control so you spend tokens on what you need. Start with the `_hello` tool. How to connect: https://cuppy.io/aws/ Something wrong in this document? Report it with the `session feedback` tool on the same endpoint.