Back to data projects
Sanius Health•Nov 2023 — Present

Clinical + wearable medallion pipeline

Built a Bronze/Silver/Gold pipeline for clinical and wearable data so sensitive analytics could move from raw capture to decision-ready models with governance built in.

Medallion architectureData quality gatesGovernance + compliance controls

Business problem

Healthcare analytics needed a trustworthy path from raw events to governed metrics while meeting compliance expectations for sensitive data and avoiding ad hoc model sprawl.

Platform scope

Sensitive clinical and wearable datasets

Cadence

Continuous event ingestion with governed layer promotions

Consumers

Analytics stakeholders, operations, and compliance-aware reporting

Thinking model

  • Separate reliability concerns by layer: raw capture, cleaned data, and decision-ready models.
  • Attach governance controls where data changes state, not only at final dashboards.
  • Make quality checks part of promotion criteria between layers.

Constraints

  • Sensitive healthcare data required governance and traceability to exist throughout the pipeline, not only in served dashboards.
  • The modeling approach needed to reduce ambiguity for downstream teams without slowing delivery to a crawl.

Architecture

Ingest

Clinical + wearable sources

Storage

Bronze layer

Process

Silver layer

Serve

Gold layer metrics

Ops

RBAC + lineage + audit

Operational guardrails

Promotion rules

Datasets only moved layer-to-layer after passing targeted quality checks.

RBAC + lineage

Access and lineage were attached to transformation boundaries for auditability.

Audit visibility

Change paths stayed inspectable for sensitive data operations and investigations.

Sensitive data controls

Governance logic was built into the system design instead of isolated in reporting tools.

Flow checkpoints

raw eventsClinical + wearable sourcesBronze layercleansed + normalizedBronze layerSilver layerbusiness modelsSilver layerGold layer metricsgovernance hooksBronze layerRBAC + lineage + auditlineage checkpointsSilver layerRBAC + lineage + auditaccess controlsGold layer metricsRBAC + lineage + audit

Design note

The medallion layering reduced downstream ambiguity and created one reusable promotion path for sensitive datasets.

Design note

Governance controls were embedded through transformation boundaries instead of being bolted onto dashboards later.

Delivery

Platform work

  • Implemented Bronze/Silver/Gold lifecycle patterns for healthcare analytics workflows.
  • Integrated RBAC, lineage, and auditability into the pipeline path rather than adding them downstream.
  • Mapped application workflows to governed datasets so operational and analytical views stayed consistent.

Quality controls

  • Layer-specific checks applied before model promotion.
  • Audit-friendly visibility around sensitive dataset changes.

Observability

  • Monitoring centered on layer freshness and service continuity risks.
  • Operational visibility via Azure Monitor and Log Analytics.

Impact

Trust model

Bronze/Silver/Gold became the standard promotion path for sensitive analytics data.

Compliance posture

RBAC, lineage, and audit controls aligned with HIPAA/GDPR-oriented operating requirements.

Analytics readiness

Consumers worked from decision-ready models instead of raw healthcare events.

Tradeoffs

  • Introduced extra transformation stages to improve trust and governability.
  • Accepted additional modeling overhead in exchange for stronger data contracts and clearer audit paths.

Confidentiality note

  • Sensitive healthcare entity mappings are omitted while the implementation approach and control model are retained.

Work with me

Need a governed lakehouse for sensitive data?

I work with teams that need better modeling boundaries, promotion criteria, and compliance-aware data operations.

Review the architecture