Medallion Architecture on Databricks: A Data Architecture Guide

Medallion architecture is a simple way to organize a lakehouse into three data layers—bronze, silver, and gold—so teams can iterate safely and ship analytics. Bronze stores ingested, minimally processed raw data. The silver layer standardizes and cleans it. The gold layer publishes business-ready marts for dashboards and machine learning. Databricks popularized this data design pattern, but you can run it on any modern platform. Below you’ll see what belongs in each layer, how to model it with dbt and SQL, and answers to common pitfalls like, “Are silver tables production-ready?” and “Where are processing boundaries in a data lake?”

What is medallion architecture?

Medallion architecture (sometimes called the medallion data architecture) is a layered approach to organizing a lakehouse:

• Bronze: Ingested, minimally transformed records kept close to source data and original data formats.
• Silver: Standardized, cleaned, and conformed tables with consistent schema and business-ready keys.
• Gold: Curated aggregates and dimensional models for BI, self-serve analytics, and data scientists.

Databricks brought this approach to the forefront for lakehouse architecture, but it’s not vendor-locked. Treat it as a lightweight data architecture convention that makes your transformation steps explicit and auditable.

Bronze, Silver, Gold at a glance

Why teams adopt the medallion architecture

• Clarity of processing boundaries: Each layer marks a checkpoint in the pipeline. You always know what was done and where.
• Governance and rollback: You can rebuild silver from bronze, and gold from silver, without re-pulling from a fragile data source.
• Speed to value: Ship bronze quickly, iterate silver, and publish gold without blocking the whole pipeline.
• Data quality: Isolate validation and tests at the right step to improve data quality without hiding issues.
• Portability: Works on Databricks, open lakehouse, or traditional data warehouses.

Is medallion architecture ELT or ETL?

Both can work. In cloud data engineering, you’ll see mostly ELT: load to bronze, then transform to silver and gold in-platform. If you already do heavy upstream transformations (e.g., flattening nested JSON), that’s closer to ETL, but you still land the results into bronze and proceed through the layers.

Building a medallion architecture step by step

Below is a practical layout that runs well on a modern data platform like Databricks or similar lake engines, with dbt orchestrating the transformation graph and your scheduler of choice handling the pipeline runs.

1) Sources and bronze ingestion

Bronze stores minimally processed records. Keep original columns, minimal typing, and add a few control fields (load_dt, _ingest_file). This preserves raw data fidelity while making it queryable. It’s common to ingest multiple data formats (CSV, JSON, Parquet) and both semi-structured data and structured data.

# dbt_project.yml (excerpt)
# Use schemas (namespaces) per layer
models:
  project_name:
    bronze:
      +schema: bronze
    silver:
      +schema: silver
    gold:
      +schema: gold

# models/sources.yml
version: 2
sources:
  - name: app
    schema: raw
    tables:
      - name: orders_files
      - name: customers_files

-- models/bronze/br_orders.sql
-- Minimal parsing and type coercion, append-only
{{ config(materialized='incremental', unique_key='order_id') }}

with src as (
  select
    cast(value:order_id as string)        as order_id,
    cast(value:customer_id as string)     as customer_id,
    cast(value:order_ts as timestamp)     as order_ts,
    cast(value:status as string)          as status,
    _metadata.file_path                   as _ingest_file,
    current_timestamp()                   as _load_ts
  from {{ source('app', 'orders_files') }}
)

select * from src
{% if is_incremental() %}
  where order_ts > (select coalesce(max(order_ts), '1900-01-01') from {{ this }})
{% endif %}

If you receive CDC feeds, land them first. Many teams use change data capture logs in bronze, then upsert in silver.

2) Silver: conformance and cleaning

The silver layer standardizes identifiers, fixes types, deduplicates, and resolves keys. This is where you manage SCD logic and data transformation that makes data reliable. For a deep dive on SCD patterns, see choosing SCD Type 1 vs Type 2 vs Type 3 and the dbt snapshots playbook.

-- models/silver/sl_orders.sql
{{ config(materialized='incremental', unique_key='order_id') }}

with base as (
  select
    order_id,
    customer_id,
    date_trunc('day', order_ts) as order_date,
    status,
    _ingest_file,
    _load_ts
  from {{ ref('br_orders') }}
),

-- Example of CDC upsert logic in silver
ranked as (
  select
    *,
    row_number() over (partition by order_id order by _load_ts desc) as rn
  from base
)

select
  order_id,
  customer_id,
  order_date,
  status,
  _load_ts
from ranked
where rn = 1
{% if is_incremental() %}
  qualify _load_ts > (select coalesce(max(_load_ts), '1900-01-01') from {{ this }})
{% endif %}

Silver tables should be testable. Add basic constraints so you can improve data quality over time without surprises.

# models/silver/_sl_orders.yml
version: 2
models:
  - name: sl_orders
    tests:
      - not_null:
          column_name: order_id
      - unique:
          column_name: order_id

3) Gold: business-ready outputs

The gold layer publishes curated aggregates and dimensional models for BI and ML. Think revenue by week, cohort retention, and dimensional tables for a data model your stakeholders trust. When dimensional modeling choices matter, skim our summary and then use the guide on star schema design. We also map gold models to data marts so downstream contracts stay stable.

-- models/gold/g_orders_daily_revenue.sql
{{ config(materialized='table') }}

select
  order_date,
  sum(case when status = 'completed' then 1 else 0 end) as orders_completed,
  sum(case when status = 'completed' then amount else 0 end) as revenue
from (
  select o.order_date, o.status, p.amount
  from {{ ref('sl_orders') }} o
  join {{ ref('sl_payments') }} p using (order_id)
)
where order_date >= dateadd('day', -90, current_date)
group by 1

Whether gold is materialized tables, incremental tables, or views depends on SLAs and cost. Views are common at gold when latency is low and compute is cheap; otherwise, build tables. The gold layer often maps 1:1 to published data products. If you run on a lake, you’re literally creating the gold layer of the lakehouse.

Naming, schemas, and folder layout

Use obvious prefixes and schema separation:

• bronze.br_<object>
• silver.sl_<object>
• gold.g_<domain>_<subject>

In dbt, stage models (bronze/silver) are separate from marts (gold). For a full walkthrough of the project layout, see dbt project structure: staging, intermediate, and marts.

An end-to-end example: 40M-order retail dataset

Scenario: your orders table has 40M rows and grows by ~200k/day. You ingest event logs to bronze, standardize in silver, and publish daily and weekly rollups in gold.

1. Bronze: Append files hourly. Keep event payloads for audit. This is your repeatable data ingestion step.
2. Silver: Upsert latest order status per order_id. Enforce keys, type casts, and dedupe. Add indexes/partitioning by order_date.
3. Gold: Materialize daily revenue and order counts, plus a dim_customer and fct_orders for dashboards.

-- models/silver/sl_orders_incremental.sql
{{ config(materialized='incremental', unique_key='order_id', on_schema_change='sync_all_columns') }}

with newest as (
  select * from {{ ref('br_orders') }}
),

ranked as (
  select *, row_number() over (partition by order_id order by _load_ts desc) rn from newest
)

select * from ranked where rn = 1

{% if is_incremental() %}
  -- MERGE pattern for warehouses/lake engines that support it
  {{
    dbt_utils.merge(
      target=ref('sl_orders_incremental'),
      source=ref('sl_orders_incremental__dbt_tmp'),
      unique_key='order_id'
    )
  }}
{% endif %}

For aggregation performance, limit gold queries to the last N days and pre-compute rolling windows. If you’re on Databricks, consider Z-Ordering or similar clustering; on other engines, use partitions and statistics. If costs matter, we summarized tactics in BigQuery cost optimization techniques even though the ideas generalize.

Where medallion architecture fits in modern stacks

Medallion architecture thrives in a lakehouse or data lake with compute close to storage. Databricks popularized it, but nothing stops you from implementing it in Snowflake, BigQuery, or relational engines. Treat bronze/silver/gold as logical zones of your data layer, not a hard dependency on a vendor.

Governance, orchestration, and SLAs

Each layer is a contract:

• Bronze: Contract is “we captured what the source sent.”
• Silver: Contract is “keys and types are correct; duplicates removed.”
• Gold: Contract is “metrics definitions are stable.”

Use your scheduler (Airflow, Prefect, etc.) to orchestrate the pipeline across layers. If you’re deciding on a scheduler, compare options in Airflow vs Prefect. In dbt, layer boundaries are just model dependencies. Add freshness and tests to enforce SLAs.

Databricks specifics (brief)

On Databricks, Delta tables plus Auto Loader or Delta Live Tables fit the bronze-to-gold flow well. Check the official docs for Delta Live Tables if you want managed DAGs and quality rules baked into the pipeline. Official docs

Common questions and concise answers

Is medallion architecture ELT or ETL?

Usually ELT on cloud and lake engines: land to bronze, then transform to silver and gold. If upstream tools reshape data first, that’s ETL feeding bronze, followed by more ELT internally.

What are the cons of medallion architecture?

• Extra storage and compute: Multiple copies across layers, especially when tables are materialized at each step.
• Latency stack-up: Each hop adds delay; streaming mitigates this.
• Over-abstraction: Too many micro-layers between bronze and gold slow teams. Keep it to three unless there’s a clear need.
• Contract drift: If no one owns layer contracts, assumptions change and break downstream marts.

Does Snowflake use medallion architecture?

Snowflake doesn’t enforce it, but you can implement the pattern there. The same is true for other data warehouses. The approach is platform-agnostic.

Who came up with medallion architecture?

Databricks popularized the pattern for lakehouses and documented the bronze/silver/gold approach widely. Many teams adopted similar layering independently, but Databricks made the naming mainstream.

And what you are saying about views is just a gold layer no?

Views are often used in the gold layer to define business metrics with low-latency recompute. But gold can be a mix: some views, some materialized tables for heavy joins or strict SLAs. The key is a stable interface for consumers.

But then as all these transformations happen in the data lake, their results being saved as temporary files, where are the boundaries of processing?

Boundaries are your tables and models, not temp files. Each materialized table (or managed streaming sink) is a checkpoint. In dbt, the boundary is a model dependency. In Databricks Delta Live Tables, it’s a declared table/view in the flow. Persisted artifacts (bronze->silver->gold) form the durable edges of the process.

How does the Medallion Architecture differ from the Semantic Layer?

The medallion architecture structures data processing into physical/logical zones. A Semantic Layer defines business metrics and dimensions on top (think “revenue,” “active_users”) and can point at gold or even silver. Medallion is about architecture and processing flow; Semantic Layer is about shared business meaning.

However, the introduction of data lakes and ELT processes also introduced a problem: How should you organize the data in data lakes, and design ELT processes accordingly?

That’s the point of medallion architecture: use bronze for capture, silver for standardization, and gold for publication. It provides an opinionated way to organize data, design the ELT steps, and avoid a swamp of ad-hoc tables.

Is data in the Silver tier production-ready?

Often yes for internal consumers and downstream gold builds. Silver should be clean enough to join reliably and to backfill gold. For executive dashboards, publish curated gold with governance, docs, and SLAs.

Medallion Architecture — anyone moving away from it?

Teams don’t usually abandon it; they adapt it. Some collapse silver+gold for simpler domains, or adopt domain-oriented marts (data mesh) while keeping bronze and silver. Others move heavy business logic into a Semantic Layer on top of gold. The pattern stays, the implementation varies.

From bronze to insights: practical guidance

Bronze layer: do’s and don’ts

• Do keep fidelity to source data and original columns where possible.
• Do add minimal metadata for lineage and auditing.
• Don’t embed business logic; bronze is not the place to calculate KPIs.
• Do store bronze data in append-only fashion; keep late-arriving records.

Silver layer: standardize intentionally

• Conform IDs, timestamps, currencies, and booleans.
• Handle nullability and type mismatches explicitly. This is where you ensure clean data.
• Use SCD and snapshots strategically; see our dbt snapshots guide.

Gold layer: publish and protect

• Dimensional models and aggregates should have stable contracts. Link to our warehouse schema design guide for modeling choices.
• Document metrics and add freshness checks.
• Materialize when performance matters; otherwise, use views and cache.

Views vs tables in gold

Views are great to keep logic centralized and up-to-date. Tables are better when recomputation is expensive or SLAs are strict. Many teams mix both: a gold KPI view on top of a materialized fact table. Either way, data from the bronze layer should never feed dashboards directly; route through silver at minimum.

Quality and testing across layers

Test minimal assumptions in bronze (row count, basic not-null on keys if present). Strengthen tests in silver (keys, uniqueness, ref integrity). Gold gets semantic checks—do aggregates and ratios make sense? This layered approach helps you organize data validation work and steadily improve data quality.

# models/gold/_g_orders_daily_revenue.yml
version: 2
models:
  - name: g_orders_daily_revenue
    description: Daily revenue and order counts
    tests:
      - not_null:
          column_name: order_date

Performance and cost notes

• Partition large silver and gold tables by natural filters (e.g., order_date) to help the query planner.
• Only scan what you need in gold (e.g., last 90 days), and pre-aggregate.
• Use incremental models for wide tables and heavy joins. This reduces the pipeline footprint.

How medallion architecture supports different consumers

• BI and dashboards: Gold models expose consistent KPIs.
• Ad-hoc analysis: Power users can explore silver for flexibility.
• Data scientists: Feature tables can live in gold or read from silver to feed machine learning workflows.

Schema evolution and change management

Expect upstream changes. Keep bronze flexible (allow new columns), then explicitly adopt or reject them in silver. For gold, communicate changes via versioned models or release notes. This keeps the pipeline resilient as producers evolve.

Organizing your medallion layers alongside marts

Use medallion zones for processing flow and data marts for domain boundaries. It’s common to align gold to a domain-oriented set of marts (e.g., sales, marketing). For a project layout that cleanly separates staging from marts, see staging, intermediate, and marts in dbt and visit our Architecture topic hub for more patterns.

When medallion architecture is not enough

• Cross-domain governance needs a Semantic Layer to centralize metric definitions across multiple gold models.
• Real-time SLAs may push you to streaming silver and gold with stricter checkpointing and watermarks.
• Domain ownership may require a mesh approach where domains own their gold and parts of silver, while platform teams maintain bronze.

FAQ: Additional practical points

How many times should I say “architecture” in docs and naming?

Keep names practical (br_, sl_, g_). Reserve the word architecture for docs and diagrams. The code should be obvious without over-labeling.

Do I need all three layers for small teams?

Yes, but keep it light. A single silver model can feed a couple of gold views. Avoid creating extra hops unless they earn their keep.

What about CDC and late data?

Land CDC into bronze as-is; resolve upserts and late-arriving events in silver. Gold should never be where you reconcile events. This pattern keeps late records from polluting metrics and still allows rebuilds.

Cheat sheet: What belongs where

Glossary tie-ins (keep expectations aligned)

• Bronze: capture zone; cheap to write, tolerant of messy inputs.
• Silver layer: transformation and standardization zone; where you fix types and keys.
• Gold layer: publish zone; optimized for consumers and SLAs.

Putting it all together

Think of medallion architecture as a compact, battle-tested way to organize data processing in a lake or warehouse. Bronze protects ingestion and lineage. Silver makes data trustworthy. Gold makes it useful. With simple conventions, a lean set of dbt models, and a reliable schedule, you can move from source to insight consistently—and rebuild any dataset when requirements change.

If you’re building a medallion architecture now, start small: one domain, three layers, a handful of models. Prove freshness and correctness. Then scale domains and add marts as needs grow.

Related resources on this site:

• dbt Project Structure: Staging, Intermediate, and Marts Done Right
• Data Warehouse Schema Design: Star Schema to Galaxy
• Slowly Changing Dimension: SCD Type 1, 2, 3—What to Use
• dbt Snapshots: dbt snapshot Playbook
• Apache Airflow vs Prefect: Which Scheduler?

Final note on terminology: this data design pattern fits equally well if you treat your lake as the system of record and your gold as published data sets for consumption. Whether you run it on Databricks, a data lake with open formats, or classic engines, the outcomes are the same: clearer processing flow, better contracts, and faster iteration.

Want to practice the concepts with real SQL and dbt? Try the free graded exercises at /practice.