/

Data

How to set data freshness SLAs by dataset tier

How to set data freshness SLAs by dataset tier

Data

Data

Mahesh Chandran

Mahesh Chandran

CEO

CEO

Guide cover: How to set data freshness SLAs. Set freshness targets from when the business reads the data.

What time does the finance team open the revenue dashboard, and what time does the data behind it finish loading? The gap between those two times is the freshness target that matters to the people reading it.

A data freshness SLA turns that gap into a number. Without one, “the data is late” means something different to every person who says it, and nobody can tell whether a pipeline that finished at 9:40 was fine or a problem.

This guide shows how to set data freshness SLAs for the datasets the business depends on. It covers how to choose each target from when people read the data, how to write it down so everyone agrees on it, how to measure it in dbt, Snowflake and BigQuery, how to report on it every week, and what to do when targets slip. It includes a worked example, the queries and the mistakes to avoid.

The short answer

Set each freshness target from when the data is read and used, working back from the destination. For each Tier 1 and Tier 2 dataset, record the time by which it must be updated and how old its newest data may be at that time. Measure both with freshness checks on the dataset and on its sources, record whether each target was met every day, and report the share of days each target was met. Fix misses at the slowest upstream step. Tier 3 data does not need a target.

Key terms

  • Freshness: how old the newest data in a dataset is at a given moment.

  • Ready-by time: the time by which a dataset must be updated for the people or systems that read it.

  • Maximum age: how old the newest data in a dataset may be at its ready-by time.

  • Freshness SLA: an agreed target for freshness, such as data no older than 6 hours by 8 a.m. Eastern. Google’s SRE guidance defines an SLO as a target value for a measured service level, and an SLA as a contract with consequences for meeting or missing it. Most data teams say SLA for both.

  • Hit rate: the share of days, or runs, on which a freshness target was met.

The method at a glance

  1. Find when each Tier 1 and Tier 2 dataset is read, from query history and the readers themselves.

  2. Agree with the business owner how old the data may be when it is read.

  3. Work back through the lineage to the time each source must arrive.

  4. Write each target down as a ready-by time and a maximum age, in a named time zone.

  5. Add freshness checks to the dataset and its sources, with warning and error thresholds.

  6. Record whether each target was met every day, and report the hit rate every week.

  7. Fix misses at the slowest upstream step, starting with Tier 1.

Why does freshness need a number?

Without a target, a late table is a matter of opinion. The pipeline team sees a job that finished, and the finance analyst sees a dashboard that still shows yesterday. Both are right, and nobody can decide whether to page someone, rerun a job or wait.

A target settles that argument before it starts. It also gives the data team something to plan against. A pipeline that meets its target every day can be left alone, and one that misses it twice a week needs work, whatever its run time looks like on its own.

Freshness is one of several ways data goes wrong. A table can arrive on time with half its rows missing, or with every row present and a column calculated wrongly. Freshness targets catch the first kind of failure early and cheaply, which is why they are a good place to start. Row counts and tests on values catch the others, and How to reduce data alert fatigue covers how to keep all of those checks from flooding the team.

How do you choose the target?

Start from the reader. Pipeline schedules are often set for convenience, and a target taken from the schedule only describes what already happens.

  • Find when the data is read. Query history shows when dashboards refresh and when people query a table. Ask the business owner when decisions are made from it, such as a daily trading meeting or a month-end close.

  • Find how old the data may be. A daily revenue report may only need yesterday’s complete data. A fraud model may need data no older than 15 minutes. The business owner decides this, because they know what an older number costs them.

  • Work back through the lineage. Each step between the source and the destination takes time. Add up the run times along the slowest path to find when each source must arrive for the destination to be ready.

These questions help in the conversation with the business owner:

  • When do you or your systems first look at this data each day?

  • If it were an hour late, what would you do differently, and what would it cost?

  • Does yesterday’s data need to be complete, or is a partial day useful?

  • Are there days when it matters more, such as month-end, a launch or a board meeting?

  • Who should hear about it when the data is late, and how?

The result for each dataset is two numbers: a ready-by time and a maximum age. For example, invoice_lines must be updated by 6 a.m. Eastern with data no older than 6 hours.

A worked example

Here is how the numbers come together for one table. The times are invented to show the method.

The billing team runs invoices at 6:30 a.m. Eastern and needs invoice_lines updated by 6:00, with all of the previous day’s transactions. The table is built from two raw sources. Raw invoices land from the billing platform at about 2:30. Raw payments land from the payment provider at about 3:15. Over the last month, the staging models took up to 20 minutes, and the final model took up to 40 minutes.

  • Slowest path: raw payments, then staging, then invoice_lines.

  • Time needed after the last source arrives: up to 60 minutes.

  • Buffer for a retry: 45 minutes, agreed with the technical owner.

  • Latest acceptable source arrival: 4:15 a.m. Eastern.

The freshness target for invoice_lines becomes ready by 6:00 a.m. Eastern, with data no older than 6 hours. The two sources get their own targets of arrival by 4:15. If raw payments arrives at 4:40, the team knows before 5 a.m. that invoice_lines is at risk, and has time to act or warn the billing team.

What about data that updates during the day?

Some datasets are read many times a day, such as an operations dashboard or a stock level feed. A single ready-by time does not describe what those readers need. Set a maximum age that applies during the hours the data is used, such as no older than 30 minutes between 7 a.m. and 7 p.m. Eastern, and check it on a schedule that matches.

Report these targets as the share of checks that passed during the agreed hours, in place of a daily yes or no. A dataset checked every 15 minutes over a 12-hour day gives 48 results a day, which shows whether misses cluster at particular times, such as the morning load or a vendor’s batch at noon.

What targets suit each tier?

How to decide which datasets your business depends on ranks datasets into tiers by where their data lands. The tier sets how strict the target should be and what happens when it is missed.

Freshness targets by tier: Tier 1 has a ready-by time with paging on a miss, Tier 2 has a daily target with alerts in working hours, Tier 3 has no target.
  • Tier 1: a ready-by time set with the business owner, an error threshold that pages the technical owner, and a hit rate reported every week.

  • Tier 2: a daily target, with misses sent to the owning team’s channel during working hours.

  • Tier 3: no target. The person using the data notices if it matters.

How do you write the target down?

Record the target next to the code that builds the dataset, so it changes with the code and tools can read it. A meta block on the dbt model works well:

models:
  - name: invoice_lines
    config:
      meta:
        tier: 1
        ready_by: "06:00"
        ready_by_timezone: America/New_York
        max_age_hours: 6
        business_owner: head-of-billing@example.com
        technical_owner

models:
  - name: invoice_lines
    config:
      meta:
        tier: 1
        ready_by: "06:00"
        ready_by_timezone: America/New_York
        max_age_hours: 6
        business_owner: head-of-billing@example.com
        technical_owner

models:
  - name: invoice_lines
    config:
      meta:
        tier: 1
        ready_by: "06:00"
        ready_by_timezone: America/New_York
        max_age_hours: 6
        business_owner: head-of-billing@example.com
        technical_owner

The agreement behind it can be short. For each Tier 1 dataset, write down:

  • The dataset and what it feeds.

  • The ready-by time, with its time zone, and the maximum age.

  • Which days the target applies, including weekends and holidays.

  • Who is told when it is at risk and when it is missed, and how.

  • The hit rate the business owner expects, such as met on every working day.

  • The date of the next review.

Both owners agree the target. How to assign data ownership covers who those owners are and how to record them.

How do you measure freshness?

In dbt

dbt can check freshness on sources and models, warning or erroring when the newest data is older than a set age. In recent versions, freshness and loaded_at_field sit under config. On tables and incremental models, loaded_at_field is optional, and dbt can use warehouse metadata when it is not set. Views need a loaded_at_field or a loaded_at_query. The dbt freshness command checks every source and model with freshness configured.

sources:
  - name: billing
    config:
      loaded_at_field: _loaded_at
      freshness:
        warn_after: {count: 4, period: hour}
        error_after: {count: 6, period: hour}
    tables:
      - name: raw_invoices
      - name

sources:
  - name: billing
    config:
      loaded_at_field: _loaded_at
      freshness:
        warn_after: {count: 4, period: hour}
        error_after: {count: 6, period: hour}
    tables:
      - name: raw_invoices
      - name

sources:
  - name: billing
    config:
      loaded_at_field: _loaded_at
      freshness:
        warn_after: {count: 4, period: hour}
        error_after: {count: 6, period: hour}
    tables:
      - name: raw_invoices
      - name

Set the thresholds from the target, with the warning early enough to act before the error. Check the sources as well as the final table. A source that arrives late explains the miss before the final table is due, which gives the owner time to act. On BigQuery, dbt’s metadata-based checks are unreliable for wildcard tables such as events_*, so set a loaded_at_field for those.

In Snowflake

The most reliable measure is a load timestamp column written by the pipeline, compared with the current time:

SELECT
  'invoice_lines' AS dataset,
  MAX(_loaded_at) AS newest_load,
  DATEDIFF('minute', MAX(_loaded_at), CURRENT_TIMESTAMP()) AS minutes_old
FROM

SELECT
  'invoice_lines' AS dataset,
  MAX(_loaded_at) AS newest_load,
  DATEDIFF('minute', MAX(_loaded_at), CURRENT_TIMESTAMP()) AS minutes_old
FROM

SELECT
  'invoice_lines' AS dataset,
  MAX(_loaded_at) AS newest_load,
  DATEDIFF('minute', MAX(_loaded_at), CURRENT_TIMESTAMP()) AS minutes_old
FROM

The LAST_ALTERED column in Snowflake’s TABLES view is tempting, but it changes with DDL, with DML even when no rows are affected, and with background maintenance. A table can look recently updated when no new data has arrived, so use it only as a rough signal.

In BigQuery

A load timestamp column works the same way in BigQuery. For partitioned tables, the PARTITIONS view records when data was last written to each partition and how many rows it holds, which also shows whether the latest partition arrived with a normal number of rows:

SELECT
  table_name,
  partition_id,
  last_modified_time,
  total_rows
FROM analytics.INFORMATION_SCHEMA.PARTITIONS
WHERE table_name = 'invoice_lines'
ORDER BY partition_id DESC
LIMIT 7

SELECT
  table_name,
  partition_id,
  last_modified_time,
  total_rows
FROM analytics.INFORMATION_SCHEMA.PARTITIONS
WHERE table_name = 'invoice_lines'
ORDER BY partition_id DESC
LIMIT 7

SELECT
  table_name,
  partition_id,
  last_modified_time,
  total_rows
FROM analytics.INFORMATION_SCHEMA.PARTITIONS
WHERE table_name = 'invoice_lines'
ORDER BY partition_id DESC
LIMIT 7

A recent last_modified_time shows that something was written. It does not prove the newest data is recent, because a reload of old data also updates it. Compare it with the load timestamp column when the two disagree.

How do you handle time zones, weekends and calendars?

Freshness targets fail in surprising ways around the calendar.

  • Write targets in a named time zone. A target of 6 a.m. Eastern stays correct when clocks change. A target written in UTC shifts by an hour against the readers’ day twice a year.

  • Say which days count. Many Tier 2 datasets are not read on weekends. Leaving weekends out of the target stops weekend misses from paging anyone, and keeps the hit rate honest.

  • Plan for heavy days. Month-end, quarter-end and Mondays after weekend loads often take longer. Check the last few of each against the target, and agree an extra buffer or an earlier start for those days.

  • Know your vendors’ calendars. A vendor that does not send files on its own public holidays will cause misses the pipeline team cannot fix. Record those dates in the agreement.

How do you report on freshness targets?

Record whether each target was met each day, and report the hit rate every week. The record can be as simple as one row per dataset per day:




The rows are invented to show the format. With the log stored as a table, the weekly report is one query:

SELECT
  dataset,
  tier,
  COUNT(*) AS days,
  SUM(CASE WHEN met THEN 1 ELSE 0 END) AS days_met,
  ROUND(100.0 * SUM(CASE WHEN met THEN 1 ELSE 0 END) / COUNT(*), 1) AS hit_rate_pct
FROM governance.freshness_log
WHERE log_date >= CURRENT_DATE - 28
GROUP BY dataset, tier
ORDER BY tier,

SELECT
  dataset,
  tier,
  COUNT(*) AS days,
  SUM(CASE WHEN met THEN 1 ELSE 0 END) AS days_met,
  ROUND(100.0 * SUM(CASE WHEN met THEN 1 ELSE 0 END) / COUNT(*), 1) AS hit_rate_pct
FROM governance.freshness_log
WHERE log_date >= CURRENT_DATE - 28
GROUP BY dataset, tier
ORDER BY tier,

SELECT
  dataset,
  tier,
  COUNT(*) AS days,
  SUM(CASE WHEN met THEN 1 ELSE 0 END) AS days_met,
  ROUND(100.0 * SUM(CASE WHEN met THEN 1 ELSE 0 END) / COUNT(*), 1) AS hit_rate_pct
FROM governance.freshness_log
WHERE log_date >= CURRENT_DATE - 28
GROUP BY dataset, tier
ORDER BY tier,

Over a few weeks, the report shows which targets are met reliably and which miss on particular days, such as Mondays after weekend loads or the first of the month. Share it with the business owners of Tier 1 datasets. A hit rate they can see is the clearest evidence that the data team is meeting its commitments.

Who should hear about a miss, and when?

Alerts should reach the person who can act, early enough to matter.

  • Warning threshold reached: a message to the technical owner, so they can look before the target is missed.

  • Tier 1 error threshold reached: a page to the technical owner, and a short note to the business owner saying the data is late and when it is expected.

  • Tier 2 miss: a message to the owning team’s channel during working hours.

  • Readers: a status note on the dashboard or report, so people do not make decisions from stale numbers without knowing.

Telling readers is easy to forget, and it matters most. A late number that everyone knows is late is far less dangerous than one that looks current.

What should a freshness alert say?

A good alert lets the person who receives it act without opening another tool first. Include:

  • The dataset, its tier and what it feeds.

  • The target, the current age of the data and how far it is from the target.

  • Which upstream sources or steps are late, if the checks show it.

  • The technical owner, the business owner and the runbook link.

  • When it last met its target.

An alert that only says a check failed sends the owner hunting for context while the clock runs. One that names the late source and the business owner lets them fix the problem or warn the right person in minutes.

What do you do when a target keeps slipping?

Look for the slowest step on the path to the dataset. The fix is usually one of these:

  • A source arrives late. Agree an arrival time with the team or vendor that sends it, and check it at the source.

  • The job waits behind other work. Run Tier 1 pipelines first and give them their own compute. How to prioritize data pipelines covers how.

  • A step takes longer as data grows. Make the slow model incremental, or move it to a larger warehouse during the critical window.

  • The target is wrong. If nobody reads the data until later in the day, move the ready-by time and free the early slot for data that is read early.

How do you roll freshness targets out?

Setting targets for every dataset at once is hard to finish. A phased rollout over about a month is easier to complete:

  1. Week 1: list the Tier 1 datasets, find when each is read from query history, and add freshness checks to them and their sources with loose thresholds, so you can see current behavior without paging anyone.

  2. Week 2: meet the business owner of each Tier 1 dataset, agree the ready-by time and maximum age, and write the agreement down.

  3. Week 3: set the warning and error thresholds from the agreed targets, turn on alerts to the technical owners, and start the daily log.

  4. Week 4: send the first hit-rate report to the business owners, pick the dataset with the worst hit rate and fix its slowest step.

Add Tier 2 datasets once Tier 1 targets are being met. By then the checks, the log and the report already exist, so each new dataset takes minutes to add.

What mistakes should you avoid?

  • Copying the schedule into the target. A target taken from when the job runs today cannot show whether today is good enough.

  • Setting targets for every table. Tier 3 targets create alerts nobody acts on. Keep targets for the data the business depends on.

  • Checking only the final table. By the time the final table is late, it is too late to act. Check the sources too.

  • Trusting metadata timestamps alone. Last-modified times change for reasons that have nothing to do with new data. Use a load timestamp where the data matters.

  • Never revisiting targets. Readers change their routines. Review Tier 1 targets every quarter with the business owners.

Questions people ask

Is it a data freshness SLA or an SLO?

Strictly, an SLA is an agreement with consequences, and an internal target is an SLO. Most data teams say SLA for both. What matters is that the target is written down and agreed with the business owner.

How strict should the first targets be?

Set them from what the readers need, then check them against the last month of load times. If a target would have been missed half the time, fix the pipeline or agree a looser target before announcing it.

Do streaming datasets need freshness targets?

Yes, with a maximum age in minutes or seconds in place of a ready-by time. The same rule applies. Set it from how fresh the readers need the data to be.

Who should agree freshness targets?

The business owner agrees the target and the technical owner agrees that it can be met.

Should freshness targets apply on weekends?

Only where someone reads the data on weekends. Systems such as billing or fraud detection may run every day, so their targets do too. Most internal reports can leave weekends out.

What if a vendor sends data late?

Give the source its own arrival target, check it, and share the hit rate with whoever manages the vendor. A record of late deliveries is what makes the conversation with the vendor productive.

Does a freshness check prove the data is correct?

No. It proves that new data arrived in time. Pair freshness checks on Tier 1 datasets with a row count compared against a normal day and tests on the values that matter most, such as totals that should reconcile with the source system.

How do you measure freshness for a dashboard?

Measure the table the dashboard reads, and check when the dashboard itself last refreshed. A dashboard that caches results can show old numbers even when the table is fresh, so its refresh schedule needs to fall after the table’s ready-by time. Put the time the data was last updated on the dashboard, so readers can see it.

Where Dataring fits

We built Dataring to rank every dataset by how much the business depends on it, then act on that ranking. The datasets behind billing, finance and customer-facing products get the resources and the run order they need to stay fresh, and pipelines nobody depends on are retired within limits each company sets, under its supervision.

Book a free call to see it on your data estate. For the wider process this guide fits into, read How to audit your data estate in seven steps.

Newsletter

Product news by email. We use your address for nothing else.

© 2026 ShareData Inc.Last updated October 29, 2026