/

Data

How to reduce data alert fatigue

How to reduce data alert fatigue

Data

Data

Mahesh Chandran

Mahesh Chandran

CEO

CEO

Guide cover: How to reduce data alert fatigue. Route each data alert by how much the business depends on the data.

How many data alerts fired last week, and how many of them changed what someone did? When the second number is much smaller than the first, people learn to ignore the channel.

That gap is data alert fatigue. Checks get added table by table, every failure goes to the same channel, and after a few weeks of alerts that turn out not to matter, people stop reading them. When an alert that matters arrives, it looks the same as the others and goes unread.

This guide shows how to reduce data alert fatigue by deciding which alerts deserve attention based on how much the business depends on the data behind them, then routing, merging, tuning and retiring the rest. It includes a template for auditing current alerts and dbt settings for routing by tier.

The short answer

Data alert fatigue comes from treating every failed check the same way. To reduce it, rank datasets by how much the business depends on them, then route each alert by its dataset’s tier. Tier 1 alerts page a named owner. Tier 2 alerts go to the owning team during working hours. Tier 3 alerts go into a daily digest. Then cut alerts that repeat the same upstream problem, raise thresholds that fire on normal variation, and remove checks on data nobody reads.

Key terms

  • Data alert: a notification sent when a check on data fails, such as a freshness, volume, schema or test failure.

  • Alert fatigue: the point at which people receiving alerts stop reading or acting on them because too many turn out not to matter.

  • Actionable alert: an alert that tells a specific person something they need to do now.

  • Alert routing: sending each alert to a destination and urgency that match the data it concerns.

  • Dataset tier: a rank from 1 to 3 that records how much the business depends on a dataset. Tier 1 is the highest.

The method at a glance

  1. Count every alert from the last 30 days and record whether anyone acted on it.

  2. Give every alerting dataset a tier and a named owner.

  3. Route alerts by tier, so only Tier 1 failures interrupt anyone.

  4. Alert once at the source of a problem, and stop downstream checks repeating it.

  5. Set thresholds that separate a real problem from normal variation.

  6. Remove alerts on data nobody reads, and review the rest every month.

Why does data alert fatigue happen?

Software reliability teams worked through the same problem years ago. Google’s guidance on alerting says that pages should be urgent, important, actionable and real, and that teams should err on the side of removing noisy alerts, because over-monitoring is a harder problem to solve than under-monitoring. Data alerting often breaks those rules in a few predictable ways.

  • Every table gets the same checks. Tests and monitors are added wherever they are easy to add, so a scratch table gets the same alerting as the table behind the invoice run.

  • Every alert goes to the same place. A shared channel collects failures from every pipeline. Nobody owns the channel, so everybody assumes someone else is looking.

  • One problem fires many alerts. A late source table makes every model built from it fail its freshness check. The person on call sees twenty alerts for one cause.

  • Thresholds fire on normal variation. A volume check set to exact counts fails every weekend, every holiday and every month-end. After the third false alarm, people stop opening it.

  • Alerts outlive the data. Checks keep running on tables that nobody reads anymore, so their failures are noise by definition.

The common thread is that alerts are set up table by table, without asking what each table is for. The fix is to ask that question first.

How do you audit your current alerts?

Start with a count. Export every alert from the last 30 days from the tools that send them, such as the orchestrator, the test runner and any monitoring tool, and group them by the check and dataset that raised them. For each group, record a few fields:




The rows above are invented to show the pattern. Three columns do most of the work:

  • acted_on shows whether the alert led to a fix, a rerun or a warning to readers. An alert group with zero actions over a month is a candidate for rerouting, retuning or removal.

  • duplicate_of shows when an alert only repeats a problem already reported upstream.

  • owner shows who should hear about it. A blank owner on a Tier 1 or Tier 2 dataset is a gap to close before changing anything else.

The audit splits alerts into the groups that led to action and the groups that did not. The rest of this guide works through the second list.

How do you route alerts by tier?

The same failed check means different things on different data. A late table behind the invoice run needs someone now. A late table in an exploration sandbox can wait until morning, or until the analyst using it notices. Routing makes that difference visible.

First, give every dataset that raises alerts a tier, based on where its data lands and what happens when it is late or wrong. How to decide which datasets your business depends on sets out a scoring method. Then match each tier to a destination:

  • Tier 1: page the named owner straight away, and hold the downstream runs that would spread the problem. These are the only alerts that should interrupt anyone.

  • Tier 2: send a message to the owning team’s channel during working hours, with the dataset, the check and what it feeds.

  • Tier 3: collect failures into a daily digest. No page and no channel message.

Every failed check routed by dataset tier: Tier 1 pages the named owner and holds downstream runs, Tier 2 messages the owning team during working hours, Tier 3 goes into a daily digest.

Each alert should say what the data feeds, so the person reading it can judge the stakes without looking it up.

Routing in dbt

Teams that run tests in dbt can carry the tier and owner in the project itself. dbt groups give a set of models a named owner with an email address, and the dbt platform can email model owners about failures, including test failures on their models. The severity setting decides whether a failed test stops the run or only warns. A tier can be recorded in each model’s meta config for alerting tools to read.

groups:
  - name: billing
    owner:
      name: Billing data team
      email: billing-data@example.com

models:
  - name: invoice_lines
    config:
      group: billing
      meta:
        tier: 1
    data_tests:
      - dbt_utils.recency:
          arguments:
            datepart: hour
            field: loaded_at
            interval: 6
          config:
            severity: error

  - name: web_events_daily
    config:
      meta:
        tier: 3
    data_tests:
      - dbt_utils.recency:
          arguments:
            datepart: day
            field: event_date
            interval: 2
          config:
            severity

groups:
  - name: billing
    owner:
      name: Billing data team
      email: billing-data@example.com

models:
  - name: invoice_lines
    config:
      group: billing
      meta:
        tier: 1
    data_tests:
      - dbt_utils.recency:
          arguments:
            datepart: hour
            field: loaded_at
            interval: 6
          config:
            severity: error

  - name: web_events_daily
    config:
      meta:
        tier: 3
    data_tests:
      - dbt_utils.recency:
          arguments:
            datepart: day
            field: event_date
            interval: 2
          config:
            severity

groups:
  - name: billing
    owner:
      name: Billing data team
      email: billing-data@example.com

models:
  - name: invoice_lines
    config:
      group: billing
      meta:
        tier: 1
    data_tests:
      - dbt_utils.recency:
          arguments:
            datepart: hour
            field: loaded_at
            interval: 6
          config:
            severity: error

  - name: web_events_daily
    config:
      meta:
        tier: 3
    data_tests:
      - dbt_utils.recency:
          arguments:
            datepart: day
            field: event_date
            interval: 2
          config:
            severity

With this setup, a failure on invoice_lines stops the run and notifies the billing data team, while a failure on web_events_daily only produces a warning that can go into the digest.

How do you stop one problem firing many alerts?

Google’s guidance draws a line between symptoms and causes, and recommends alerting on symptoms and keeping cause-based detail on dashboards. In a data pipeline, the useful version of that idea is to alert once, at the point where a problem enters, and let downstream checks stay quiet when the cause is already known.

  • Check sources first. Put freshness and volume checks on the tables where data arrives. When one of those fails, the alert names the cause.

  • Skip downstream checks when an upstream check fails. Most orchestrators and test runners can stop downstream models from running when an upstream test fails. A model that does not run cannot raise a second alert about the same problem.

  • Group what remains. When several alerts share an upstream cause, send them as one message that lists the affected datasets.

How do you stop alerts firing on normal variation?

People stop reading a volume alert that fires on every weekend dip. Set thresholds from the dataset’s own history, with room for the patterns that are normal for it, such as lower volumes on weekends and holidays or a spike at month-end.

dbt’s error_if and warn_if settings let a test warn on a small number of failing rows and error only above a larger number. That suits checks such as uniqueness on a column where a handful of duplicates is a known, tolerable issue, while hundreds would mean something has broken.

data_tests:
  - unique:
      config:
        severity: error
        warn_if: ">10"
        error_if: ">1000"
data_tests:
  - unique:
      config:
        severity: error
        warn_if: ">10"
        error_if: ">1000"
data_tests:
  - unique:
      config:
        severity: error
        warn_if: ">10"
        error_if: ">1000"

How do you retire alerts nobody needs?

An alert on a table nobody reads can only ever be noise. Before tuning those alerts, check whether the table should still exist. How to find and retire unused data pipelines shows how to find tables that are written but never read, and how to retire them in stages. Retiring the pipeline removes its alerts along with its compute cost.

For the alerts that remain, set a simple rule. An alert group that nobody acted on for two months in a row gets retuned, rerouted to a lower tier or removed.

How do you keep alerts from creeping back?

Alert fatigue returns as new checks get added. A short review each month keeps it in check:

  • Count alerts by tier and compare the number of Tier 1 pages with the number that led to action.

  • Look at who found each data incident. When readers keep finding problems before the alerts do, the checks are on the wrong datasets.

  • Confirm that every Tier 1 and Tier 2 dataset still has a named owner.

  • Require a tier and an owner for every new check before it can send alerts.

Detection speed is what alerts are for. The cost of bad data grows with how long it goes unnoticed, so a smaller number of alerts that people read and act on catches more than a larger number that nobody opens.

Questions people ask

How many data alerts are too many?

There is no fixed number. The useful measure is the share of alerts that lead to action. When most Tier 1 pages lead to a fix, a rerun or a warning to readers, the volume is about right. When most do not, there are too many, whatever the count.

Should we delete tests to reduce alert fatigue?

Route and tune first. Many tests are worth keeping as warnings or digest items even when they should not interrupt anyone. Delete tests on data nobody reads, and tests that duplicate a check already made upstream.

Who should receive data alerts?

The owner of the dataset, by name. For Tier 1 data that feeds billing or finance, the business owner should also hear about problems that affect what they report, so they can hold off acting on the numbers.

Is alert fatigue only a problem for large data teams?

No. A small team with one shared channel can reach the point of ignoring alerts faster, because the same few people receive everything. Tiers and routing help at any size.

What should a good data alert contain?

The dataset, the check that failed, the tier, what the dataset feeds, the owner and a link to where to look first. A reader should be able to judge the stakes from the alert itself.

Where Dataring fits

We built Dataring to rank every dataset by how much the business depends on it, then act on that ranking. Monitoring depth and alert routing follow the ranking, so the closest checks and the fastest alerts go to the data behind billing, finance and customer-facing products, and data nobody depends on stops sending alerts. Pipelines nobody depends on are retired within limits each company sets, under its supervision.

Book a free call to see it on your data estate. For the wider process this guide fits into, read How to audit your data estate in seven steps.

Newsletter

Product news by email. We use your address for nothing else.

© 2026 ShareData Inc.Last updated October 29, 2026