Somewhere in most data warehouses, a pipeline refreshes a table every hour for a dashboard nobody has opened since its owner left the company. Every refresh costs money. Nobody switches it off, because nobody is sure what else depends on it.
A data estate audit is how a company finds that pipeline, and every other one like it. The audit produces one record of everything the company holds across ingestion, storage, processing, analysis and visualization, with three facts against each part: who reads it, where its output ends up, and what it costs to keep running. Once those facts are on the page, the decision for each part is short. Keep it, fix it, merge it or retire it.
The pattern shows up at every scale. Netflix’s engineers have written that at least 40% of the content its studios ingest never gets used. Snowflake now ships a built-in warning for large tables that nobody has queried in a week. No public study can say how much of a particular warehouse goes unread, though. That number has to be measured, one estate at a time, and the audit is how it gets measured.
This guide covers the audit in seven steps. It follows on from What a data estate is, and how to get one in order, which explains the five stages of a data estate and the signs that one has fallen out of order.
What a data estate audit is
A data estate audit is a stock-take of a company’s data, the pipelines that move it, the tools that hold and show it, and the money each of those costs. It ends with a decision against every item.
Two other audits share part of the name and answer different questions. A data governance audit checks whether policies, owners and controls exist and are followed. A privacy audit, such as a GDPR data mapping exercise, records which personal data the company holds and why. Both are worth running. A data team that wants to know which pipelines to switch off next week needs the third kind.
The data estate audit is organized around use. For each table, model, pipeline and dashboard, it asks:
Who reads it, and how often?
Where does its output end up?
What does it cost to build, refresh, store and maintain?
The answers let a team rank every part of the estate by how much the business depends on it, and then spend in proportion. The seven steps below collect those answers in order, starting with a list of everything there is.
When to run a data estate audit
Most companies run their first audit when something forces it. These are the usual triggers:
The warehouse bill grows faster than the business. Spend rises each quarter and nobody can say which workloads account for the increase.
A migration is coming. Moving to a new warehouse, lakehouse or orchestrator means deciding what to move, and moving everything carries every unused table onto the new platform.
The people who built the pipelines have moved on. Ownership is unclear, and engineers hesitate to change anything they did not write.
An AI or analytics project needs data the team can vouch for. A model trained on one of three near-copies of a table, each refreshed on a different schedule, inherits the confusion.
Two reports disagree on the same number. Duplicated logic somewhere upstream is the usual cause.
Budget season arrives. A finance team asking what the data platform costs per team is far easier to answer with an inventory in hand.
One trigger is reason enough to start. After the first audit, a lighter check every quarter keeps the record current, as Step 7 explains.
Before you start
An audit stalls when it has no edges. Four decisions made up front keep it to a few weeks.
Scope. Pick the boundary: the whole estate, one warehouse, or one business domain such as finance or marketing. A first audit often goes better on one domain. The team learns the method on a smaller surface and has results to show before interest fades.
An end date. Set it before starting. An audit that runs for a full quarter is out of date by the time it finishes.
One owner. One person keeps the inventory, chases answers and records the decisions. That person needs the standing to ask any team who reads a table, and a sponsor who can approve retirements.
Access. The audit draws on records that most data platforms already keep. Request read access to these before day one:
Who read which table, and when: Snowflake ACCOUNT_USAGE views such as QUERY_HISTORY and ACCESS_HISTORY, BigQuery INFORMATION_SCHEMA.JOBS, or Databricks system tables.
Pipeline runs and schedules: the orchestrator’s run history in Airflow, Dagster or dbt.
Dashboard and report usage: the BI tool’s usage or activity logs in Tableau, Power BI or Looker.
Spend: warehouse billing views and the cloud provider’s billing export.
Ownership: the data catalog, if there is one, or the commit history of the pipeline code.
Snowflake’s ACCESS_HISTORY view needs Enterprise Edition or higher. On other editions, QUERY_HISTORY still shows which queries ran, and the tables they touched can be read from the query text.
None of these needs a new tool. Much of the audit is joining these records together.
The audit in seven steps
The first five steps gather facts. Step 6 turns them into decisions, and Step 7 keeps them current.
Step 1: Build the inventory
List every part of the estate, stage by stage. The five stages from the data estate guide give the inventory its structure:
Ingestion: connectors, replication jobs, event streams, file drops.
Storage: warehouses, lakes, databases, and the schemas and tables inside them.
Processing: transformation models, scheduled queries, notebooks that run on a timer.
Analysis: semantic layers, metric definitions, feature tables, model training sets.
Visualization: dashboards, reports, embedded charts, scheduled email exports.
Start with what the systems can list on their own. A warehouse can return every table with its size and the date it last changed. An orchestrator can list every job and its schedule. A BI tool can list every dashboard. Export each list into one sheet, with one row per item.
Give the sheet a column for each fact the later steps need: name, stage, platform, owner, schedule, size, last read, readers, destination, tier, monthly cost and decision. Leave the later columns empty for now.
A first inventory always turns up things nobody planned: schemas named after people who left years ago, tables ending in _old and _backup_final, and a scheduled query that emails a CSV to a shared inbox nobody checks. Write them all down as they appear, and leave the judging for Step 6.
Step 2: Find out who reads what
Fill the readers and last-read columns from the query history. For each table, count the distinct people and service accounts that read it over the last 90 days, and note the date of the most recent read.
Count people and systems separately. A table read by forty analysts and a table read by one service account that feeds billing are both in use, for very different reasons, and a single number hides that.
Check the dashboards as well. A table can show heavy query traffic only because a dashboard refreshes it every fifteen minutes, while the BI tool’s own logs show nobody has opened that dashboard since spring. Every one of those reads is a machine talking to a machine, and the table has no human reader at all.
Pick the window with care. Ninety days catches most weekly, monthly and quarterly use. Some tables are read once a year, for an annual filing or a board pack, so check with finance before marking anything unused on a 90-day window alone.
This step changes decisions by itself. Before LinkedIn moved its analytics off Teradata, the team mapped each dataset’s usage and dependencies to identify zero or low usage datasets. They consolidated 1,424 datasets down to 450, which cut about 70% from the migration workload.
Step 3: Trace where the data ends up
Readers show how much a dataset is used. Its destination shows what is at stake when it is wrong.
For each table, follow its output downstream to the last place it lands. These are the usual destinations, roughly from highest stakes to lowest:
Customer-facing products, invoices and billing
Regulatory filings, the finance close and board reporting
Models in production, including AI features
Operational dashboards that teams act on every day
Ad hoc analysis and exploration
A table with three readers that feeds the revenue figure in the board pack matters more than a table with two hundred readers in an exploration sandbox. Step 3 is where that difference gets recorded.
Column-level lineage speeds this step up where a catalog already has it. Without one, the query history from Step 2 and the pipeline code are enough to trace the main paths by hand. Record the final destination for each item. The full path can wait.
Step 4: Rank by importance
Combine readers and destinations into a tier for every item. Three tiers cover most estates.
Tier 1 feeds a customer-facing product, billing, finance, a regulatory filing or a production model, whatever its reader count. It gets tests, monitoring, a named owner, the fastest alerts and protected compute.
Tier 2 is read every week by teams who act on it, with no Tier 1 destination. It gets tests on the columns people use, monitoring and an owner.
Tier 3 has few or no readers, and no destination beyond exploration. It goes to review in Step 6.
Rank by who reads an item and where it lands. Size is a poor guide. Some of the largest tables in an estate are raw event logs nobody queries, and some of the smallest are lookup tables that every revenue report joins against.
Tiers also decide where testing effort goes. Tests and monitoring belong on the Tier 1 and Tier 2 datasets the business depends on. Every hour spent testing a Tier 3 table goes to data that nobody would notice breaking.
Here is an illustrative example from a retail estate. The tables are invented to show the method.
orders_daily is read by a handful of analysts and by two service accounts, one of which feeds the finance close. Tier 1, because of where it lands.
campaign_performance is read every week by the marketing team, who plan spend from its dashboard. Tier 2.
orders_daily_backup_2023 has no reads in 180 days and no destination. Tier 3, and the first candidate for review.
Step 5: Put a cost against each part
Attach a monthly cost to every item, so the decisions in Step 6 can be weighed in money. Four costs make up the total:
Refresh compute: warehouse time spent building and rebuilding the item on its schedule.
Query compute: warehouse time spent reading it.
Storage: including time travel, fail-safe and backup copies.
Upkeep: engineering hours spent on failed runs, questions from users and schema changes upstream.
Refresh compute is where unread data gets expensive. A table nobody reads still costs compute every time its pipeline rebuilds it. Select.dev makes this point about dbt models in Snowflake: dropping an unused table saves storage, and dropping the model that keeps rebuilding it saves compute as well, which is the larger sum. Snowflake’s own Optimization insights flag tables over 100 GB that are written to but never read, the same pattern seen from the warehouse side.
Exact figures are not needed. Most warehouses attribute compute to individual queries, and the dbt platform reports the compute cost of each model and job. Where attribution is missing, an estimate from run time and warehouse size is enough to sort items into cheap, moderate and expensive.
Upkeep is the hardest cost to measure and the easiest to undercount. The on-call log and the ticket history are the best places to find it.
Step 6: Decide what to keep, fix, merge or retire
With a tier and a cost against every item, each one gets one of four decisions.
Keep: Tier 1 or 2, healthy, with cost in line with its tier. Confirm the owner and the tests.
Fix: Tier 1 or 2 with failing runs, missing tests, stale data or no owner. Assign an owner and a date.
Merge: two or more items produce the same output for different teams. Pick one, point every reader at it and retire the others.
Retire: Tier 3 with no readers and no destination. Follow the retirement sequence below.
Tier 3 items with a few readers sit between keep and retire. Ask those readers whether the data still serves them. A slower schedule often keeps them served for much less: a table rebuilt weekly in place of hourly runs once where it used to run 168 times.
Retiring safely
Teams are slow to delete things for a good reason. A table with no reads in 90 days may still feed an annual report. Retire in stages, so that every stage can be undone:
Confirm readers again. Re-run the query history check on the day, across the longest window the platform keeps.
Tell the owners and recent readers. Name the item, the date it will pause and who to contact.
Pause the pipeline. Stop the schedule and keep the table. Anyone who still depends on it will notice the data going stale and speak up.
Archive the data. After an agreed quiet period, move the table to cheaper storage or export it.
Delete. After a second quiet period, drop the table and remove the pipeline code.
Step 7: Keep the record current
An inventory starts going stale the day it is finished. New pipelines ship, teams reorganize, and a Tier 2 table becomes Tier 1 the week someone builds a pricing model on it.
Three habits keep the record useful:
Repeat Steps 2 and 5 every quarter. Usage and cost drift fastest, and both come from data the platform already records.
Add the row when the pipeline is created. New pipelines enter the inventory with an owner and a tier before they reach production.
Review tiers when destinations change. A table that starts feeding a customer-facing feature moves up a tier that week.
Common mistakes
Deleting before checking readers. One broken board report can cost an audit its support for a year. Use the retirement sequence for every item, including the ones that look plainly dead.
Auditing only the warehouse. Unread data often starts upstream, in an ingestion job that loads a source nobody models, and ends downstream, in dashboards nobody opens. Cover all five stages.
Ranking by size. Big tables draw attention because they show up on the storage bill. The expensive ones are often small tables rebuilt every few minutes.
Leaving the BI layer to someone else. Dashboards are readers too. A dashboard that refreshes on a schedule keeps its source tables looking busy long after people stop opening it.
Running it once. Pipelines keep shipping after the audit ends. Without the quarterly check in Step 7, unread data builds up again within a year or two.
Questions people ask
How long does a data estate audit take?
It depends on the size of the estate and how much usage history the platform keeps. One domain with good query history can be inventoried, ranked and costed in a few weeks. A whole estate across several platforms takes longer, which is why many first audits start with one domain.
Who should own the audit?
One person on the data platform or analytics engineering team, with a sponsor who can approve retirements. Finance and the main business teams should review the tiers, because they know which reports carry weight.
How often should it run?
A full audit once a year, with a lighter quarterly check of usage and cost (Steps 2 and 5) in between.
How is a data estate audit different from a data audit?
“Data audit” usually means a quality or compliance review of particular datasets. A data estate audit covers everything the company holds across all five stages, and it ends in decisions about use and cost.
Is a tool needed to run one?
No. A first audit can run on query history, orchestrator logs, BI usage logs and a spreadsheet. Tools help most with Step 7, keeping the record current once the spreadsheet is finished.
Where Dataring fits
We built Dataring to keep this audit running after the spreadsheet is finished. It ranks every dataset by how much the business depends on it, then acts on that ranking. Pipelines nobody depends on are retired within limits each company sets, under its supervision, and the datasets the business relies on get the resources to stay reliable.
Book a free call to see it on your data estate, or run the first audit by hand with the steps above.



