Which pipeline in the warehouse costs the most to run, and when did a person last read the table it builds? Most data teams can answer the first half from the bill. The second half usually takes a week of asking around.
Unused data pipelines keep running because nothing in the stack flags them. The job succeeds, the table refreshes, and the monthly invoice has no line for data nobody read. The only way to see them is to put the write history next to the read history and look for the gap.
This guide shows how to find unused data pipelines with the query history your warehouse already keeps, check what still depends on them, and retire them without breaking a report. It includes queries for Snowflake, BigQuery and Databricks. It takes a closer look at one part of a data estate audit: the point where tables with no readers become decisions.
Why unused pipelines survive
A pipeline is easy to start and hard to stop. Starting one takes a ticket and an afternoon. Stopping one takes proof that nobody needs it, and that proof is scattered across query logs, BI tools and the memories of people who may have left.
Pipelines outlive their purpose in a few predictable ways:
The question was a one-off. A launch analysis, a board request, an investigation into one bad week. The dashboard was read for a month, and the pipeline behind it still runs.
The consumer moved. Marketing switched BI tools, a model was retrained on a different feature table, or a report was rebuilt on a new source. The old path kept refreshing.
A copy replaced it. Two teams built the same revenue table with slightly different logic. One became the standard and the other was never turned off.
The owner left. Nobody else knows whether it is safe to touch, so nobody touches it.
The cost is easy to underestimate because most of it sits in compute. Select.dev points out that for tables rebuilt by dbt, dropping the model saves the compute spent refreshing it on top of the storage. A table rebuilt every hour runs about 720 times a month whether anyone reads it or not. Each failure in that pipeline also pages an engineer, adds to the alerts people have learned to ignore, and takes a slot in the queue ahead of jobs the business depends on.
Step 1: Find tables that are written but never read
A pipeline becomes a candidate when the tables it writes have not been read by anything except the pipeline itself for a set period. Ninety days is a sensible default, because it catches weekly, monthly and quarterly use. Step 2 deals with the readers that a 90-day window misses.
Snowflake, BigQuery and Databricks all keep the history needed to find these tables. Each query below lists tables that were written in the last 90 days and not read in that time. Change the region, the window and the service accounts to match the setup.
Snowflake
Snowflake’s ACCESS_HISTORY view records reads in base_objects_accessed and writes in objects_modified. It covers the last 365 days, can lag by up to three hours, and needs Enterprise Edition or higher.
The filter on the pipeline’s own user matters. Incremental models read their own target table on every run, and without the filter every one of them would look busy. The same filter hides reads by downstream models that run under that user, which is why Step 2 checks the model graph separately. Snowflake also notes that not every statement in QUERY_HISTORY is recorded in ACCESS_HISTORY, so treat each result as a table to check.
BigQuery
In BigQuery, the INFORMATION_SCHEMA.JOBS view lists the tables each query referenced, keeps 180 days of history, and needs a region qualifier such as region-us.
Two details change the result. The referenced_tables column is only filled for queries that are not served from cache, so a dashboard that always hits cached results can make its source table look unread. And JOBS covers one project, so reads from other projects need the JOBS_BY_ORGANIZATION view, which requires a Google Cloud organization.
Databricks
Databricks records reads and writes by notebooks, jobs, pipelines and dashboards in the system.access.table_lineage system table, which keeps a rolling 395 days.
Databricks only emits lineage records when it can infer lineage, so the table holds a subset of all events. Cross-check each result against the job’s run history before moving on.
The BI layer and the pipeline behind each table
A table can be read every fifteen minutes by a dashboard that no person has opened since spring. Pull view history from the BI tool and add tables whose only readers are unviewed dashboards to the list.
Then map each table back to the pipeline that writes it: the dbt model of the same name, the Airflow or Dagster task that loads it, or the scheduled query. From here on, the list is a list of pipelines.
Step 2: Check what still depends on each candidate
An empty read history is a reason to look closer. It does not prove that nobody needs the data. These are the places where readers usually hide.
Annual and quarterly work. Year-end close, audits, tax filings and board packs read some tables once or twice a year. Check with finance before treating anything as unused on a 90-day window.
Downstream models. A table with no direct readers may feed another model that people do read. In dbt,
dbt ls --select model_name+lists everything downstream of a model. If anything in that list has readers, the candidate stays.Exports and syncs. Data copied out to a CRM, a partner’s storage bucket or a spreadsheet often reads through a service account that the Step 1 query filtered out, or through a tool whose reads never reach the warehouse logs.
Machine learning features. A feature table read once a quarter for retraining still matters to the model that depends on it.
Rare human readers. One query last month from the analyst who prepares the CFO’s numbers counts as a reader.
Shares and other accounts. Tables shared with another account, or exposed through an authorized view, can be read in places the provider account’s history does not cover. Check the sharing setup for each candidate.
For each candidate that survives these checks, record an owner. If the owner has left, the commit history of the pipeline code usually names the last person who changed it, and that person is the best place to start.
Step 3: Decide what each candidate gets
Deletion is one of four options. The lighter ones are often the right answer, and each can be reversed more easily than the one after it.
Refresh less often. Moving an hourly pipeline to daily cuts its runs from about 720 a month to 30. This suits tables with rare but real readers, such as a quarterly report.
Stop the schedule and keep the table. The data freezes at its last refresh. Anyone who needs it fresh will notice and say so, and the table is still there for anyone who only needs history.
Archive. Export the table to cheaper storage, or move it to a cold tier, and drop the warehouse copy. Write down where the archive lives and how to restore it.
Delete. Remove the table, the pipeline code and the schedule.
Match the option to what Step 2 found. A table with a yearly reader gets a slower schedule timed to that reader. A table that only an abandoned dashboard reads can go straight to a stopped schedule, along with the dashboard. A duplicate of a table that another team maintains gets merged: point its readers at the standard copy, then retire it. Deletion is for tables with no readers, no downstream models and no owner who objects.
Step 4: Retire in stages
A retirement that breaks one board report can cost the whole cleanup its support. Staging each retirement keeps every step reversible until the last one.
Announce it. Post the pipeline name, the table it writes, the pause date and a contact in the channel where data consumers ask questions. Message every reader the history found, however rare.
Pause the schedule. Stop the pipeline and leave the table in place. Some teams also rename the table with a suffix such as
_deprecated, so any query against the old name fails with a clear error and whoever owns that query comes forward.Wait out a quiet period. Two to four weeks is common. If a month-end or quarter-end close falls inside the window, extend it past the close.
Archive. Move the data to cheaper storage and confirm it can be restored.
Delete. Drop the table, remove the pipeline code and its schedule, and update the catalog and any documentation that mentions it.
Keep a short log as you go: what was retired, when, who approved it, where the archive sits and how to bring it back. Agree up front on who can approve a retirement and how long each quiet period lasts. The log turns a later question about a missing table into a five-minute answer.
Step 5: Measure the savings and stop regrowth
Record what each pipeline cost before pausing it. Most warehouses attribute compute to individual queries, and the dbt platform reports the compute cost of each model and job. Compare the same figure a month after retirement and add it up across everything retired. That total is what makes the case for the next round, and it is worth sending to whoever approved this one.
Unused pipelines come back unless something stops them. Three habits keep the list short:
Give every new pipeline an owner and a review date. Make both part of the pull request that creates it.
Run the Step 1 query every quarter. The history is already there, and a quarterly run keeps each review small.
Use the warning the warehouse already gives. Snowflake’s Optimization insights flag large tables that have not been queried in the last week, and tables over 100 GB that are written to but never read. Route those flags to the owner of each table.
Questions people ask
How long should a table go unread before it counts as unused?
Ninety days is a good starting window, because it covers weekly, monthly and quarterly use. Tables tied to annual work need a full year of history, which Snowflake and Databricks keep and BigQuery’s 180-day job history does not.
Is it safe to delete a pipeline that nobody has read in 90 days?
Only after Step 2 and a quiet period with the schedule paused. Pausing first costs a few weeks and catches the readers that query history misses.
What about tables that only other pipelines read?
Follow the chain to its end. If nothing at the end of the chain has a human or business reader, the whole chain is a candidate, and it should be retired from the last table backward.
How much does retiring unused pipelines save?
It depends on how often each pipeline refreshes and how much it processes, and there is no reliable industry figure to borrow. Measure each pipeline’s cost before pausing it, as Step 5 describes, and report your own total.
Where Dataring fits
We built Dataring to do this work continuously. It ranks every dataset by how much the business depends on it, and retires the pipelines nobody depends on within limits each company sets, under its supervision. The datasets the business relies on get the resources to stay reliable.
Book a free call to see it on your data estate. For the wider process this guide fits into, read How to audit your data estate in seven steps.



