< blog
8 min read

The Warehouse That Ran All Year to Refresh Ten Tables

Person working at a laptop in a bright office with Snowflake logos and a small iceberg on the desk

A customer had a medium warehouse that never suspended. Not overnight, not on weekends. Its only job was to run ten metadata refresh queries, one per Iceberg table, once a minute.

Each of those queries finished in milliseconds. The warehouse was billed for every second of every minute, all year.

This shape of waste is expensive and close to invisible in the places most teams look.

Find out where you stand

Let us take a peek under the hood with a free assessment and no commitment.

Find your savings

Why query-level cost analysis misses it

If you check QUERY_ATTRIBUTION_HISTORY, these queries look free, if they appear at all: the view leaves out warehouse idle time and skips queries that run for about 100 milliseconds or less. They are tiny. They read metadata, not data. Nothing about them suggests a problem.

The cost is not in the query. It is in the warehouse being awake to run it. Snowflake bills a running warehouse by the second, with a sixty-second minimum each time it starts or resumes. It suspends only after a stretch with no queries, set by AUTO_SUSPEND, which defaults to ten minutes. A query that runs every sixty seconds never leaves that gap, so the warehouse stays up and billable around the clock while doing almost no work. A shorter AUTO_SUSPEND does not rescue it: each resume is billed for a full sixty seconds, so a query every minute still means paying for every minute.

The ratio is what makes this painful. Ten queries a minute, each finishing in milliseconds, were holding up a medium warehouse for every second of every hour.

How to tell whether you have it

The signature is visible in WAREHOUSE_METERING_HISTORY, and once you have seen it once you will recognize it immediately. Plot credits per hour for a warehouse over two weeks and look for a flat line where you expect a business-hours curve. A warehouse serving analysts has peaks and troughs. A warehouse held up by a cron task is level, including at three in the morning on a Sunday. WAREHOUSE_EVENTS_HISTORY confirms it: over the same two weeks, a warehouse held up this way logs no suspend events with the reason WAREHOUSE_AUTOSUSPEND.

Then cross-check against query history for the same warehouse. The tell is high uptime with near-zero total execution time. If a warehouse is awake around the clock and running nothing but short, repeated queries, something is keeping it alive rather than using it.

Three patterns produce this shape, and all three are worth checking:

  • Scheduled tasks that run on a warehouse more often than its auto-suspend setting, particularly anything running every minute
  • Metadata or catalog refresh operations pointed at an external table format
  • Health checks and monitoring queries that someone set up years ago and nobody has revisited

The fix

Snowflake can keep Iceberg metadata in sync with an external catalog without using a warehouse at all. Auto-refresh runs on Snowpipe’s serverless compute, its charges appear on the Snowpipe line of your bill instead of on a warehouse, and it removes the reason the warehouse was awake.

CREATE CATALOG INTEGRATION auto_refresh_catalog_integration
CATALOG_SOURCE            = GLUE
CATALOG_NAMESPACE         = 'my_catalog_namespace'
TABLE_FORMAT              = ICEBERG
GLUE_AWS_ROLE_ARN         = 'arn:aws:iam::123456789123:role/my-catalog-role'
GLUE_CATALOG_ID           = '123456789123'
ENABLED                   = TRUE
REFRESH_INTERVAL_SECONDS  = 60;

If your Glue catalog is in a different AWS region from your Snowflake account, or the account runs on Azure or Google Cloud, add GLUE_REGION = ‘<aws-region>’ to that statement. Snowflake can read the catalog only after you run DESCRIBE CATALOG INTEGRATION and add the IAM user ARN and external ID it returns to the role’s trust policy.

If your tables already use a Glue catalog integration, set the interval on that one instead of creating a new one:

ALTER CATALOG INTEGRATION my_glue_integration SET REFRESH_INTERVAL_SECONDS = 60;

The integration on its own refreshes nothing. Auto-refresh is off by default on Iceberg tables that use a Glue integration, so switch it on for each of the ten tables:

ALTER ICEBERG TABLE analytics.event_log SET AUTO_REFRESH = TRUE;

Then suspend or drop the cron tasks that were doing the same job. That last step is the one people forget, and leaving them running means paying for both.

REFRESH_INTERVAL_SECONDS is the freshness dial. It accepts any value from 30 to 86,400 seconds and defaults to 30. A lower value narrows the lag between the external catalog and Snowflake, but Snowflake warns that a short interval on frequently updated tables can cause performance degradation. Sixty seconds matched what the old tasks were doing, so it was the like-for-like comparison.

For a new setup, Snowflake now recommends connecting to Glue through its Iceberg REST endpoint instead. Tables already on a Glue integration like this one can be moved to a REST integration with SYSTEM$SET_CATALOG_INTEGRATION, as long as neither integration uses vended credentials.

Verifying it worked

Two checks. The first confirms auto-refresh is actually running:

SELECT SYSTEM$AUTO_REFRESH_STATUS('my_db.analytics.event_log');

The function needs a role that owns the table. An executionState of RUNNING says auto-refresh is healthy, not that it is keeping up, so also check that pendingSnapshotCount falls and that the gap between lastSnapshotTime and lastUpdatedTime stays short. SHOW ICEBERG TABLES returns the same status for all ten tables at once, in its auto_refresh_status column.

The second shows what it costs, so you can put the two meters side by side. Snowflake records auto-refresh usage under each Iceberg table’s name, so filter on the table names (one is shown here). The view can run up to three hours behind:

SELECT *
FROM   SNOWFLAKE.ACCOUNT_USAGE.PIPE_USAGE_HISTORY
WHERE  PIPE_NAME ILIKE 'event_log'
AND    START_TIME >= DATEADD(day, -7, CURRENT_DATE);

Graph PIPE_USAGE_HISTORY against WAREHOUSE_METERING_HISTORY for the same period. You want to see the warehouse line drop and the pipe line stay close to the floor. If the warehouse line does not move, something else is holding it up and the cron tasks were not the only cause.

What auto-refresh still cannot handle

Position-delete files are no longer a blocker. Snowflake has supported row-level deletes on externally managed Iceberg tables since August 2025, and its documentation no longer lists automated refresh as unsupported with them.

A few limits remain. Snowflake does not support equality deletes. Auto-refresh applies schema changes only when they arrive with data changes, so a schema change made through DDL alone still needs a manual refresh. Streams on externally managed Iceberg v2 tables are insert-only, so they will not show deleted rows. And Snowflake warns that too many position deletes, dangling ones in particular, may stop refreshes from running, so keep up table maintenance in the engine that writes the table. This is worth establishing before you plan the migration rather than during it.

The general lesson

Look at warehouse uptime, not just query cost. Query-level attribution is the right tool for finding expensive queries and the wrong tool for finding cheap queries that are expensive in aggregate.

Small, frequent, trivial tasks are the ones worth auditing, because a warehouse is billed for the time it is awake, not for the work it does. A heavy query that runs twice a day is usually fine. A trivial one that runs every minute may not be. This applies to standard warehouses. Adaptive Warehouses, generally available since June 2026, bill per query rather than for time awake.