ABOUT THIS BLOG

In this blog, we’ll discuss all things data quality and data observability. We’ll present interviews with leading data scientists, and we’ll discuss our own working building data observability solutions for data engineers and data scientists working in a broad range of industries.

Verify Your Snowflake Data Freshness Free with Data Culpa Fresh

by | Jul 13, 2022 | Data Culpa

One of the fundamental challenges of data quality is freshness: Did the data pipelines run and push new data into your data warehouse tables as expected?

With all the tools available for transforming and loading data, it can be difficult to ensure that all data has been updated according to plan. Different tables may run on different schedules. Perhaps some data refreshes hourly, while other data refreshes once a week. This can challenge traditional monitoring solutions.

At Data Culpa, we want to empower all data teams, no matter how big or small, to have timely information about the status and quality of their data. Some teams are reluctant to take on a new monitoring solution, because they’re concerned about extensive testing or departmental buy-in. While they wait, they continue to struggle with problems of data freshness, sometimes jeopardizing data analytics or business operations.

At Data Culpa, we want to help. And we want to make that help fast and straightforward.

That’s why we’re excited to announce Data Culpa Fresh, a SaaS service that brings the ultimate self-service monitoring experience to all customers using Snowflake. You can sign up for the waiting list at the bottom of this post.

You can set up Data Culpa Fresh in just minutes and begin checking the data in your Snowflake tables for freshness. Once an update interval is learned, Fresh will begin to automatically alert you if things begin looking stale. When it discovers stale data, Data Culpa Fresh sends you a Slack alert.

How Does It Work?

Data Culpa Fresh looks at two key metrics: the count of rows and the max value on columns. Over time we infer which columns are likely to represent motion and reduce our queries to just a few that are likely to show motion. We also build a time model representing when we expect data motion to occur. We check for motion hourly.

If your data starts to look stale, we push an alert to Slack letting you know how long it’s been since the data moved and when we expected it to move.

Screenshot showing Slack integration of Data Culpa Fresh alerts
An example of a Slack alert from Data Culpa Fresh.

Start Monitoring 10 Snowflake Tables Free

With the free plan, you can monitor up to 10 tables for free. A set of additional 25 tables is just $25 per month. If you decide to upgrade to our full monitoring solution, Fresh is included for free for all your tables monitored with Data Culpa Validator.

So go ahead and monitor freshness for free with your 10 most important tables. You can set up a limited service account to authenticate your warehouse with Data Culpa and verify its operation all in just a few clicks in our UI.

Sign up for a free account by filling out the form below, and we’ll contact you as soon as Data Culpa Fresh goes live.

Have Questions?

Feel free to reach out to us!

NEWSLETTER SIGN UP

Subscribe to the Data Culpa Newsletter