In this blog, we’ll discuss all things data quality and data observability. We’ll present interviews with leading data scientists, and we’ll discuss our own working building data observability solutions for data engineers and data scientists working in a broad range of industries.
Recently I was talking to a customer who had used a competitor’s product for data monitoring. It sounded like the product had the users specify parameters about the data; e.g., “this column should never be null.” This is all well and good, except that the product then...
BOSTON, MA. September 13, 2022 – Data Culpa, Inc., a provider of high-scale, low-touch data observability solutions for data engineering projects of all kinds, today announced that the US Patent & Trademark Office has issued patent #11,429,614 to Data Culpa for...
There’s a common ingest pattern we see across customers. Data lands in some file/object storage, e.g., S3, as a JSON/Parquet/CSV/etc, and then is ingested into Snowflake, potentially into two tables: one for a raw load and one or more that transform the data in some...
Just as Maslow identified a hierarchy of needs for people, data teams have a hierarchy of needs, beginning with data freshness; including volumes, schemas, and values; and culminating with lineage. In this blog post, which was published in the Data Science area of the...
One of the best features we offer customers who are just getting started with Data Culpa is our "timeshift" feature. Timeshift lets us extract a point in time for a row or a document and have Validator evaluate the data contained within that row or document as if it...
A company’s data is its most valuable asset — that's a commonplace observation in board rooms and IT labs today. Every organization recognizes the importance of its data. Somewhere in the mission statement for most digital transformation projects, there’s probably a...
Lauren S. Moores, a.k.a. “the Data Queen,” is Head of Data Sciences Innovation for FL65 and Invaio Sciences, two companies launched by Flagship Pioneering, a venture capital and private equity firm dedicated to creating breakthroughs in human health and sustainability...
Lauren S. Moores, a.k.a. “the Data Queen,” is Head of Data Sciences Innovation for FL65 and Invaio Sciences, two companies launched by Flagship Pioneering, a venture capital and private equity firm dedicated to creating breakthroughs in human health and sustainability...
Lauren S. Moores, a.k.a. “the Data Queen,” is Head of Data Sciences Innovation for FL65 and Invaio Sciences, two companies launched by Flagship Pioneering, a venture capital and private equity firm dedicated to creating breakthroughs in human health and sustainability...
Data engineering teams need to be able to find and fix data quality problems quickly. In part 2 of our interview with Michal Klos, Michal discusses the challenges of discovering and troubleshooting data quality problems in pipelines. Michal is Sr. Director of...
Data quality is an ongoing challenge for data scientists and data engineers. In this interview, Michal Klos share the insights he's gleaned from a long career building data pipelines and ensuring they deliver high-quality results. Michal is Sr. Director of Engineering...
At Data Culpa, we're building solutions to help data engineers and data scientists catch data quality problems before they jeopardize data pipeline results. Nearly every business today is becoming more data-centric. Data not only drives operations and decision making;...
Here’s what we’re writing on Medium:
The Data Quality Gap (June 11, 2020)
4 Lessons for Data Scientists from the UK’s A-Levels Algorithm Debacle (August 18, 2020)
Feel free to reach out to us!
Subscribe to the Data Culpa Newsletter