When consuming data from a third party, you and your data teams need to be sure the data is in the format you expect.
Data Culpa monitors the hierarchy of JSON documents over time across your environment, whether it is data at rest or in motion, and enables you to perform cutting edge monitoring and comparisons over time and environments.
- When did a field first appear?
- When did a subtree stop being used?
- How is a subtree different from its siblings?
- Are array elements consistent?
Data Culpa Validator helps answer these questions. We provide deep insights into JSON data feeds, including understanding schema variations and differences, subtree changes including new fields, probable typos, and type changes, all in addition to our usual value monitoring that we do for flat structured data.
Get immediate alerts when a schema contains new fields or when fields go missing. You can visually compare deep and wide schemas across your pipelines, data lakes, or databases in a variety of ways to help understand the volume of records using specific schemas and probability distributions across corresponding but different subtrees within the hierarchy.
You can provide Data Culpa feedback, too: tell Validator when something has gone wrong to remember it as an error, thus helping define what we call the gold schema.