Skip to main content

​
What is a Delta File?

Delta Files are a new delivery format for data licenses that contains just the changes between the current delivery and the previous one (i.e. the “delta” between the two updates). Delta Files are designed to be significantly smaller both in terms of file size and number of records, with the goal of being simpler, faster and easier for users to ingest into their data pipeline. Delta File Deliveries are organized into 5 subfolders containing records that have changed between the previous delivery and the current one. The subfolders are:
  1. Merged IDs
  2. Added Records
  3. Updated Records
  4. Opted Out IDs
  5. Deleted IDs
LimitationsDelta Files are currently only available for Person Data Licenses and are not yet available for delivery through Snowflake.

​
How does it work?

Delta Files are custom generated for each customer based on their current data license slice and delivery cadences. When building the data delivery for each customer, we compare the most recent build of our data against the last delivered set of data received by the customer. We then extract just the subset of records that have changed and group them into the 5 directories listed above, which is delivered as a Delta File delivery. Delta files are delivered using the existing set delivery channels available to data license customers. During the initial onboarding period, customers will receive both the full data license as well as the delta file delivery in order to support transitioning to the new delivery format.
Important RemindersThere are a couple important requirements when using Delta Files to update your dataset:
  • You must be up-to-date with your last full license delivery before switching over to Delta Files (meaning you have completed ingesting it). This is to ensure that you do not retain any out-of-date records in your pipeline.
  • If you decide to change the data slice that you are licensing from PDL (i.e. any change to the underlying query), you will need to first re-ingest an initial full license delivery before switching back to Delta Files.

​
How to get access

Delta Files are available as an add-on for Data License Customers. As of our May 2024 (v26.1) release, this offering is currently in an open beta available for customers to evaluate the new format and share any additional feedback. At this time, the delta file format is not supported on Snowflake and only supported for Person data licenses, with potential to provide similar delta files for our Snowflake customers and company data in the future. To sign up for beta access to the Delta Files, please reach out to your Customer Success team