Skip to main content

​
What is this?

Daily Job Posting Data Deliveries enable customers to receive daily refreshes of PDL’s Job Posting Data. Unlike traditional “push” deliveries, this is a pull-based system, meaning:
  • Pull: Customers retrieve data from a PDL-hosted S3 bucket using credentials provisioned by PDL.
  • Push: PDL delivers data directly to the customer’s S3 bucket or cloud storage.
This approach is intended to simplify the delivery process and likelihood of delivery errors due to logistical errors. With this approach, customers no longer need to manage or configure credentials to receive their data and can access it on-demand and as many times as needed. For more information see: Data Delivery Using Direct Download
Important Notes
  • Daily deliveries are currently only available for Job Posting Data.
  • Initial access is limited to S3 buckets; future expansions will include Snowflake and other platforms.

​
How does it work?

OverviewAs a Daily Delivery Job Posting Customer, you will receive 2 different types of deliveries:
  1. Full Update (delivered each month, as well as for your very first delivery)
  2. Delta Update (delivered each day)
This data will be delivered to a PDL-hosted S3 bucket specific to your individual customer account.

​
S3 Bucket Details

Daily Job Posting Data is organized in a customer-specific S3 bucket. Each day has its own folder with the format:
If that day’s delivery corresponds to a Full Update (delivered once each month), the data will be contained in a full/ subdirectory:
Otherwise, that day’s delivery will correspond to a Delta Update (delivered daily when no full update is scheduled for the day). Delta updates will be contained in a daily/ subdirectory:

​
Full Updates (Monthly Release + Initial Release)

Initial Release The first delivery you will receive will be a full version of the data. The full version of the data contains every job posting record PDL has collected since 10/1/2024, in gzipped, jsonl files. Monthly Releases When PDL releases a new version of our company data as part of our monthly release process, the major_version and minor_version of the data will change. This means that a significant number of the historic job postings could have changes to their title tagging, location standardization, company mapping, etc. As such, we will deliver a fully updated dump of the job posting data during every monthly release in the same format as the initial dump.

​
Delta Updates (Daily Release):

Each daily release is a delta release. It includes:
  • COMPLETED files in the top-level delivery folder (which can be used to trigger data ingestion workflows)
  • Delta files – organized into the following subfolders:
    • Added: New records (first_seen date after last release)
    • Updated: Records updated since last release (based on last_verified date)
    • Deactivated: Newly deactivated records (based on deactivated_date)
    • Deleted: Records removed since the previous release

​
Delivery Failures

Beta NoticeDaily Job Posting Updates are currently in beta. Given the early-stage nature of this new delivery mechanism, customers should set expectations around:
  • Occasional issues or missed daily updates
  • Potential changes to delivery method or format based on feedback
Unlike monthly deliveries, daily deliveries only have one attempt per day. Potential failure scenarios include:
  • Outages in partner crawls
  • Data quality issues from partners
  • Pipeline failures within PDL
In the event of a failed delivery the PDL Engineering team is constantly monitoring for quick fixes, but there may still be some days with no delivery. Customers can configure notifications from their API Dashboard to be alerted in the case when data is not delivered. Note that missed data will be backfilled in the next release window (usually the following day).
Configuring Delivery NotificationsCustomers can configure notifications via API Dash > License Delivery > Settings.By default:
  • Notifications for both successes and failures are enabled
  • Customers can adjust settings to meet their preferences

​
How to Get Access

Daily Job Posting Data Deliveries are available as an add-on for Job Posting Data customers. As of October 2025 (v32.0): The feature is in open beta
  • Feedback is welcomed to help improve delivery reliability and features
  • Currently not available for Snowflake
  • Only supported for Job Posting Data licenses
To sign up for beta access, reach out to your Customer Success team.

Related topics

October 2025 Release NotesFebruary 2026 Release NotesJuly 2026 Release NotesJob Posting Data OverviewGetting Started with a Data License