Data Delivery Using Databricks

​

Set Up Delivery

1. Locate your Databricks sharing identifier. This is a string in the format <cloud>:<region>:<uuid>, for example:
   aws:eu-west-1:b0c978c8-3e68-4cdf-94af-d05c120ed1ef
   
   You can find your identifier in either of two ways:
   • Catalog Explorer
   • Notebook or SQL editor

   1. In your Databricks workspace, click Catalog.
   2. At the top of the Catalog pane, click the gear icon and select Delta Sharing (or, in the upper-right corner, click Share > Delta Sharing).
   3. On the Shared with me tab, click your organization name in the upper right and select Copy sharing identifier.

   Run the following on Unity-Catalog-enabled compute (standard or dedicated access mode):

   SQL

   SELECT CURRENT_METASTORE();
   
2. Contact your CSM and share your sharing identifier with them. We will create a recipient and share for your account and notify you, along with the provider and share names, once your data is available.
3. Create a catalog from the share. Once your delivery is complete and you are notified, a metastore admin (or a user with the CREATE CATALOG and USE PROVIDER privileges) will need to run the following:
   SQL

   CREATE CATALOG IF NOT EXISTS pdl_delivery
   USING SHARE peopledatalabs.{{share name}};
   
   Replace {{share name}} with the share name from our email. The delivered tables will now appear under the pdl_delivery catalog as read-only tables, queryable like any other catalog in Unity Catalog.
4. Copy the shared data into a table in your own instance. A Delta Sharing catalog is read-only. Copy the shared tables directly into your own Databricks catalog or workspace storage before querying them — querying local tables significantly improves query performance and speed for downstream transformations, joins, and analysis compared to querying directly from the share. Run the following to create a local writable copy:
   SQL

   CREATE OR REPLACE TABLE main.my_schema.pdl_person AS
   SELECT * FROM pdl_delivery.{{schema}}.{{table}};
   
   Using CREATE OR REPLACE TABLE ... AS SELECT handles both the first run and any later schema changes in a single statement, without requiring the destination table to already exist.
5. Assign permissions so your team can query the delivered data. Grant USE CATALOG, USE SCHEMA, and SELECT to the user or group that needs access:
   SQL

   GRANT USE CATALOG ON CATALOG pdl_delivery            TO `data-team`;
   GRANT USE SCHEMA  ON SCHEMA  pdl_delivery.{{schema}} TO `data-team`;
   GRANT SELECT      ON SCHEMA  pdl_delivery.{{schema}} TO `data-team`;
   
   Replace data-team with your target user or group. SELECT granted at the schema level applies to every table in that schema.

​

Note on Compute

​

Note on Read-Only Shares

​

Permission Inheritance