> ## Documentation Index
> Fetch the complete documentation index at: https://docs.peopledatalabs.com/llms.txt
> Use this file to discover all available pages before exploring further.
# Person Search API [Deprecated]
The endpoint for the person search api is `https://api.peopledatalabs.com/v5/person/search`.
# Person Search API Access and Billing.
We charge per record retrieved. Each person record in the "data" array of the response counts as a single "credit" against your total package.
# Usage
PDL's Search API is perfect for finding specific segments of people that you need in order to power your projects and products. This product gives you direct access to query our full [API dataset](/docs/datasets#api-dataset). There are many degrees of freedom which allow you to find any kind of person(s) with a single query.
## Requests
See [Authentication](/docs/authentication) and [Requests](/docs/requests) to see possible ways to input requests. We recommend using a JSON object to capture request parameters and will do so in the examples below.
## Rate Limiting
The current default rate limit is 10 requests per minute.
## Input Parameters
| Parameter Name | Description | Default | Example |
| -------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -------- | ---------------------------------------------------------------- |
| `query` | An Elasticsearch (v7.7) [query](https://www.elastic.co/guide/en/elasticsearch/reference/7.7/query-dsl.html). See our underlying Elasticsearch [mapping](#full-field-mapping) for reference. | | `{"query": {"term": {"job_company_name": "people data labs"}}}` |
| `sql` | A SQL query of the format: `SELECT * FROM person WHERE XXX`, where XXX is a standard SQL boolean query involving PDL [fields](/docs/fields). Any use of column selections or the LIMIT keyword will be ignored. | | `SELECT * FROM person WHERE job_company_name='people data labs'` |
| `size` | The batch size or the maximum number of matched records to return for this query if they exist. Must be between 1 and 100. | `1` | `100` |
| `from` | **\[LEGACY]** An offset value for paginating between batches. Can be a number between 0 and 9999. Pagination can be executed up to a maximum of 10,000 records per query. **NOTE, FROM CAN NOT BE USED WITH SCROLL\_TOKEN IN THE SAME REQUEST** | `0` | `0`, `100`, `200` ... |
| `scroll_token` | An offset key for paginating between batches. Unlike the legacy `from` parameter can be used for any number of records. Each search API response returns a scroll\_token which can be used to fetch the next `size` records. | `None` | `104$14.278746` |
| `dataset` | Specify which [dataset](/docs/datasets) the API should search against. You can input multiple datasets by separating with a comma. Valid names are `resume`, `email`, `phone`, `mobile_phone`, `street_address`, `consumer_social`, `developer`, `all`. You can also exclude datasets using `-` as the first character. | `resume` | `all` |
| `titlecase` | All text in the data of API responses is returned as lowercase by default. Setting `titlecase` to `true` will titlecase any records returned. | `false` | `true` |
| `pretty` | Whether the output should have [human-readable](http://jsonprettyprint.net/) indentation. | `false` | `true` |
| `api_key` | Your API key (note this can also be provided in the request header instead, as shown on the [Authentication](/docs/authentication) page) | | |
## Response
The HTTP Response code will be `200` for any valid request, regardless of whether records were found for your query or not. For that reason, pay close attention to the "total" value in your response object to understand query success. Each person record in the "data" array of the response counts as a single "credit" against your total package - this value has a maximum of one record by default to prevent happy accidents.
### Response Fields
| Field | Description | Type |
| --------------- | ------------------------------------------------------------------------------------------------------------------------------------ | --------------- |
| `status` | Response code. See a description of our [Error Codes](/docs/errors). | `Integer` |
| `error` | Error Details | `Object` |
| `error.type` | Error Details | `List (String)` |
| `error.message` | Error Details | `String` |
| `data` | Data returned. See the [full example response](#full-example-response) or the [example company record](/docs/example-company-record) | `Object` |
| `total` | Number of records matching a given `query` or `sql` input. | `Integer` |
| `scroll_token` | Scroll value used for pagination | `String` |
Abridged 200 Response Example (full example [here](#full-example-response)):
```
{
"status": 200,
"data": [
{
"id": "qEnOZ5Oh0poWnQ1luFBfVw_0000",
"full_name": "sean thorne",
...
},
...
],
"scroll_token": "1117$12.176522"
"total": 99
}
```
# Building a Query
It is required to provide a value for either the `query` parameter or the `sql` parameter in order to receive a successful response. The `query` value should align directly with the [Elasticsearch DSL](https://www.elastic.co/guide/en/elasticsearch/reference/7.7/query-dsl.html). SQL queries are executed using [Elasticsearch SQL](https://www.elastic.co/what-is/elasticsearch-sql). Most typical query types are available but some are excluded. For all available query types see [here](#query-limitations).
When an API request is executed, the query is run directly against our [API dataset](/docs/stats) without doing any additional cleaning or pre-processing. This means that you have a *ton* of freedom to explore the dataset and return the perfect records. It also means that understanding the available fields can be very helpful to making successful queries.
Field descriptions can be found [here](/docs/fields) and the Elasticsearch mapping underlying this api can be found [here](#full-field-mapping). To help you identify how to best query for specific sub-entities (schools, companies, and locations), we offer a suite of enrichment APIs for these sub-entities called the [Cleaner APIs](/docs/cleaner-apis)
# Walkthroughs
All code is Python, cURL, JavaScript and Ruby.
## Basic Usage
*"I want to make a query and save the results to a file."*
```python Python3 SDK (Elasticsearch) expandable theme={null}
import json
# See https://github.com/peopledatalabs/peopledatalabs-python
from peopledatalabs import PDLPY
# Create a client, specifying an API key
client = PDLPY(
api_key="YOUR API KEY",
)
ES_QUERY = {
"query": {
"bool": {
"must": [
{"term": {"location_country": "mexico"}},
{"term": {"job_title_role": "health"}},
{"exists": {"field": "phone_numbers"}}
]
}
}
}
P = {
'query': ES_QUERY,
'size': 10,
'pretty': True
}
response = client.person.search(**P).json()
if response["status"] == 200:
data = response['data']
with open("my_pdl_search.jsonl", "w") as out:
for record in data:
out.write(json.dumps(record) + "\n")
print(f"successfully grabbed {len(data)} records from pdl")
print(f"{response['total']} total pdl records exist matching this query")
else:
print("NOTE. The carrier pigeons lost motivation in flight. See error and try again.")
print("Error:", response)
```
```python Python3 SDK (SQL) expandable theme={null}
import json
# See https://github.com/peopledatalabs/peopledatalabs-python
from peopledatalabs import PDLPY
# Create a client, specifying an API key
client = PDLPY(
api_key="YOUR API KEY",
)
SQL_QUERY = \
"""
SELECT * FROM person
WHERE location_country='mexico'
AND job_title_role='health'
AND phone_numbers IS NOT NULL;
"""
P = {
'sql': SQL_QUERY,
'size': 10,
'pretty': True
}
response = client.person.search(**P).json()
if response["status"] == 200:
data = response['data']
with open("my_pdl_search.jsonl", "w") as out:
for record in data:
out.write(json.dumps(record) + "\n")
print(f"successfully grabbed {len(data)} records from pdl")
print(f"{response['total']} total pdl records exist matching this query")
else:
print("NOTE. The carrier pigeons lost motivation in flight. See error and try again.")
print("Error:", response)
```
```bash cURL theme={null}
# Elasticsearch
curl -X GET 'https://api.peopledatalabs.com/v5/person/search' \
-H 'X-Api-Key: xxxx' \
--data-raw '{
"size": 10,
"query": {
"bool": {
"must": [
{"term": {"location_country": "mexico"}},
{"term": {"job_title_role": "health"}},
{"exists": {"field": "phone_numbers"}}
]
}
}
}'
# SQL
curl -X GET \
'https://api.peopledatalabs.com/v5/person/search' \
-H 'X-Api-Key: xxxx' \
--data-raw '{
"size": 10,
"sql": "SELECT * FROM person WHERE location_country='\''mexico'\'' AND job_title_role='\''health'\'' AND phone_numbers IS NOT NULL;"
}'
```
```javascript JavaScript (Elasticsearch) theme={null}
// See https://github.com/peopledatalabs/peopledatalabs-js
import PDLJS from 'peopledatalabs';
const PDLJSClient = new PDLJS({ apiKey: "YOUR API KEY" });
const esQuery = {
query: {
bool: {
must:[
{term: {location_country: "mexico"}},
{term: {job_title_role: "health"}},
{exists: {field: "phone_numbers"}}
]
}
}
}
const params = {
searchQuery: esQuery,
size: 10,
pretty: true
}
PDLJSClient.person.search.elastic(params).then((data) => {
console.log(`Number of records found: ${data['total']}`);
}).catch((error) => {
console.log(error);
});
```
```javascript JavaScript (SQL) theme={null}
// See https://github.com/peopledatalabs/peopledatalabs-js
import PDLJS from 'peopledatalabs';
const PDLJSClient = new PDLJS({ apiKey: "YOUR API KEY" });
const sqlQuery = `SELECT * FROM person
WHERE location_country='mexico'
AND job_title_role='health'
AND phone_numbers IS NOT NULL;`
const params = {
searchQuery: sqlQuery,
size: 10,
pretty: true
}
PDLJSClient.person.search.sql(params).then((data) => {
console.log(`Number of records found: ${data['total']}`);
}).catch((error) => {
console.log(error);
});
```
```ruby Ruby (Elasticsearch) theme={null}
require 'json'
# See https://github.com/peopledatalabs/peopledatalabs-ruby
require 'peopledatalabs'
Peopledatalabs.api_key = 'YOUR API KEY'
ES_QUERY = {
"query": {
"bool": {
"must": [
{"term": {"location_country": "mexico"}},
{"term": {"job_title_role": "health"}},
{"exists": {"field": "phone_numbers"}}
]
}
}
}
response = Peopledatalabs::Search.people(searchType: 'elastic', query: ES_QUERY, size: 10, pretty: true)
if response['status'] == 200
data = response['data']
File.open("my_pdl_search.jsonl", "w") do |out|
data.each { |record| out.write(JSON.dump(record) + "\n") }
end
puts "successfully grabbed #{data.length()} records from pdl"
puts "#{response['total']} total pdl records exist matching this query"
else
puts "NOTE. The carrier pigeons lost motivation in flight. See error and try again."
puts "Error: #{response}"
end
```
```ruby Ruby (SQL) theme={null}
require 'json'
# See https://github.com/peopledatalabs/peopledatalabs-ruby
require 'peopledatalabs'
Peopledatalabs.api_key = 'YOUR API KEY'
SQL_QUERY = \
"""
SELECT * FROM person
WHERE location_country='mexico'
AND job_title_role='health'
AND phone_numbers IS NOT NULL;
"""
response = Peopledatalabs::Search.people(searchType: 'sql', query: SQL_QUERY, size: 10, pretty: true)
if response['status'] == 200
data = response['data']
File.open("my_pdl_search.jsonl", "w") do |out|
data.each { |record| out.write(JSON.dump(record) + "\n") }
end
puts "successfully grabbed #{data.length()} records from pdl"
puts "#{response['total']} total pdl records exist matching this query"
else
puts "NOTE. The carrier pigeons lost motivation in flight. See error and try again."
puts "Error: #{response}"
end
```
```python Python3 (Elasticsearch) expandable theme={null}
import requests, json
API_KEY = # YOUR API KEY
PDL_URL = "https://api.peopledatalabs.com/v5/person/search"
H = {
'Content-Type': "application/json",
'X-api-key': API_KEY
}
ES_QUERY = {
"query": {
"bool": {
"must": [
{"term": {"location_country": "mexico"}},
{"term": {"job_title_role": "health"}},
{"exists": {"field": "phone_numbers"}}
]
}
}
}
P = {
'query': json.dumps(ES_QUERY),
'size': 10,
'pretty': True
}
response = requests.get(
PDL_URL,
headers=H,
params=P
).json()
if response["status"] == 200:
data = response['data']
with open("my_pdl_search.jsonl", "w") as out:
for record in data:
out.write(json.dumps(record) + "\n")
print(f"successfully grabbed {len(data)} records from pdl")
print(f"{response['total']} total pdl records exist matching this query")
else:
print("NOTE. The carrier pigeons lost motivation in flight. See error and try again.")
print("Error:", response)
```
```python Python3 (SQL) expandable theme={null}
import requests, json
API_KEY = # YOUR API KEY
PDL_URL = "https://api.peopledatalabs.com/v5/person/search"
H = {
'Content-Type': "application/json",
'X-api-key': API_KEY
}
SQL_QUERY = \
"""
SELECT * FROM person
WHERE location_country='mexico'
AND job_title_role='health'
AND phone_numbers IS NOT NULL;
"""
P = {
'sql': SQL_QUERY,
'size': 10,
'pretty': True
}
response = requests.get(
PDL_URL,
headers=H,
params=P
).json()
if response["status"] == 200:
data = response['data']
with open("my_pdl_search.jsonl", "w") as out:
for record in data:
out.write(json.dumps(record) + "\n")
print(f"successfully grabbed {len(data)} records from pdl")
print(f"{response['total']} total pdl records exist matching this query")
else:
print("NOTE. The carrier pigeons lost motivation in flight. See error and try again.")
print("error:", response)
```
## Using POST Requests
"I would like to use POST requests to query instead of GET requests so that I can make queries with a lot of parameters"
**Difference between GET and POST requests**
See here for a comparison of the practical [differences between GET and POST requests](https://www.w3schools.com/tags/ref_httpmethods.asp). Perhaps most practically, POST requests do not have any limit on the amount of data that can be passed in the request.
```python Python3 (Elasticsearch) expandable theme={null}
import requests, json
API_KEY = # YOUR API KEY
PDL_URL = "https://api.peopledatalabs.com/v5/person/search"
H = {
'Content-Type': "application/json",
'X-api-key': API_KEY
}
ES_QUERY = {
"query": {
"bool": {
"must": [
{"term": {"location_country": "mexico"}},
{"term": {"job_title_role": "health"}},
{"exists": {"field": "phone_numbers"}}
]
}
}
}
P = {
'query': ES_QUERY, # This is different from using GET requests
'size': 10,
'pretty': True
}
response = requests.post( # Using POST method
PDL_URL,
headers=H,
json=P # Passing the data directly as a JSON object
# data=json.dumps(P) # This is an alternative way of passing data using a string
).json()
if response["status"] == 200:
data = response['data']
with open("my_pdl_search.jsonl", "w") as out:
for record in data:
out.write(json.dumps(record) + "\n")
print(f"successfully grabbed {len(data)} records from pdl")
print(f"{response['total']} total pdl records exist matching this query")
else:
print("NOTE. The carrier pigeons lost motivation in flight. See error and try again.")
print("Error:", response)
```
```python Python3 (SQL) expandable theme={null}
import requests, json
API_KEY = # YOUR API KEY
PDL_URL = "https://api.peopledatalabs.com/v5/person/search"
H = {
'Content-Type': "application/json",
'X-api-key': API_KEY
}
SQL_QUERY = \
"""
SELECT * FROM person
WHERE location_country='mexico'
AND job_title_role='health'
AND phone_numbers IS NOT NULL;
"""
P = {
'sql': SQL_QUERY, # This is different from using GET requests
'size': 10,
'pretty': True
}
response = requests.post( # Using POST method
PDL_URL,
headers=H,
json=P # Passing the data directly as a JSON object
# data=json.dumps(P) # This is an alternative way of passing data using a string
).json()
if response["status"] == 200:
data = response['data']
with open("my_pdl_search.jsonl", "w") as out:
for record in data:
out.write(json.dumps(record) + "\n")
print(f"successfully grabbed {len(data)} records from pdl")
print(f"{response['total']} total pdl records exist matching this query")
else:
print("NOTE. The carrier pigeons lost motivation in flight. See error and try again.")
print("Error:", response)
```
```bash cURL theme={null}
# Elasticsearch
curl -X POST 'https://api.peopledatalabs.com/v5/person/search' \
-H 'X-Api-Key: xxxx' \
--data-raw '{
"size": 10,
"query": {
"bool": {
"must": [
{"term": {"location_country": "mexico"}},
{"term": {"job_title_role": "health"}},
{"exists": {"field": "phone_numbers"}}
]
}
}
}'
# SQL
curl -X POST \
'https://api.peopledatalabs.com/v5/person/search' \
-H 'X-Api-Key: xxxx' \
--data-raw '{
"size": 10,
"sql": "SELECT * FROM person WHERE location_country='\''mexico'\'' AND job_title_role='\''health'\'' AND phone_numbers IS NOT NULL;"
}'
```
## Searching Specific Datasets
"I want to run a simple query against PDL's [phone dataset](/docs/datasets#phone-slice)"
**Maintaining Backwards Compatibility**
The `dataset` parameter was introduced with the [July 2021 release](/changelog/july-2021-release-notes#changing-the-default-de-duplication-in-the-search-api), which also changed the default dataset from `all` to `resume`. For users that want to maintain the same performance in their queries prior to this change, set the `dataset` parameter to `all` as shown in the example below.
```python Python3 SDK (Elasticsearch) expandable theme={null}
import json
# See https://github.com/peopledatalabs/peopledatalabs-python
from peopledatalabs import PDLPY
# Create a client, specifying an API key
client = PDLPY(
api_key="YOUR API KEY",
)
ES_QUERY = {
"query": {
"bool": {
"must": [
{"term": {"location_country": "mexico"}},
{"term": {"job_title_role": "health"}}
]
}
}
}
P = {
'query': ES_QUERY,
'size': 10,
'pretty': True,
'dataset': "phone" # Use search against all PDL records with a phone number
}
response = client.person.search(**P).json()
if response["status"] == 200:
data = response['data']
with open("my_pdl_search.jsonl", "w") as out:
for record in data:
out.write(json.dumps(record) + "\n")
print(f"successfully grabbed {len(data)} records from pdl")
print(f"{response['total']} total pdl records exist matching this query")
else:
print("NOTE. The carrier pigeons lost motivation in flight. See error and try again.")
print("Error:", response)
```
```python Python3 SDK (SQL) expandable theme={null}
import json
# See https://github.com/peopledatalabs/peopledatalabs-python
from peopledatalabs import PDLPY
# Create a client, specifying an API key
client = PDLPY(
api_key="YOUR API KEY",
)
SQL_QUERY = \
"""
SELECT * FROM person
WHERE location_country='mexico'
AND job_title_role='health'
AND phone_numbers IS NOT NULL;
"""
P = {
'sql': SQL_QUERY,
'size': 10,
'pretty': True,
'dataset': "phone" # Use all to search against all PDL records with a phone number
}
response = client.person.search(**P).json()
if response["status"] == 200:
data = response['data']
with open("my_pdl_search.jsonl", "w") as out:
for record in data:
out.write(json.dumps(record) + "\n")
print(f"successfully grabbed {len(data)} records from pdl")
print(f"{response['total']} total pdl records exist matching this query")
else:
print("NOTE. The carrier pigeons lost motivation in flight. See error and try again.")
print("Error:", response)
```
```bash cURL theme={null}
# Elasticsearch
curl -X GET 'https://api.peopledatalabs.com/v5/person/search' \
-H 'X-Api-Key: xxxx' \
--data-raw '{
"size": 10,
"dataset": "all",
"query": {
"bool": {
"must": [
{"term": {"location_country": "mexico"}},
{"term": {"job_title_role": "health"}},
]
}
}
}'
# SQL
curl -X GET \
'https://api.peopledatalabs.com/v5/person/search' \
-H 'X-Api-Key: xxxx' \
--data-raw '{
"size": 10,
"dataset: "phone",
"sql": "SELECT * FROM person WHERE location_country='\''mexico'\'' AND job_title_role='\''health'\'' AND phone_numbers IS NOT NULL;"
}'
```
```javascript JavaScript (Elasticsearch) theme={null}
// See https://github.com/peopledatalabs/peopledatalabs-js
import PDLJS from 'peopledatalabs';
const PDLJSClient = new PDLJS({ apiKey: "YOUR API KEY" });
const esQuery = {
query: {
bool: {
must:[
{term: {location_country: "mexico"}},
{term: {job_title_role: "health"}},
]
}
}
}
const params = {
searchQuery: esQuery,
size: 10,
pretty: true,
dataset: "phone" // Use search against all PDL records with a phone number
}
PDLJSClient.person.search.elastic(params).then((data) => {
console.log(`Number of records found: ${data['total']}`);
}).catch((error) => {
console.log(error);
});
```
```javascript JavaScript (SQL) theme={null}
// See https://github.com/peopledatalabs/peopledatalabs-js
import PDLJS from 'peopledatalabs';
const PDLJSClient = new PDLJS({ apiKey: "YOUR API KEY" });
const sqlQuery = `SELECT * FROM person
WHERE location_country='mexico'
AND job_title_role='health';`
const params = {
searchQuery: sqlQuery,
size: 10,
pretty: true,
dataset: "phone" // Use search against all PDL records with a phone number
}
PDLJSClient.person.search.sql(params).then((data) => {
console.log(`Number of records found: ${data['total']}`);
}).catch((error) => {
console.log(error);
});
```
```ruby Ruby (Elasticsearch) theme={null}
require 'json'
# See https://github.com/peopledatalabs/peopledatalabs-ruby
require 'peopledatalabs'
Peopledatalabs.api_key = 'YOUR API KEY'
ES_QUERY = {
"query": {
"bool": {
"must": [
{"term": {"location_country": "mexico"}},
{"term": {"job_title_role": "health"}}
]
}
}
}
response = Peopledatalabs::Search.people(searchType: 'elastic', query: ES_QUERY, size: 10, pretty: true, 'dataset': 'phone')
if response['status'] == 200
data = response['data']
File.open("my_pdl_search.jsonl", "w") do |out|
data.each { |record| out.write(JSON.dump(record) + "\n") }
end
puts "successfully grabbed #{data.length()} records from pdl"
puts "#{response['total']} total pdl records exist matching this query"
else
puts "NOTE. The carrier pigeons lost motivation in flight. See error and try again."
puts "Error: #{response}"
end
```
```ruby Ruby (SQL) theme={null}
require 'json'
# See https://github.com/peopledatalabs/peopledatalabs-ruby
require 'peopledatalabs'
Peopledatalabs.api_key = 'YOUR API KEY'
SQL_QUERY = \
"""
SELECT * FROM person
WHERE location_country='mexico'
AND job_title_role='health'
AND phone_numbers IS NOT NULL;
"""
response = Peopledatalabs::Search.people(searchType: 'sql', query: SQL_QUERY, size: 10, pretty: true, 'dataset': 'phone')
if response['status'] == 200
data = response['data']
File.open("my_pdl_search.jsonl", "w") do |out|
data.each { |record| out.write(JSON.dump(record) + "\n") }
end
puts "successfully grabbed #{data.length()} records from pdl"
puts "#{response['total']} total pdl records exist matching this query"
else
puts "NOTE. The carrier pigeons lost motivation in flight. See error and try again."
puts "Error: #{response}"
end
```
```python Python3 (Elasticsearch) expandable theme={null}
import requests, json
API_KEY = # YOUR API KEY
PDL_URL = "https://api.peopledatalabs.com/v5/person/search"
H = {
'Content-Type': "application/json",
'X-api-key': API_KEY
}
ES_QUERY = {
"query": {
"bool": {
"must": [
{"term": {"location_country": "mexico"}},
{"term": {"job_title_role": "health"}}
]
}
}
}
P = {
'query': json.dumps(ES_QUERY),
'size': 10,
'pretty': True,
'dataset': "phone" # Use search against all PDL records with a phone number
}
response = requests.get(
PDL_URL,
headers=H,
params=P
).json()
if response["status"] == 200:
data = response['data']
with open("my_pdl_search.jsonl", "w") as out:
for record in data:
out.write(json.dumps(record) + "\n")
print(f"successfully grabbed {len(data)} records from pdl")
print(f"{response['total']} total pdl records exist matching this query")
else:
print("NOTE. The carrier pigeons lost motivation in flight. See error and try again.")
print("Error:", response)
```
```python Python3 (SQL) expandable theme={null}
import requests, json
API_KEY = # YOUR API KEY
PDL_URL = "https://api.peopledatalabs.com/v5/person/search"
H = {
'Content-Type': "application/json",
'X-api-key': API_KEY
}
SQL_QUERY = \
"""
SELECT * FROM person
WHERE location_country='mexico'
AND job_title_role='health'
AND phone_numbers IS NOT NULL;
"""
P = {
'sql': SQL_QUERY,
'size': 10,
'pretty': True,
'dataset': "phone" # Use all to search against all PDL records with a phone number
}
response = requests.get(
PDL_URL,
headers=H,
params=P
).json()
if response["status"] == 200:
data = response['data']
with open("my_pdl_search.jsonl", "w") as out:
for record in data:
out.write(json.dumps(record) + "\n")
print(f"successfully grabbed {len(data)} records from pdl")
print(f"{response['total']} total pdl records exist matching this query")
else:
print("NOTE. The carrier pigeons lost motivation in flight. See error and try again.")
print("error:", response)
```
### Excluding Datasets
*"I want to run a simple query against all PDL datasets except the [email](/docs/email-stats) and [phone](/docs/datasets#phone-slice) datasets"*
```python Python3 SDK (Elasticsearch) expandable theme={null}
import json
# See https://github.com/peopledatalabs/peopledatalabs-python
from peopledatalabs import PDLPY
# Create a client, specifying an API key
client = PDLPY(
api_key="YOUR API KEY",
)
ES_QUERY = {
"query": {
"bool": {
"must": [
{"term": {"location_country": "mexico"}},
{"term": {"job_title_role": "health"}}
]
}
}
}
P = {
'query': ES_QUERY,
'size': 10,
'pretty': True,
'dataset': "-email,phone" # Use search against all PDL datasets EXCEPT the email and phone slices
}
response = client.person.search(**P).json()
if response["status"] == 200:
data = response['data']
with open("my_pdl_search.jsonl", "w") as out:
for record in data:
out.write(json.dumps(record) + "\n")
print(f"successfully grabbed {len(data)} records from pdl")
print(f"{response['total']} total pdl records exist matching this query")
else:
print("NOTE. The carrier pigeons lost motivation in flight. See error and try again.")
print("Error:", response)
```
```javascript JavaScript (Elasticsearch) theme={null}
// See https://github.com/peopledatalabs/peopledatalabs-js
import PDLJS from 'peopledatalabs';
const PDLJSClient = new PDLJS({ apiKey: "YOUR API KEY" });
const esQuery = {
query: {
bool: {
must:[
{term: {location_country: "mexico"}},
{term: {job_title_role: "health"}},
]
}
}
}
const params = {
searchQuery: esQuery,
size: 10,
pretty: true,
dataset: "-email,phone" // Use search against all PDL datasets EXCEPT the email and phone slices
}
PDLJSClient.person.search.elastic(params).then((data) => {
console.log(`Number of records found: ${data['total']}`);
}).catch((error) => {
console.log(error);
});
```
```ruby Ruby (Elasticsearch) theme={null}
require 'json'
# See https://github.com/peopledatalabs/peopledatalabs-ruby
require 'peopledatalabs'
Peopledatalabs.api_key = 'YOUR API KEY'
ES_QUERY = {
"query": {
"bool": {
"must": [
{"term": {"location_country": "mexico"}},
{"term": {"job_title_role": "health"}}
]
}
}
}
response = Peopledatalabs::Search.people(searchType: 'elastic', query: ES_QUERY, size: 10, pretty: true, 'dataset': '-email,phone')
if response['status'] == 200
data = response['data']
File.open("my_pdl_search.jsonl", "w") do |out|
data.each { |record| out.write(JSON.dump(record) + "\n") }
end
puts "successfully grabbed #{data.length()} records from pdl"
puts "#{response['total']} total pdl records exist matching this query"
else
puts "NOTE. The carrier pigeons lost motivation in flight. See error and try again."
puts "Error: #{response}"
end
```
```python Python3 (Elasticsearch) expandable theme={null}
import requests, json
API_KEY = # YOUR API KEY
PDL_URL = "https://api.peopledatalabs.com/v5/person/search"
H = {
'Content-Type': "application/json",
'X-api-key': API_KEY
}
ES_QUERY = {
"query": {
"bool": {
"must": [
{"term": {"location_country": "mexico"}},
{"term": {"job_title_role": "health"}}
]
}
}
}
P = {
'query': json.dumps(ES_QUERY),
'size': 10,
'pretty': True,
'dataset': "-email,phone" # Use search against all PDL datasets EXCEPT the email and phone slices
}
response = requests.get(
PDL_URL,
headers=H,
params=P
).json()
if response["status"] == 200:
data = response['data']
with open("my_pdl_search.jsonl", "w") as out:
for record in data:
out.write(json.dumps(record) + "\n")
print(f"successfully grabbed {len(data)} records from pdl")
print(f"{response['total']} total pdl records exist matching this query")
else:
print("NOTE. The carrier pigeons lost motivation in flight. See error and try again.")
print("Error:", response)
```
## Bulk Retrieval
*"I want to pull all the current employees at Amazon and save their profiles to a csv file."*
**High Credit Usage Code Below**
The code example below illustrates pulling all the employee profiles in a large company, and is meant primarily for demonstrating the use of the `scroll_token` parameter when retrieving large amounts of records. As a result this code mostly illustrative meaning it can use up a lot of credits, and doesn't have any error handling. The `MAX_NUM_RECORDS_LIMIT` parameter in the example below sets the maximum number of profiles (e.g. credits) that will be pulled, so please set that accordingly when testing this example.
```python Python3 SDK (Elasticsearch) expandable theme={null}
import json, time, csv
# See https://github.com/peopledatalabs/peopledatalabs-python
from peopledatalabs import PDLPY
# Limit the number of records to pull (to prevent accidentally using up
# more credits than expected when testing out this code).
MAX_NUM_RECORDS_LIMIT = 150 # The maximum number of records to retrieve
USE_MAX_NUM_RECORDS_LIMIT = True # Set to False to pull all available records
# Create a client, specifying an API key
client = PDLPY(
api_key="YOUR API KEY",
)
ES_QUERY = {
"query": {
"bool": {
"must": [
{"term": {"job_company_id": "amazon"}}
]
}
}
}
P = {
'query': ES_QUERY,
'size': 100,
'pretty': True
}
# Pull all results in multiple batches
batch = 1
all_records = []
start_time = time.time()
found_all_records = False
continue_scrolling = True
while continue_scrolling and not found_all_records:
# Check if we reached the maximum number of records we wanted to pull
if USE_MAX_NUM_RECORDS_LIMIT:
num_records_to_request = MAX_NUM_RECORDS_LIMIT - len(all_records)
P['size'] = max(0, min(100, num_records_to_request))
if num_records_to_request == 0:
print(f"Stopping - reached maximum number of records to pull "
f"[MAX_NUM_RECORDS_LIMIT = {MAX_NUM_RECORDS_LIMIT}]")
break
# Send Response
response = client.person.search(**P).json()
# Check response status code:
if response['status'] == 200:
all_records.extend(response['data'])
print(f"Retrieved {len(response['data'])} records in batch {batch} "
f"- {response['total'] - len(all_records)} records remaining")
else:
print(f"Error retrieving some records:\n\t"
f"[{response['status']} - {response['error']['type']}] "
f"{response['error']['message']}")
# Get scroll_token from response
if 'scroll_token' in response:
P['scroll_token'] = response['scroll_token']
else:
continue_scrolling = False
print(f"Unable to continue scrolling")
batch += 1
found_all_records = (len(all_records) == response['total'])
time.sleep(6) # avoid hitting rate limit thresholds
end_time = time.time()
runtime = end_time - start_time
print(f"Successfully recovered {len(all_records)} profiles in "
f"{batch} batches [{round(runtime, 2)} seconds]")
# Save profiles to csv (utility function)
def save_profiles_to_csv(profiles, filename, fields=[], delim=','):
# Define header fields
if fields == [] and len(profiles) > 0:
fields = profiles[0].keys()
# Write csv file
with open(filename, 'w') as csvfile:
writer = csv.writer(csvfile, delimiter=delim)
# Write Header:
writer.writerow(fields)
# Write Body:
count = 0
for profile in profiles:
writer.writerow([ profile[field] for field in fields ])
count += 1
print(f"Wrote {count} lines to: '{filename}'")
# Use utility function to save profiles to csv
csv_header_fields = ['work_email', 'full_name', "linkedin_url",
'job_title', 'job_company_name']
csv_filename = "all_employee_profiles.csv"
save_profiles_to_csv(all_records, csv_filename, csv_header_fields)
```
```python Python3 SDK (SQL) expandable theme={null}
import json, time, csv
# See https://github.com/peopledatalabs/peopledatalabs-python
from peopledatalabs import PDLPY
# Limit the number of records to pull (to prevent accidentally using up
# more credits than expected when testing out this code).
MAX_NUM_RECORDS_LIMIT = 150 # The maximum number of records to retrieve
USE_MAX_NUM_RECORDS_LIMIT = True # Set to False to pull all available records
# Create a client, specifying an API key
client = PDLPY(
api_key="YOUR API KEY",
)
SQL_QUERY = \
"""
SELECT * FROM person
WHERE job_company_id='amazon';
"""
P = {
'sql': SQL_QUERY,
'size': 100,
'pretty': True
}
# Pull all results in multiple batches
batch = 1
all_records = []
start_time = time.time()
found_all_records = False
continue_scrolling = True
while continue_scrolling and not found_all_records:
# Check if we reached the maximum number of records we wanted to pull
if USE_MAX_NUM_RECORDS_LIMIT:
num_records_to_request = MAX_NUM_RECORDS_LIMIT - len(all_records)
P['size'] = max(0, min(100, num_records_to_request))
if num_records_to_request == 0:
print(f"Stopping - reached maximum number of records to pull "
f"[MAX_NUM_RECORDS_LIMIT = {MAX_NUM_RECORDS_LIMIT}]")
break
# Send Response
response = client.person.search(**P).json()
# Check response status code:
if response['status'] == 200:
all_records.extend(response['data'])
print(f"Retrieved {len(response['data'])} records in batch {batch} "
f"- {response['total'] - len(all_records)} records remaining")
else:
print(f"Error retrieving some records:\n\t"
f"[{response['status']} - {response['error']['type']}] "
f"{response['error']['message']}")
# Get scroll_token from response
if 'scroll_token' in response:
P['scroll_token'] = response['scroll_token']
else:
continue_scrolling = False
print(f"Unable to continue scrolling")
batch += 1
found_all_records = (len(all_records) == response['total'])
time.sleep(6) # avoid hitting rate limit thresholds
end_time = time.time()
runtime = end_time - start_time
print(f"Successfully recovered {len(all_records)} profiles in "
f"{batch} batches [{round(runtime, 2)} seconds]")
# Save profiles to csv (utility function)
def save_profiles_to_csv(profiles, filename, fields=[], delim=','):
# Define header fields
if fields == [] and len(profiles) > 0:
fields = profiles[0].keys()
# Write csv file
with open(filename, 'w') as csvfile:
writer = csv.writer(csvfile, delimiter=delim)
# Write Header:
writer.writerow(fields)
# Write Body:
count = 0
for profile in profiles:
writer.writerow([ profile[field] for field in fields ])
count += 1
print(f"Wrote {count} lines to: '{filename}'")
# Use utility function to save profiles to csv
csv_header_fields = ['work_email', 'full_name', "linkedin_url",
'job_title', 'job_company_name']
csv_filename = "all_employee_profiles.csv"
save_profiles_to_csv(all_records, csv_filename, csv_header_fields)
```
```javascript JavaScript (Elasticsearch) expandable theme={null}
// See https://github.com/peopledatalabs/peopledatalabs-js
import PDLJS from 'peopledatalabs';
// See https://www.npmjs.com/package/csv-writer
import * as csvwriter from 'csv-writer';
const PDLJSClient = new PDLJS({ apiKey: "YOUR API KEY" });
// Limit the number of records to pull (to prevent accidentally using up
// more credits than expected when testing out this code).
const maxNumRecordsLimit = 150; // The maximum number of records to retrieve
const useMaxNumRecordsLimit = true; // Set to false to pull all available records
const esQuery = {
query: {
bool: {
must:[
{term: {job_company_id: "amazon"}},
]
}
}
}
var params = {
searchQuery: esQuery,
size: 100,
scroll_token: null,
pretty: true
}
// Pull all results in multiple batches
var batch = 1;
var allRecords = [];
var startTime = Date.now();
var foundAllRecords = false;
var continueScrolling = true;
var numRetrieved = 0;
var paramQueue = [];
var scrollToken = null;
var numRecordsToRequest = 100;
while (numRecordsToRequest > 0) {
// Check if we reached the maximum number of records we wanted to pull
if (useMaxNumRecordsLimit) {
numRecordsToRequest = maxNumRecordsLimit - numRetrieved;
params.size = Math.max(0, Math.min(100, numRecordsToRequest));
numRetrieved += params.size;
// Add batch to the parameter queue
if (params.size > 0) {
paramQueue.push(JSON.parse(JSON.stringify(params)));
}
} else {
break;
}
}
// Run initial batch
runBatch();
function runBatch() {
// Get the parameters for the batch
let currParams = useMaxNumRecordsLimit ? paramQueue[batch-1] : params;
// Set the scroll_token from the previous batch
currParams.scroll_token = scrollToken;
batch++;
PDLJSClient.person.search.elastic(currParams).then((data) => {
Array.prototype.push.apply(allRecords, data.data);
// Get the scroll_token
if (data['scroll_token']) {
scrollToken = data['scroll_token'];
} else {
continueScrolling = false;
console.log("Unable to continue scrolling");
}
foundAllRecords = (allRecords.length == data['total']);
console.log("Retrieved " + data.data.length + " records in batch " + (batch-1) +
" - " + (data['total'] - allRecords.length) + " records remaining");
// Run next batch, if any
if (!foundAllRecords && (batch <= paramQueue.length || !useMaxNumRecordsLimit)) {
runBatch();
} else {
console.log("Stopping - reached maximum number of records to pull [maxNumRecordsLimit = " +
maxNumRecordsLimit + "]");
let endTime = Date.now();
let runTime = endTime - startTime;
console.log ("Successfully recovered " + allRecords.length + " profiles in " +
(batch-1) + " batches [" + Math.round(runTime/1000) + " seconds]");
// Output profiles to CSV
let csvHeaderFields = [
{id: "work_email", title: "work_email"},
{id: "full_name", title: "full_name"},
{id: "linkedin_url", title: "linkedin_url"},
{id: "job_title", title: "job_title"},
{id: "job_company_name", title: "job_company_name"}
];
let csvFilename = "all_employee_profiles.csv";
saveProfilesToCSV(allRecords, csvFilename, csvHeaderFields);
}
}).catch((error) => {
console.log(error);
});
}
// Write out CSV file using csv-writer (https://www.npmjs.com/package/csv-writer)
// $ npm i -s csv-writer
function saveProfilesToCSV(profiles, filename, fields) {
const createCsvWriter = csvwriter.createObjectCsvWriter;
const csvWriter = createCsvWriter({
path: filename,
header: fields
});
let data = [];
for (let i = 0; i < profiles.length; i++) {
let record = profiles[i];
data[i] = {};
for (let field in fields) {
data[i][fields[field].id] = record[fields[field].id];
}
}
csvWriter
.writeRecords(data)
.then(()=> console.log('The CSV file was written successfully'));
}
```
```javascript JavaScript (SQL) expandable theme={null}
// See https://github.com/peopledatalabs/peopledatalabs-js
import PDLJS from 'peopledatalabs';
// See https://www.npmjs.com/package/csv-writer
import * as csvwriter from 'csv-writer';
const PDLJSClient = new PDLJS({ apiKey: "YOUR API KEY" });
// Limit the number of records to pull (to prevent accidentally using up
// more credits than expected when testing out this code).
const maxNumRecordsLimit = 150; // The maximum number of records to retrieve
const useMaxNumRecordsLimit = true; // Set to false to pull all available records
const sqlQuery = `SELECT * FROM person
WHERE job_company_id='amazon';`;
var params = {
searchQuery: sqlQuery,
size: 100,
scroll_token: null,
pretty: true
}
// Pull all results in multiple batches
var batch = 1;
var allRecords = [];
var startTime = Date.now();
var foundAllRecords = false;
var continueScrolling = true;
var numRetrieved = 0;
var paramQueue = [];
var scrollToken = null;
var numRecordsToRequest = 100;
while (numRecordsToRequest > 0) {
// Check if we reached the maximum number of records we wanted to pull
if (useMaxNumRecordsLimit) {
numRecordsToRequest = maxNumRecordsLimit - numRetrieved;
params.size = Math.max(0, Math.min(100, numRecordsToRequest));
numRetrieved += params.size;
// Add batch to the parameter queue
if (params.size > 0) {
paramQueue.push(JSON.parse(JSON.stringify(params)));
}
} else {
break;
}
}
// Run initial batch
runBatch();
function runBatch() {
// Get the parameters for the batch
let currParams = useMaxNumRecordsLimit ? paramQueue[batch-1] : params;
// Set the scroll_token from the previous batch
currParams.scroll_token = scrollToken;
batch++;
PDLJSClient.person.search.sql(currParams).then((data) => {
Array.prototype.push.apply(allRecords, data.data);
// Get the scroll_token
if (data['scroll_token']) {
scrollToken = data['scroll_token'];
} else {
continueScrolling = false;
console.log("Unable to continue scrolling");
}
foundAllRecords = (allRecords.length == data['total']);
console.log("Retrieved " + data.data.length + " records in batch " + (batch-1) +
" - " + (data['total'] - allRecords.length) + " records remaining");
// Run next batch, if any
if (!foundAllRecords && (batch <= paramQueue.length || !useMaxNumRecordsLimit)) {
runBatch();
} else {
console.log("Stopping - reached maximum number of records to pull [maxNumRecordsLimit = " +
maxNumRecordsLimit + "]");
let endTime = Date.now();
let runTime = endTime - startTime;
console.log ("Successfully recovered " + allRecords.length + " profiles in " +
(batch-1) + " batches [" + Math.round(runTime/1000) + " seconds]");
// Output profiles to CSV
let csvHeaderFields = [
{id: "work_email", title: "work_email"},
{id: "full_name", title: "full_name"},
{id: "linkedin_url", title: "linkedin_url"},
{id: "job_title", title: "job_title"},
{id: "job_company_name", title: "job_company_name"}
];
let csvFilename = "all_employee_profiles.csv";
saveProfilesToCSV(allRecords, csvFilename, csvHeaderFields);
}
}).catch((error) => {
console.log(error);
});
}
// Write out CSV file using csv-writer (https://www.npmjs.com/package/csv-writer)
// $ npm i -s csv-writer
function saveProfilesToCSV(profiles, filename, fields) {
const createCsvWriter = csvwriter.createObjectCsvWriter;
const csvWriter = createCsvWriter({
path: filename,
header: fields
});
let data = [];
for (let i = 0; i < profiles.length; i++) {
let record = profiles[i];
data[i] = {};
for (let field in fields) {
data[i][fields[field].id] = record[fields[field].id];
}
}
csvWriter
.writeRecords(data)
.then(()=> console.log('The CSV file was written successfully'));
}
```
```ruby Ruby (Elasticsearch) expandable theme={null}
require 'json'
require 'csv'
# See https://github.com/peopledatalabs/peopledatalabs-ruby
require 'peopledatalabs'
Peopledatalabs.api_key = 'YOUR API KEY'
# Limit the number of records to pull (to prevent accidentally using up
# more credits than expected when testing out this code).
MAX_NUM_RECORDS_LIMIT = 150 # The maximum number of records to retrieve
USE_MAX_NUM_RECORDS_LIMIT = true # Set to false to pull all available records
ES_QUERY = {
"query": {
"bool": {
"must": [
{"term": {"job_company_id": "amazon"}}
]
}
}
}
# Pull all results in multiple batches
batch = 1
all_records = []
start_time = Time.now
found_all_records = false
continue_scrolling = true
scroll_token = {}
while continue_scrolling && !found_all_records do
# Check if we reached the maximum number of records we wanted to pull
if USE_MAX_NUM_RECORDS_LIMIT
num_records_to_request = MAX_NUM_RECORDS_LIMIT - all_records.length()
size = [0, [100, num_records_to_request].min].max
if num_records_to_request == 0
puts "Stopping - reached maximum number of records to pull "
puts "[MAX_NUM_RECORDS_LIMIT = #{MAX_NUM_RECORDS_LIMIT}]"
break
end
end
# Send Response
response = Peopledatalabs::Search.people(searchType: 'elastic', query: ES_QUERY, size: size, scroll_token: scroll_token, pretty: true)
# Check response status code:
if response['status'] == 200
all_records += response['data']
puts "Retrieved #{response['data'].length()} records in batch #{batch} "
puts "- #{response['total'] - all_records.length()} records remaining"
else
puts "Error retrieving some records:\n\t"
puts "[#{response['status']} - #{response['error']['type']}] "
puts response['error']['message']
end
# Get scroll_token from response
if response.key?('scroll_token')
scroll_token = response['scroll_token']
else
continue_scrolling = false
puts "Unable to continue scrolling"
end
batch += 1
found_all_records = (all_records.length() == response['total'])
sleep(6) # avoid hitting rate limit thresholds
end
end_time = Time.now
runtime = end_time - start_time
puts "Successfully recovered #{all_records.length()} profiles in "
puts "#{batch} batches [#{runtime.round(2)} seconds]"
# Save profiles to csv (utility function)
def save_profiles_to_csv(profiles, filename, fields=[], delim=',')
# Define header fields
if fields == [] && profiles.length() > 0
fields = profiles[0].keys
end
count = 0
# Write csv file
CSV.open(filename, 'w') do |writer|
# Write Header:
writer << fields
# Write Body:
profiles.each do |profile|
record = []
fields.each do |field|
record << profile[field]
count += 1
end
writer << record
end
end
puts "Wrote #{count} lines to: '#{filename}'"
end
# Use utility function to save profiles to csv
csv_header_fields = ['work_email', 'full_name', "linkedin_url",
'job_title', 'job_company_name']
csv_filename = "all_company_profiles.csv"
save_profiles_to_csv(all_records, csv_filename, csv_header_fields)
```
```ruby Ruby (SQL) expandable theme={null}
require 'json'
require 'csv'
# See https://github.com/peopledatalabs/peopledatalabs-ruby
require 'peopledatalabs'
Peopledatalabs.api_key = 'YOUR API KEY'
# Limit the number of records to pull (to prevent accidentally using up
# more credits than expected when testing out this code).
MAX_NUM_RECORDS_LIMIT = 150 # The maximum number of records to retrieve
USE_MAX_NUM_RECORDS_LIMIT = true # Set to false to pull all available records
SQL_QUERY = \
"""
SELECT * FROM person
WHERE job_company_id='amazon';
"""
# Pull all results in multiple batches
batch = 1
all_records = []
start_time = Time.now
found_all_records = false
continue_scrolling = true
scroll_token = {}
while continue_scrolling && !found_all_records do
# Check if we reached the maximum number of records we wanted to pull
if USE_MAX_NUM_RECORDS_LIMIT
num_records_to_request = MAX_NUM_RECORDS_LIMIT - all_records.length()
size = [0, [100, num_records_to_request].min].max
if num_records_to_request == 0
puts "Stopping - reached maximum number of records to pull "
puts "[MAX_NUM_RECORDS_LIMIT = #{MAX_NUM_RECORDS_LIMIT}]"
break
end
end
# Send Response
response = Peopledatalabs::Search.people(searchType: 'sql', query: SQL_QUERY, size: size, scroll_token: scroll_token, pretty: true)
# Check response status code:
if response['status'] == 200
all_records += response['data']
puts "Retrieved #{response['data'].length()} records in batch #{batch} "
puts "- #{response['total'] - all_records.length()} records remaining"
else
puts "Error retrieving some records:\n\t"
puts "[#{response['status']} - #{response['error']['type']}] "
puts response['error']['message']
end
# Get scroll_token from response
if response.key?('scroll_token')
scroll_token = response['scroll_token']
else
continue_scrolling = false
puts "Unable to continue scrolling"
end
batch += 1
found_all_records = (all_records.length() == response['total'])
sleep(6) # avoid hitting rate limit thresholds
end
end_time = Time.now
runtime = end_time - start_time
puts "Successfully recovered #{all_records.length()} profiles in "
puts "#{batch} batches [#{runtime.round(2)} seconds]"
# Save profiles to csv (utility function)
def save_profiles_to_csv(profiles, filename, fields=[], delim=',')
# Define header fields
if fields == [] && profiles.length() > 0
fields = profiles[0].keys
end
count = 0
# Write csv file
CSV.open(filename, 'w') do |writer|
# Write Header:
writer << fields
# Write Body:
profiles.each do |profile|
record = []
fields.each do |field|
record << profile[field]
count += 1
end
writer << record
end
end
puts "Wrote #{count} lines to: '#{filename}'"
end
# Use utility function to save profiles to csv
csv_header_fields = ['work_email', 'full_name', "linkedin_url",
'job_title', 'job_company_name']
csv_filename = "all_company_profiles.csv"
save_profiles_to_csv(all_records, csv_filename, csv_header_fields)
```
```python Python3 (SQL) expandable theme={null}
import requests, json, time, csv
API_KEY = # ENTER YOUR API KEY
# Limit the number of records to pull (to prevent accidentally using up
# more credits than expected when testing out this code).
MAX_NUM_RECORDS_LIMIT = 150 # The maximum number of records to retrieve
USE_MAX_NUM_RECORDS_LIMIT = True # Set to False to pull all available records
PDL_URL = "https://api.peopledatalabs.com/v5/person/search"
H = {
'Content-Type': "application/json",
'X-api-key': API_KEY
}
SQL_QUERY = \
"""
SELECT * FROM person
WHERE job_company_id='amazon';
"""
P = {
'sql': SQL_QUERY,
'size': 100,
'pretty': True
}
# Pull all results in multiple batches
batch = 1
all_records = []
start_time = time.time()
found_all_records = False
continue_scrolling = True
while continue_scrolling and not found_all_records:
# Check if we reached the maximum number of records we wanted to pull
if USE_MAX_NUM_RECORDS_LIMIT:
num_records_to_request = MAX_NUM_RECORDS_LIMIT - len(all_records)
P['size'] = max(0, min(100, num_records_to_request))
if num_records_to_request == 0:
print(f"Stopping - reached maximum number of records to pull "
f"[MAX_NUM_RECORDS_LIMIT = {MAX_NUM_RECORDS_LIMIT}]")
break
# Send Response
response = requests.get(
PDL_URL,
headers=H,
params=P
).json()
# Check response status code:
if response['status'] == 200:
all_records.extend(response['data'])
print(f"Retrieved {len(response['data'])} records in batch {batch} "
f"- {response['total'] - len(all_records)} records remaining")
else:
print(f"Error retrieving some records:\n\t"
f"[{response['status']} - {response['error']['type']}] "
f"{response['error']['message']}")
# Get scroll_token from response
if 'scroll_token' in response:
P['scroll_token'] = response['scroll_token']
else:
continue_scrolling = False
print(f"Unable to continue scrolling")
batch += 1
found_all_records = (len(all_records) == response['total'])
time.sleep(6) # avoid hitting rate limit thresholds
end_time = time.time()
runtime = end_time - start_time
print(f"Successfully recovered {len(all_records)} profiles in "
f"{batch} batches [{round(runtime, 2)} seconds]")
# Save profiles to csv (utility function)
def save_profiles_to_csv(profiles, filename, fields=[], delim=','):
# Define header fields
if fields == [] and len(profiles) > 0:
fields = profiles[0].keys()
# Write csv file
with open(filename, 'w') as csvfile:
writer = csv.writer(csvfile, delimiter=delim)
# Write Header:
writer.writerow(fields)
# Write Body:
count = 0
for profile in profiles:
writer.writerow([ profile[field] for field in fields ])
count += 1
print(f"Wrote {count} lines to: '{filename}'")
# Use utility function to save profiles to csv
csv_header_fields = ['work_email', 'full_name', "linkedin_url",
'job_title', 'job_company_name']
csv_filename = "all_employee_profiles.csv"
save_profiles_to_csv(all_records, csv_filename, csv_header_fields)
```
```python Python3 (Elasticsearch) expandable theme={null}
import requests, json, time, csv
API_KEY = # ENTER YOUR API KEY
# Limit the number of records to pull (to prevent accidentally using up
# more credits than expected when testing out this code).
MAX_NUM_RECORDS_LIMIT = 150 # The maximum number of records to retrieve
USE_MAX_NUM_RECORDS_LIMIT = True # Set to False to pull all available records
PDL_URL = "https://api.peopledatalabs.com/v5/person/search"
H = {
'Content-Type': "application/json",
'X-api-key': API_KEY
}
ES_QUERY = {
"query": {
"bool": {
"must": [
{"term": {"job_company_id": "amazon"}}
]
}
}
}
P = {
'query': json.dumps(ES_QUERY),
'size': 100,
'pretty': True
}
# Pull all results in multiple batches
batch = 1
all_records = []
start_time = time.time()
found_all_records = False
continue_scrolling = True
while continue_scrolling and not found_all_records:
# Check if we reached the maximum number of records we wanted to pull
if USE_MAX_NUM_RECORDS_LIMIT:
num_records_to_request = MAX_NUM_RECORDS_LIMIT - len(all_records)
P['size'] = max(0, min(100, num_records_to_request))
if num_records_to_request == 0:
print(f"Stopping - reached maximum number of records to pull "
f"[MAX_NUM_RECORDS_LIMIT = {MAX_NUM_RECORDS_LIMIT}]")
break
# Send Response
response = requests.get(
PDL_URL,
headers=H,
params=P
).json()
# Check response status code:
if response['status'] == 200:
all_records.extend(response['data'])
print(f"Retrieved {len(response['data'])} records in batch {batch} "
f"- {response['total'] - len(all_records)} records remaining")
else:
print(f"Error retrieving some records:\n\t"
f"[{response['status']} - {response['error']['type']}] "
f"{response['error']['message']}")
# Get scroll_token from response
if 'scroll_token' in response:
P['scroll_token'] = response['scroll_token']
else:
continue_scrolling = False
print(f"Unable to continue scrolling")
batch += 1
found_all_records = (len(all_records) == response['total'])
time.sleep(6) # avoid hitting rate limit thresholds
end_time = time.time()
runtime = end_time - start_time
print(f"Successfully recovered {len(all_records)} profiles in "
f"{batch} batches [{round(runtime, 2)} seconds]")
# Save profiles to csv (utility function)
def save_profiles_to_csv(profiles, filename, fields=[], delim=','):
# Define header fields
if fields == [] and len(profiles) > 0:
fields = profiles[0].keys()
# Write csv file
with open(filename, 'w') as csvfile:
writer = csv.writer(csvfile, delimiter=delim)
# Write Header:
writer.writerow(fields)
# Write Body:
count = 0
for profile in profiles:
writer.writerow([ profile[field] for field in fields ])
count += 1
print(f"Wrote {count} lines to: '{filename}'")
# Use utility function to save profiles to csv
csv_header_fields = ['work_email', 'full_name', "linkedin_url",
'job_title', 'job_company_name']
csv_filename = "all_employee_profiles.csv"
save_profiles_to_csv(all_records, csv_filename, csv_header_fields)
```
## Sales Prospecting
*"I want to email engineering leaders at the following companies to reach out about my product: stripe.com, plaid.com, xignite.com, square.com."*
```python Python3 SDK (Elasticsearch) expandable theme={null}
import json
# See https://github.com/peopledatalabs/peopledatalabs-python
from peopledatalabs import PDLPY
DESIRED_COMPANY_DOMAINS = [
'stripe.com', 'plaid.com', 'xignite.com', 'square.com'
]
# Create a client, specifying an API key
client = PDLPY(
api_key="YOUR API KEY",
)
ES_QUERY = {
"query": {
"bool": {
"must": [
{"terms": {"job_company_website": DESIRED_COMPANY_DOMAINS}},
{"term": {"job_title_role": "engineering"}},
{"terms": {"job_title_levels": ["vp", "director", "manager"]}},
{"exists": {"field": "work_email"}}
]
}
}
}
P = {
'query': ES_QUERY,
'size': 100
}
response = client.person.search(**P).json()
if response["status"] == 200:
for record in response['data']:
# bring in leads and make $$$
print(
record['work_email'],
record['full_name'],
record['job_title'],
record['job_company_name']
)
print(f"successfully grabbed {len(response['data'])} records from pdl")
print(f"{response['total']} total pdl records exist matching this query")
else:
print("NOTE. The eager beaver was not so eager. See error and try again.")
print("error:", response)
```
```python Python3 SDK (SQL) expandable theme={null}
import json
# See https://github.com/peopledatalabs/peopledatalabs-python
from peopledatalabs import PDLPY
DESIRED_COMPANY_DOMAINS = [
'stripe.com', 'plaid.com', 'xignite.com', 'square.com'
]
COMPANY_DOMAINS_STRING_REP = ", ".join(
(f"'{site}'" for site in DESIRED_COMPANY_DOMAINS)
)
# Create a client, specifying an API key
client = PDLPY(
api_key="YOUR API KEY",
)
SQL_QUERY = \
f"""
SELECT * FROM person
WHERE job_company_website IN ({COMPANY_DOMAINS_STRING_REP})
AND job_title_role='engineering'
AND job_title_levels IN ('vp', 'director', 'manager')
AND work_email IS NOT NULL;
"""
P = {
'sql': SQL_QUERY,
'size': 100
}
response = client.person.search(**P).json()
if response["status"] == 200:
for record in response['data']:
# bring in leads and make $$$
print(
record['work_email'],
record['full_name'],
record['job_title'],
record['job_company_name']
)
print(f"successfully grabbed {len(response['data'])} records from pdl")
print(f"{response['total']} total pdl records exist matching this query")
else:
print("NOTE. The eager beaver was not so eager. See error and try again.")
print("error:", response)
```
```javascript JavaScript (Elasticsearch) expandable theme={null}
// See https://github.com/peopledatalabs/peopledatalabs-js
import PDLJS from 'peopledatalabs';
const PDLJSClient = new PDLJS({ apiKey: "YOUR API KEY" });
const desiredCompanyDomains = [
"stripe.com", "plaid.com", "xignite.com", "square.com"
];
const esQuery = {
query: {
bool: {
must:[
{terms: {job_company_website: desiredCompanyDomains}},
{term: {job_title_role: "engineering"}},
{terms: {job_title_levels: ["vp", "director", "manager"]}},
{exists: {field: "work_email"}}
]
}
}
}
const params = {
searchQuery: esQuery,
size: 100
}
PDLJSClient.person.search.elastic(params).then((data) => {
for (let i = 0; i < data.data.length; i++) {
console.log(data.data[i].work_email + ' ' + data.data[i].full_name + ' ' +
data.data[i].job_title + ' ' + data.data[i].job_company_name);
}
console.log(`Number of records found: ${data['total']}`);
}).catch((error) => {
console.log(error);
});
```
```javascript JavaScript (SQL) theme={null}
// See https://github.com/peopledatalabs/peopledatalabs-js
import PDLJS from 'peopledatalabs';
const PDLJSClient = new PDLJS({ apiKey: "YOUR API KEY" });
const desiredCompanyDomains = [
"stripe.com", "plaid.com", "xignite.com", "square.com"
];
var companyStringRep = "'" + desiredCompanyDomains.join("', '") + "'";
const sqlQuery = `SELECT * FROM person
WHERE job_company_website IN (` + companyStringRep + `)
AND job_title_role='engineering'
AND job_title_levels IN ('vp', 'director', 'manager')
AND work_email IS NOT NULL;`
const params = {
searchQuery: sqlQuery,
size: 100
}
PDLJSClient.person.search.sql(params).then((data) => {
for (let i = 0; i < data.data.length; i++) {
console.log(data.data[i].work_email + ' ' + data.data[i].full_name + ' ' +
data.data[i].job_title + ' ' + data.data[i].job_company_name);
}
console.log(`Number of records found: ${data['total']}`);
}).catch((error) => {
console.log(error);
});
```
```ruby Ruby (Elasticsearch) expandable theme={null}
require 'json'
# See https://github.com/peopledatalabs/peopledatalabs-ruby
require 'peopledatalabs'
Peopledatalabs.api_key = 'YOUR API KEY'
DESIRED_COMPANY_DOMAINS = [
'stripe.com', 'plaid.com', 'xignite.com', 'square.com'
]
ES_QUERY = {
"query": {
"bool": {
"must": [
{"terms": {"job_company_website": DESIRED_COMPANY_DOMAINS}},
{"term": {"job_title_role": "engineering"}},
{"terms": {"job_title_levels": ["vp", "director", "manager"]}},
{"exists": {"field": "work_email"}}
]
}
}
}
response = Peopledatalabs::Search.people(searchType: 'elastic', query: ES_QUERY, size: 100)
if response['status'] == 200
data = response['data']
data.each do |record|
# bring in leads and make $$$
puts "#{record['work_email']} \
#{record['full_name']} \
#{record['job_title']} \
#{record['job_company_name']}"
end
puts "successfully grabbed #{data.length()} records from pdl"
puts "#{response['total']} total pdl records exist matching this query"
else
puts "NOTE. The eager beaver was not so eager. See error and try again."
puts "Error: #{response}"
end
```
```ruby Ruby (SQL) expandable theme={null}
require 'json'
# See https://github.com/peopledatalabs/peopledatalabs-ruby
require 'peopledatalabs'
Peopledatalabs.api_key = 'YOUR API KEY'
COMPANY_DOMAINS_STRING_REP = "'stripe.com', 'plaid.com', 'xignite.com', 'square.com'"
SQL_QUERY = \
"""
SELECT * FROM person
WHERE job_company_website IN (#{COMPANY_DOMAINS_STRING_REP})
AND job_title_role='engineering'
AND job_title_levels IN ('vp', 'director', 'manager')
AND work_email IS NOT NULL;
"""
response = Peopledatalabs::Search.people(searchType: 'sql', query: SQL_QUERY, size: 100)
if response['status'] == 200
data = response['data']
data.each do |record|
# bring in leads and make $$$
puts "#{record['work_email']} \
#{record['full_name']} \
#{record['job_title']} \
#{record['job_company_name']}"
end
puts "successfully grabbed #{data.length()} records from pdl"
puts "#{response['total']} total pdl records exist matching this query"
else
puts "NOTE. The eager beaver was not so eager. See error and try again."
puts "Error: #{response}"
end
```
```python Python3 (Elasticsearch) expandable theme={null}
import requests, json
API_KEY = #YOUR API KEY
PDL_URL = "https://api.peopledatalabs.com/v5/person/search"
H = {
'Content-Type': "application/json",
'X-api-key': API_KEY
}
DESIRED_COMPANY_DOMAINS = [
'stripe.com', 'plaid.com', 'xignite.com', 'square.com'
]
# https://pdl-prod-schema.s3-us-west-2.amazonaws.com/15.0/enums/job_title_levels.txt
# for enumerated possible values of job_title_levels
ES_QUERY = {
"query": {
"bool": {
"must": [
{"terms": {"job_company_website": DESIRED_COMPANY_DOMAINS}},
{"term": {"job_title_role": "engineering"}},
{"terms": {"job_title_levels": ["vp", "director", "manager"]}},
{"exists": {"field": "work_email"}}
]
}
}
}
P = {
'query': json.dumps(ES_QUERY),
'size': 100
}
response = requests.get(
PDL_URL,
headers=H,
params=P
).json()
if response["status"] == 200:
for record in response['data']:
# bring in leads and make $$$
print(
record['work_email'],
record['full_name'],
record['job_title'],
record['job_company_name']
)
print(f"successfully grabbed {len(response['data'])} records from pdl")
print(f"{response['total']} total pdl records exist matching this query")
else:
print("NOTE. The eager beaver was not so eager. See error and try again.")
print("error:", response)
```
```python Python3 (SQL) expandable theme={null}
import requests, json
API_KEY = #YOUR API KEY
PDL_URL = "https://api.peopledatalabs.com/v5/person/search"
H = {
'Content-Type': "application/json",
'X-api-key': API_KEY
}
DESIRED_COMPANY_DOMAINS = [
'stripe.com', 'plaid.com', 'xignite.com', 'square.com'
]
COMPANY_DOMAINS_STRING_REP = ", ".join(
(f"'{site}'" for site in DESIRED_COMPANY_DOMAINS)
)
# https://pdl-prod-schema.s3-us-west-2.amazonaws.com/15.0/job_title_levels.txt
# for enumerated possible values of job_title_levels
SQL_QUERY = \
f"""
SELECT * FROM person
WHERE job_company_website IN ({COMPANY_DOMAINS_STRING_REP})
AND job_title_role='engineering'
AND job_title_levels IN ('vp', 'director', 'manager')
AND work_email IS NOT NULL;
"""
P = {
'sql': SQL_QUERY,
'size': 100
}
response = requests.get(
PDL_URL,
headers=H,
params=P
).json()
if response["status"] == 200:
for record in response['data']:
# bring in leads and make $$$
print(
record['work_email'],
record['full_name'],
record['job_title'],
record['job_company_name']
)
print(f"successfully grabbed {len(response['data'])} records from pdl")
print(f"{response['total']} total pdl records exist matching this query")
else:
print("NOTE. The eager beaver was not so eager. See error and try again.")
print("error:", response)
```
## Recruiting
*"I have a client looking for marketing managers and dishwashers in Oregon, but NOT in portland (don't ask why). They want to reach out on LinkedIn, so they asked that each candidate have a Linkedin URL. I want as many people as PDL can give me matching this criteria."*
```python Python3 SDK (Elasticsearch) expandable theme={null}
from time import sleep
import json
# See https://github.com/peopledatalabs/peopledatalabs-python
from peopledatalabs import PDLPY
# Create a client, specifying an API key
client = PDLPY(
api_key="YOUR API KEY",
)
def get_all_pdl_records_es(query):
#runs search in batches with 6 second intervals
PAGE_SIZE = 100
all_records = []
batch = 1
params = {
'query': query,
'size': PAGE_SIZE,
'dataset': "all"
}
while batch == 1 or params['scroll_token']:
response = client.person.search(**params).json()
if response['status'] == 200:
all_records.extend(response['data'])
params['scroll_token'] = response['scroll_token']
print(f"batch {batch} success!")
sleep(6)
batch += 1
else:
print("Unable to continue scrolling")
break
print("done!")
return all_records
if __name__ == '__main__':
ES_QUERY = {
"query": {
"bool": {
"must": [
{"term": {"location_region": "oregon"}},
{"bool": {
"should": [
{"match": {"job_title": "dishwasher"}},
{"bool": {
"must": [
{"term": {"job_title_role": "marketing"}},
{"term": {"job_title_levels": "manager"}}
]
}}
]
}
},
{"exists": {"field": "linkedin_url"}}
],
"must_not":[
{"term": {"location_locality": "portland"}},
]
}
}
}
recruiting_leads = get_all_pdl_records_es(ES_QUERY)
print(f"Got {len(recruiting_leads)} recruiting leads for my wealthy client!")
#GO make_money_with_data(recruiting_leads)!
```
```python Python3 SDK (SQL) expandable theme={null}
from time import sleep
import json
# See https://github.com/peopledatalabs/peopledatalabs-python
from peopledatalabs import PDLPY
# Create a client, specifying an API key
client = PDLPY(
api_key="YOUR API KEY",
)
def get_all_pdl_records_sql(query):
#runs search in batches with 6 second intervals
PAGE_SIZE = 100
all_records = []
batch = 1
params = {
'sql': query,
'size': PAGE_SIZE,
'dataset': "all"
}
while batch == 1 or params['scroll_token']:
response = client.person.search(**params).json()
if response['status'] == 200:
all_records.extend(response['data'])
params['scroll_token'] = response['scroll_token']
print(f"batch {batch} success!")
sleep(6)
batch += 1
else:
print("Unable to continue scrolling")
break
print("done!")
return all_records
SQL_QUERY = \
"""
SELECT * FROM person
WHERE location_region='oregon'
AND NOT location_locality='portland'
AND (
job_title LIKE '%dishwasher%'
OR (
job_title_role='marketing'
AND job_title_levels='manager'
)
)
AND linkedin_url IS NOT NULL;
"""
recruiting_leads = get_all_pdl_records_sql(SQL_QUERY)
print(f"Got {len(recruiting_leads)} recruiting leads for my wealthy client!")
#GO make_money_with_data(recruiting_leads)!
```
```javascript JavaScript (Elasticsearch) expandable theme={null}
// See https://github.com/peopledatalabs/peopledatalabs-js
import PDLJS from 'peopledatalabs';
const PDLJSClient = new PDLJS({ apiKey: "YOUR API KEY" });
const esQuery = {
"query": {
"bool": {
"must": [
{"term": {"location_region": "oregon"}},
{"bool": {
"should": [
{"match": {"job_title": "dishwasher"}},
{"bool": {
"must": [
{"term": {"job_title_role": "marketing"}},
{"term": {"job_title_levels": "manager"}}
]
}}
]
}
},
{"exists": {"field": "linkedin_url"}}
],
"must_not":[
{"term": {"location_locality": "portland"}},
]
}
}
}
var allRecords = [];
var scrollToken = null;
var pageSize = 100;
var batch = 1;
var params = {
searchQuery: esQuery,
size: pageSize,
scroll_token: null,
dataset: "all"
}
// Run initial batch
runBatch();
function runBatch() {
params.scroll_token = scrollToken;
PDLJSClient.person.search.elastic(params).then((data) => {
Array.prototype.push.apply(allRecords, data.data);
scrollToken = data['scroll_token'];
console.log("batch " + batch + " success!");
batch++;
// Runs search in batches with 6 second intervals
if (scrollToken) {
setTimeout(function() {
runBatch(params);
}, 6000);
}
}).catch((error) => {
console.log("Unable to continue scrolling");
console.log("done");
console.log("Got " + allRecords.length + " recruiting leads for my wealthy client!");
});
}
```
```javascript JavaScript (SQL) expandable theme={null}
// See https://github.com/peopledatalabs/peopledatalabs-js
import PDLJS from 'peopledatalabs';
const PDLJSClient = new PDLJS({ apiKey: "YOUR API KEY" });
const sqlQuery = `SELECT * FROM person
WHERE location_region='oregon'
AND NOT location_locality='portland'
AND (
job_title LIKE '%dishwasher%'
OR (
job_title_role='marketing'
AND job_title_levels='manager'
)
)
AND linkedin_url IS NOT NULL;`;
var allRecords = [];
var scrollToken = null;
var pageSize = 100;
var batch = 1;
var params = {
searchQuery: sqlQuery,
size: pageSize,
scroll_token: null,
dataset: "all"
}
// Run initial batch
runBatch();
function runBatch() {
params.scroll_token = scrollToken;
PDLJSClient.person.search.sql(params).then((data) => {
Array.prototype.push.apply(allRecords, data.data);
scrollToken = data['scroll_token'];
console.log("batch " + batch + " success!");
batch++;
// Runs search in batches with 6 second intervals
if (scrollToken) {
setTimeout(function() {
runBatch(params);
}, 6000);
}
}).catch((error) => {
console.log("Unable to continue scrolling");
console.log("done");
console.log("Got " + allRecords.length + " recruiting leads for my wealthy client!");
});
}
```
```ruby Ruby (Elasticsearch) expandable theme={null}
require 'json'
# See https://github.com/peopledatalabs/peopledatalabs-ruby
require 'peopledatalabs'
Peopledatalabs.api_key = 'YOUR API KEY'
PAGE_SIZE = 100
#runs search in batches with 6 second intervals
def get_all_pdl_records_es(query)
all_records = []
batch = 1
scroll_token = {}
while batch == 1 || !scroll_token.nil?
response = Peopledatalabs::Search.people(searchType: 'elastic', query: query, size: PAGE_SIZE, scroll_token: scroll_token, dataset: "all")
if response['status'] == 200
all_records += response['data']
scroll_token = response['scroll_token']
puts "batch #{batch} success!"
sleep(6)
batch += 1
else
puts "Unable to continue scrolling"
break
end
end
puts "done!"
return all_records
end
ES_QUERY = {
"query": {
"bool": {
"must": [
{"term": {"location_region": "oregon"}},
{"bool": {
"should": [
{"match": {"job_title": "dishwasher"}},
{"bool": {
"must": [
{"term": {"job_title_role": "marketing"}},
{"term": {"job_title_levels": "manager"}}
]
}}
]
}
},
{"exists": {"field": "linkedin_url"}}
],
"must_not":[
{"term": {"location_locality": "portland"}},
]
}
}
}
recruiting_leads = get_all_pdl_records_es(ES_QUERY)
puts "Got #{recruiting_leads.length()} recruiting leads for my wealthy client!"
#GO make_money_with_data(recruiting_leads)!
```
```ruby Ruby (SQL) expandable theme={null}
require 'json'
# See https://github.com/peopledatalabs/peopledatalabs-ruby
require 'peopledatalabs'
Peopledatalabs.api_key = 'YOUR API KEY'
PAGE_SIZE = 100
#runs search in batches with 6 second intervals
def get_all_pdl_records_sql(query)
all_records = []
batch = 1
scroll_token = {}
while batch == 1 || !scroll_token.nil?
response = Peopledatalabs::Search.people(searchType: 'sql', query: query, size: PAGE_SIZE, scroll_token: scroll_token, dataset: "all")
if response['status'] == 200
all_records += response['data']
scroll_token = response['scroll_token']
puts "batch #{batch} success!"
sleep(6)
batch += 1
else
puts "Unable to continue scrolling"
break
end
end
puts "done!"
return all_records
end
SQL_QUERY = \
"""
SELECT * FROM person
WHERE location_region='oregon'
AND NOT location_locality='portland'
AND (
job_title LIKE '%dishwasher%'
OR (
job_title_role='marketing'
AND job_title_levels='manager'
)
)
AND linkedin_url IS NOT NULL;
"""
recruiting_leads = get_all_pdl_records_sql(SQL_QUERY)
puts "Got #{recruiting_leads.length()} recruiting leads for my wealthy client!"
#GO make_money_with_data(recruiting_leads)!
```
```python Python3 (Elasticsearch) expandable theme={null}
from time import sleep
import requests, json
API_KEY = #YOUR API KEY
PDL_URL = "https://api.peopledatalabs.com/v5/person/search"
H = {
'Content-Type': "application/json",
'X-api-key': API_KEY
}
def get_all_pdl_records_es(query):
#runs search in batches with 6 second intervals
PAGE_SIZE = 100
all_records = []
batch = 1
params = {
'query': json.dumps(query),
'size': PAGE_SIZE,
'dataset': "all"
}
while batch == 1 or params['scroll_token']:
response = requests.get(
PDL_URL,
headers=H,
params=params
).json()
if response['status'] == 200:
all_records.extend(response['data'])
params['scroll_token'] = response['scroll_token']
print(f"batch {batch} success!")
sleep(6)
batch += 1
else:
print("Unable to continue scrolling")
break
print("done!")
return all_records
if __name__ == '__main__':
ES_QUERY = {
"query": {
"bool": {
"must": [
{"term": {"location_region": "oregon"}},
{"bool": {
"should": [
{"match": {"job_title": "dishwasher"}},
{"bool": {
"must": [
{"term": {"job_title_role": "marketing"}},
{"term": {"job_title_levels": "manager"}}
]
}}
]
}
},
{"exists": {"field": "linkedin_url"}}
],
"must_not":[
{"term": {"location_locality": "portland"}},
]
}
}
}
recruiting_leads = get_all_pdl_records_es(ES_QUERY)
print(f"Got {len(recruiting_leads)} recruiting leads for my wealthy client!")
#GO make_money_with_data(recruiting_leads)!
```
```python Python3 (SQL) expandable theme={null}
from time import sleep
import requests
API_KEY = #YOUR API KEY
PDL_URL = "https://api.peopledatalabs.com/v5/person/search"
H = {
'Content-Type': "application/json",
'X-api-key': API_KEY
}
def get_all_pdl_records_sql(query):
#runs search in batches with 6 second intervals
PAGE_SIZE = 100
all_records = []
batch = 1
params = {
'sql': query,
'size': PAGE_SIZE,
'dataset': "all"
}
while batch == 1 or params['scroll_token']:
response = requests.get(
PDL_URL,
headers=H,
params=params
).json()
if response['status'] == 200:
all_records.extend(response['data'])
params['scroll_token'] = response['scroll_token']
print(f"batch {batch} success!")
sleep(6)
batch += 1
else:
print("Unable to continue scrolling")
break
batch += 1
print("done!")
return all_records
SQL_QUERY = \
"""
SELECT * FROM person
WHERE location_region='oregon'
AND NOT location_locality='portland'
AND (
job_title LIKE '%dishwasher%'
OR (
job_title_role='marketing'
AND job_title_levels='manager'
)
)
AND linkedin_url IS NOT NULL;
"""
recruiting_leads = get_all_pdl_records_sql(SQL_QUERY)
print(f"got {len(recruiting_leads)} recruiting leads for my wealthy client!")
make_money_with_data(recruiting_leads)
```
## Ads
*"I want to sell yachts to rich people via ads on [Facebook](https://www.facebook.com/business/help/170456843145568?id=2469097953376494)."*
```python Python3 SDK (Elasticsearch) expandable theme={null}
from time import sleep
import json
# See https://github.com/peopledatalabs/peopledatalabs-python
from peopledatalabs import PDLPY
# Create a client, specifying an API key
client = PDLPY(
api_key="YOUR API KEY",
)
def get_all_pdl_records_es(query):
#runs search in batches with 6 second intervals
PAGE_SIZE = 100
all_records = []
batch = 1
params = {
'query': query,
'size': PAGE_SIZE,
'dataset': "all"
}
while batch == 1 or params['scroll_token']:
response = client.person.search(**params).json()
if response['status'] == 200:
all_records.extend(response['data'])
params['scroll_token'] = response['scroll_token']
print(f"batch {batch} success!")
sleep(6)
batch += 1
else:
print("Unable to continue scrolling")
break
print("done!")
return all_records
if __name__ == '__main__':
ES_QUERY = {
"query": {
"bool": {
"must": [
{"exists": {"field": "facebook_id"}},
{"prefix": {"interests": "yacht"}},
{"term": {"inferred_salary": ">250,000"}}
]
}
}
}
rich_yacht_people = get_all_pdl_records_es(ES_QUERY)
print(f"Got {len(rich_yacht_people)} rich yacht people for my wealthy client!")
#GO make_money_with_data(rich_yacht_people)!
```
```python Python3 SDK (SQL) expandable theme={null}
from time import sleep
import json
# See https://github.com/peopledatalabs/peopledatalabs-python
from peopledatalabs import PDLPY
# Create a client, specifying an API key
client = PDLPY(
api_key="YOUR API KEY",
)
def get_all_pdl_records_sql(query):
#runs search in batches with 6 second intervals
PAGE_SIZE = 100
all_records = []
batch = 1
params = {
'sql': query,
'size': PAGE_SIZE,
'dataset': "all"
}
while batch == 1 or params['scroll_token']:
response = client.person.search(**params).json()
if response['status'] == 200:
all_records.extend(response['data'])
params['scroll_token'] = response['scroll_token']
print(f"batch {batch} success!")
sleep(6)
batch += 1
else:
print("Unable to continue scrolling")
break
print("done!")
return all_records
SQL_QUERY = \
"""
SELECT * FROM person
WHERE facebook_id IS NOT NULL
AND interests LIKE 'yacht%'
AND inferred_salary='>250,000';
"""
rich_yacht_people = get_all_pdl_records_sql(SQL_QUERY)
print(f"got {len(rich_yacht_people)} rich yacht people for my wealthy client!")
#GO make_money_with_data(rich_yacht_people)
```
```javascript JavaScript (Elasticsearch) expandable theme={null}
// See https://github.com/peopledatalabs/peopledatalabs-js
import PDLJS from 'peopledatalabs';
const PDLJSClient = new PDLJS({ apiKey: "YOUR API KEY" });
const esQuery = {
"query": {
"bool": {
"must": [
{"exists": {"field": "facebook_id"}},
{"prefix": {"interests": "yacht"}},
{"term": {"inferred_salary": ">250,000"}}
]
}
}
}
var allRecords = [];
var scrollToken = null;
var pageSize = 100;
var batch = 1;
var params = {
searchQuery: esQuery,
size: pageSize,
scroll_token: null,
dataset: "all"
}
// Run initial batch
runBatch();
function runBatch() {
params.scroll_token = scrollToken;
PDLJSClient.person.search.elastic(params).then((data) => {
Array.prototype.push.apply(allRecords, data.data);
scrollToken = data['scroll_token'];
console.log("batch " + batch + " success!");
batch++;
// Runs search in batches with 6 second intervals
if (scrollToken) {
setTimeout(function() {
runBatch(params);
}, 6000);
}
}).catch((error) => {
console.log("Unable to continue scrolling");
console.log("done");
console.log("Got " + allRecords.length + " rich yacht people for my wealthy client!");
});
}
```
```javascript JavaScript (SQL) expandable theme={null}
// See https://github.com/peopledatalabs/peopledatalabs-js
import PDLJS from 'peopledatalabs';
const PDLJSClient = new PDLJS({ apiKey: "YOUR API KEY" });
const sqlQuery = `SELECT * FROM person
WHERE facebook_id IS NOT NULL
AND interests LIKE 'yacht%'
AND inferred_salary='>250,000';`;
var allRecords = [];
var scrollToken = null;
var pageSize = 100;
var batch = 1;
var params = {
searchQuery: sqlQuery,
size: pageSize,
scroll_token: null,
dataset: "all"
}
// Run initial batch
runBatch();
function runBatch() {
params.scroll_token = scrollToken;
PDLJSClient.person.search.sql(params).then((data) => {
Array.prototype.push.apply(allRecords, data.data);
scrollToken = data['scroll_token'];
console.log("batch " + batch + " success!");
batch++;
// Runs search in batches with 6 second intervals
if (scrollToken) {
setTimeout(function() {
runBatch(params);
}, 6000);
}
}).catch((error) => {
console.log("Unable to continue scrolling");
console.log("done");
console.log("Got " + allRecords.length + " rich yachting people for my wealthy client!");
});
}
```
```ruby Ruby (Elasticsearch) expandable theme={null}
require 'json'
# See https://github.com/peopledatalabs/peopledatalabs-ruby
require 'peopledatalabs'
Peopledatalabs.api_key = 'YOUR API KEY'
PAGE_SIZE = 100
#runs search in batches with 6 second intervals
def get_all_pdl_records_es(query)
all_records = []
batch = 1
scroll_token = {}
while batch == 1 || !scroll_token.nil?
response = Peopledatalabs::Search.people(searchType: 'elastic', query: query, size: PAGE_SIZE, scroll_token: scroll_token, dataset: "all")
if response['status'] == 200
all_records += response['data']
scroll_token = response['scroll_token']
puts "batch #{batch} success!"
sleep(6)
batch += 1
else
puts "Unable to continue scrolling"
break
end
end
puts "done!"
return all_records
end
ES_QUERY = {
"query": {
"bool": {
"must": [
{"exists": {"field": "facebook_id"}},
{"prefix": {"interests": "yacht"}},
{"term": {"inferred_salary": ">250,000"}}
]
}
}
}
rich_yacht_people = get_all_pdl_records_es(ES_QUERY)
puts "Got #{rich_yacht_people.length()} rich yacht people for my wealthy client!"
#GO make_money_with_data(rich_yacht_people)!
```
```ruby Ruby (SQL) expandable theme={null}
require 'json'
# See https://github.com/peopledatalabs/peopledatalabs-ruby
require 'peopledatalabs'
Peopledatalabs.api_key = 'YOUR API KEY'
PAGE_SIZE = 100
#runs search in batches with 6 second intervals
def get_all_pdl_records_sql(query)
all_records = []
batch = 1
scroll_token = {}
while batch == 1 || !scroll_token.nil?
response = Peopledatalabs::Search.people(searchType: 'sql', query: query, size: PAGE_SIZE, scroll_token: scroll_token, dataset: "all")
if response['status'] == 200
all_records += response['data']
scroll_token = response['scroll_token']
puts "batch #{batch} success!"
sleep(6)
batch += 1
else
puts "Unable to continue scrolling"
break
end
end
puts "done!"
return all_records
end
SQL_QUERY = \
"""
SELECT * FROM person
WHERE facebook_id IS NOT NULL
AND interests LIKE 'yacht%'
AND inferred_salary='>250,000';
"""
rich_yacht_people = get_all_pdl_records_sql(SQL_QUERY)
puts "Got #{rich_yacht_people.length()} rich yacht people for my wealthy client!"
#GO make_money_with_data(rich_yacht_people)!
```
```python Python3 (Elasticsearch) expandable theme={null}
from time import sleep
import requests, json
API_KEY = #YOUR API KEY
PDL_URL = "https://api.peopledatalabs.com/v5/person/search"
H = {
'Content-Type': "application/json",
'X-api-key': API_KEY
}
def get_all_pdl_records_es(query):
#runs search in batches with 6 second intervals
PAGE_SIZE = 100
all_records = []
batch = 1
params = {
'query': json.dumps(query),
'size': PAGE_SIZE,
'dataset': "all"
}
while batch == 1 or params['scroll_token']:
response = requests.get(
PDL_URL,
headers=H,
params=params
).json()
if response['status'] == 200:
all_records.extend(response['data'])
params['scroll_token'] = response['scroll_token']
print(f"batch {batch} success!")
sleep(6)
batch += 1
else:
print("Unable to continue scrolling")
break
print("done!")
return all_records
if __name__ == '__main__':
ES_QUERY = {
"query": {
"bool": {
"must": [
{"exists": {"field": "facebook_id"}},
{"prefix": {"interests": "yacht"}},
{"term": {"inferred_salary": ">250,000"}}
]
}
}
}
rich_yacht_people = get_all_pdl_records_es(ES_QUERY)
print(f"Got {len(rich_yacht_people)} rich yacht people for my wealthy client!")
#GO make_money_with_data(rich_yacht_people)!
```
```python Python3 (SQL) expandable theme={null}
from time import sleep
import requests, json
API_KEY = #YOUR API KEY
PDL_URL = "https://api.peopledatalabs.com/v5/person/search"
H = {
'Content-Type': "application/json",
'X-api-key': API_KEY
}
def get_all_pdl_records_sql(query):
#runs search in batches with 6 second intervals
PAGE_SIZE = 100
all_records = []
batch = 1
params = {
'sql': query,
'size': PAGE_SIZE,
'dataset': "all"
}
while batch == 1 or params['scroll_token']:
response = requests.get(
PDL_URL,
headers=H,
params=params
).json()
if response['status'] == 200:
all_records.extend(response['data'])
params['scroll_token'] = response['scroll_token']
print(f"batch {batch} success!")
sleep(6)
batch += 1
else:
print("Unable to continue scrolling")
break
print("done!")
return all_records
SQL_QUERY = \
"""
SELECT * FROM person
WHERE facebook_id IS NOT NULL
AND interests LIKE 'yacht%'
AND inferred_salary='>250,000';
"""
rich_yacht_people = get_all_pdl_records_sql(SQL_QUERY)
print(f"got {len(rich_yacht_people)} rich yacht people for my wealthy client!")
#GO make_money_with_data(rich_yacht_people)
```
## Customer Insights
*"I want to discover some things about my biggest customer, [Zenefits](http://www.zenefits.com)."*
```python Python3 SDK (Elasticsearch) expandable theme={null}
from time import sleep
import json
# See https://github.com/peopledatalabs/peopledatalabs-python
from peopledatalabs import PDLPY
# Create a client, specifying an API key
client = PDLPY(
api_key="YOUR API KEY",
)
def get_all_pdl_records_es(query):
#runs search in batches with 6 second intervals
PAGE_SIZE = 100
all_records = []
batch = 1
params = {
'query': query,
'size': PAGE_SIZE,
'dataset': "all"
}
while batch == 1 or params['scroll_token']:
response = client.person.search(**params).json()
if response['status'] == 200:
all_records.extend(response['data'])
params['scroll_token'] = response['scroll_token']
print(f"batch {batch} success!")
sleep(6)
batch += 1
else:
print("Unable to continue scrolling")
break
print("done!")
return all_records
if __name__ == '__main__':
ES_QUERY = {
"query": {
"term": {"job_company_website": "zenefits.com"}
}
}
all_zenefits_employees = get_all_pdl_records_es(ES_QUERY)
skills_agg = {}
titles_agg = {}
schools_agg = {}
other_companies_agg = {}
for record in all_zenefits_employees:
for skill in record['skills']:
skills_agg.setdefault(skill, 0)
skills_agg[skill] += 1
if record['job_title']:
titles_agg.setdefault(record['job_title'], 0)
titles_agg[record['job_title']] += 1
for edu in record['education']:
if edu['school'] and edu['school']['type'] == "post-secondary institution":
schools_agg.setdefault(edu['school']['name'], 0)
schools_agg[edu['school']['name']] += 1
for exp in record['experience']:
if exp['company'] and exp['company']['name'] != 'zenefits':
other_companies_agg.setdefault(exp['company']['name'], 0)
other_companies_agg[exp['company']['name']] += 1
print("top 10 skills for zenefits employees:")
for skill, count in sorted(
skills_agg.items(), key = lambda x: x[1], reverse=True
)[:10]:
print(count, skill)
print("top 10 titles for zenefits employees:")
for title, count in sorted(
titles_agg.items(), key = lambda x: x[1], reverse=True
)[:10]:
print(count, title)
print("top 10 universities for zenefits employees:")
for school, count in sorted(
schools_agg.items(), key = lambda x: x[1], reverse=True
)[:10]:
print(count, school)
print("top 10 former companies for zenefits employees:")
for company, count in sorted(
other_companies_agg.items(), key = lambda x: x[1], reverse=True
)[:10]:
print(count, company)
```
```python Python3 SDK (SQL) expandable theme={null}
from time import sleep
import json
# See https://github.com/peopledatalabs/peopledatalabs-python
from peopledatalabs import PDLPY
# Create a client, specifying an API key
client = PDLPY(
api_key="YOUR API KEY",
)
def get_all_pdl_records_sql(query):
#runs search in batches with 6 second intervals
PAGE_SIZE = 100
all_records = []
batch = 1
params = {
'sql': query,
'size': PAGE_SIZE,
'dataset': "all"
}
while batch == 1 or params['scroll_token']:
response = client.person.search(**params).json()
if response['status'] == 200:
all_records.extend(response['data'])
params['scroll_token'] = response['scroll_token']
print(f"batch {batch} success!")
sleep(6)
batch += 1
else:
print("Unable to continue scrolling")
break
print("done!")
return all_records
SQL_QUERY = \
"""
SELECT * FROM person
WHERE job_company_website='zenefits.com';
"""
all_zenefits_employees = get_all_pdl_records_sql(SQL_QUERY)
skills_agg = {}
titles_agg = {}
schools_agg = {}
other_companies_agg = {}
for record in all_zenefits_employees:
for skill in record['skills']:
skills_agg.setdefault(skill, 0)
skills_agg[skill] += 1
if record['job_title']:
titles_agg.setdefault(record['job_title'], 0)
titles_agg[record['job_title']] += 1
for edu in record['education']:
if edu['school'] and edu['school']['type'] == "post-secondary institution":
schools_agg.setdefault(edu['school']['name'], 0)
schools_agg[edu['school']['name']] += 1
for exp in record['experience']:
if exp['company'] and exp['company']['name'] != 'zenefits':
other_companies_agg.setdefault(exp['company']['name'], 0)
other_companies_agg[exp['company']['name']] += 1
print("top 10 skills for zenefits employees:")
for skill, count in sorted(
skills_agg.items(), key = lambda x: x[1], reverse=True
)[:10]:
print(count, skill)
print("top 10 titles for zenefits employees:")
for title, count in sorted(
titles_agg.items(), key = lambda x: x[1], reverse=True
)[:10]:
print(count, title)
print("top 10 universities for zenefits employees:")
for school, count in sorted(
schools_agg.items(), key = lambda x: x[1], reverse=True
)[:10]:
print(count, school)
print("top 10 former companies for zenefits employees:")
for company, count in sorted(
other_companies_agg.items(), key = lambda x: x[1], reverse=True
)[:10]:
print(count, company)
```
```javascript JavaScript (Elasticsearch) expandable theme={null}
// See https://github.com/peopledatalabs/peopledatalabs-js
import PDLJS from 'peopledatalabs';
const PDLJSClient = new PDLJS({ apiKey: "YOUR API KEY" });
const esQuery = {
"query": {
"term": {"job_company_website": "zenefits.com"}
}
}
var allRecords = [];
var scrollToken = null;
var pageSize = 100;
var batch = 1;
var params = {
searchQuery: esQuery,
size: pageSize,
scroll_token: null,
dataset: "all"
}
// Run initial batch
runBatch();
function runBatch() {
params.scroll_token = scrollToken;
PDLJSClient.person.search.elastic(params).then((data) => {
Array.prototype.push.apply(allRecords, data.data);
scrollToken = data['scroll_token'];
console.log("batch " + batch + " success!");
batch++;
// Runs search in batches with 6 second intervals
if (scrollToken) {
setTimeout(function() {
runBatch(params);
}, 6000);
}
}).catch((error) => {
console.log("Unable to continue scrolling");
console.log("done");
printResults();
});
}
function printResults() {
var skillsAgg = {};
var titlesAgg = {};
var schoolsAgg = {};
var otherCompaniesAgg = {};
var sortable;
for (let record in allRecords) {
for (let skill in allRecords[record]["skills"]) {
if (!skillsAgg[allRecords[record]["skills"][skill]]) {
skillsAgg[allRecords[record]["skills"][skill]] = 0;
}
skillsAgg[allRecords[record]["skills"][skill]]++;
}
if (allRecords[record]["job_title"]) {
if (!titlesAgg[allRecords[record]["job_title"]]) {
titlesAgg[allRecords[record]["job_title"]] = 0;
}
titlesAgg[allRecords[record]["job_title"]]++;
for (let edu in allRecords[record]["education"]) {
if (allRecords[record]["education"][edu]["school"] &&
allRecords[record]["education"][edu]["school"]["type"] == "post-secondary institution") {
if (!schoolsAgg[allRecords[record]["education"][edu]["school"]["name"]]) {
schoolsAgg[allRecords[record]["education"][edu]["school"]["name"]] = 0;
}
schoolsAgg[allRecords[record]["education"][edu]["school"]["name"]]++;
}
}
}
for (let exp in allRecords[record]["experience"]) {
if (allRecords[record]["experience"][exp]["company"] &&
allRecords[record]["experience"][exp]["company"]["name"] != "zenefits") {
if (!otherCompaniesAgg[allRecords[record]["experience"][exp]["company"]["name"]]) {
otherCompaniesAgg[allRecords[record]["experience"][exp]["company"]["name"]] = 0;
}
otherCompaniesAgg[allRecords[record]["experience"][exp]["company"]["name"]]++
}
}
}
console.log("top 10 skills for zenefits employees:");
sortAndPrint(skillsAgg);
console.log("top 10 titles for zenefits employees:");
sortAndPrint(titlesAgg);
console.log("top 10 universities for zenefits employees:");
sortAndPrint(schoolsAgg);
console.log("top 10 former companies for zenefits employees:");
sortAndPrint(otherCompaniesAgg);
}
function sortAndPrint(object) {
var sortable = [];
for (let field in object) {
sortable.push([field, object[field]]);
}
sortable.sort(function(a, b) {
return b[1] - a[1];
});
for (let i = 0; i < 10; i++) {
console.log(sortable[i][0]);
}
}
```
```javascript JavaScript (SQL) expandable theme={null}
// See https://github.com/peopledatalabs/peopledatalabs-js
import PDLJS from 'peopledatalabs';
const PDLJSClient = new PDLJS({ apiKey: "YOUR API KEY" });
const sqlQuery = `SELECT * FROM person
WHERE job_company_website='zenefits.com';`;
var allRecords = [];
var scrollToken = null;
var pageSize = 100;
var batch = 1;
var params = {
searchQuery: sqlQuery,
size: pageSize,
scroll_token: null,
dataset: "all"
}
// Run initial batch
runBatch();
function runBatch() {
params.scroll_token = scrollToken;
PDLJSClient.person.search.sql(params).then((data) => {
Array.prototype.push.apply(allRecords, data.data);
scrollToken = data['scroll_token'];
console.log("batch " + batch + " success!");
batch++;
// Runs search in batches with 6 second intervals
if (scrollToken) {
setTimeout(function() {
runBatch(params);
}, 6000);
}
}).catch((error) => {
console.log("Unable to continue scrolling");
console.log("done");
printResults();
});
}
function printResults() {
var skillsAgg = {};
var titlesAgg = {};
var schoolsAgg = {};
var otherCompaniesAgg = {};
var sortable;
for (let record in allRecords) {
for (let skill in allRecords[record]["skills"]) {
if (!skillsAgg[allRecords[record]["skills"][skill]]) {
skillsAgg[allRecords[record]["skills"][skill]] = 0;
}
skillsAgg[allRecords[record]["skills"][skill]]++;
}
if (allRecords[record]["job_title"]) {
if (!titlesAgg[allRecords[record]["job_title"]]) {
titlesAgg[allRecords[record]["job_title"]] = 0;
}
titlesAgg[allRecords[record]["job_title"]]++;
for (let edu in allRecords[record]["education"]) {
if (allRecords[record]["education"][edu]["school"] &&
allRecords[record]["education"][edu]["school"]["type"] == "post-secondary institution") {
if (!schoolsAgg[allRecords[record]["education"][edu]["school"]["name"]]) {
schoolsAgg[allRecords[record]["education"][edu]["school"]["name"]] = 0;
}
schoolsAgg[allRecords[record]["education"][edu]["school"]["name"]]++;
}
}
}
for (let exp in allRecords[record]["experience"]) {
if (allRecords[record]["experience"][exp]["company"] &&
allRecords[record]["experience"][exp]["company"]["name"] != "zenefits") {
if (!otherCompaniesAgg[allRecords[record]["experience"][exp]["company"]["name"]]) {
otherCompaniesAgg[allRecords[record]["experience"][exp]["company"]["name"]] = 0;
}
otherCompaniesAgg[allRecords[record]["experience"][exp]["company"]["name"]]++
}
}
}
console.log("top 10 skills for zenefits employees:");
sortAndPrint(skillsAgg);
console.log("top 10 titles for zenefits employees:");
sortAndPrint(titlesAgg);
console.log("top 10 universities for zenefits employees:");
sortAndPrint(schoolsAgg);
console.log("top 10 former companies for zenefits employees:");
sortAndPrint(otherCompaniesAgg);
}
function sortAndPrint(object) {
var sortable = [];
for (let field in object) {
sortable.push([field, object[field]]);
}
sortable.sort(function(a, b) {
return b[1] - a[1];
});
for (let i = 0; i < 10; i++) {
console.log(sortable[i][0]);
}
}
```
```ruby Ruby (Elasticsearch) expandable theme={null}
# See https://github.com/peopledatalabs/peopledatalabs-ruby
require 'peopledatalabs'
Peopledatalabs.api_key = 'YOUR API KEY'
PAGE_SIZE = 100
#runs search in batches with 6 second intervals
def get_all_pdl_records_es(query)
all_records = []
batch = 1
scroll_token = {}
while batch == 1 || !scroll_token.nil?
response = Peopledatalabs::Search.people(searchType: 'elastic', query: query, size: PAGE_SIZE, scroll_token: scroll_token, dataset: "all")
if response['status'] == 200
all_records += response['data']
scroll_token = response['scroll_token']
puts "batch #{batch} success!"
sleep(6)
batch += 1
else
puts "Unable to continue scrolling"
break
end
end
puts "done!"
return all_records
end
ES_QUERY = {
"query": {
"term": {"job_company_website": "zenefits.com"}
}
}
all_zenefits_employees = get_all_pdl_records_es(ES_QUERY)
skills_agg = {}
titles_agg = {}
schools_agg = {}
other_companies_agg = {}
all_zenefits_employees.each do |record|
record['skills'].each do |skill|
skills_agg[skill] = skills_agg.fetch(skill, 0)
skills_agg[skill] += 1
end
if record.key?('job_title')
titles_agg[record['job_title']] = titles_agg.fetch(record['job_title'], 0)
titles_agg[record['job_title']] += 1
record['education'].each do |edu|
if edu.key?('school') && !edu['school'].nil? && edu['school']['type'] == "post-secondary institution"
schools_agg[edu['school']['name']] = schools_agg.fetch(edu['school']['name'], 0)
schools_agg[edu['school']['name']] += 1
end
end
end
record['experience'].each do |exp|
if exp.key?('company') && !exp['company'].nil? && exp['company']['name'] != 'zenefits'
other_companies_agg[exp['company']['name']] = other_companies_agg.fetch(exp['company']['name'], 0)
other_companies_agg[exp['company']['name']] += 1
end
end
end
puts "top 10 skills for zenefits employees:"
skills_agg.sort_by(&:last).reverse.first(10).each { |key, value| puts "#{key} #{value}" }
puts "top 10 titles for zenefits employees:"
titles_agg.sort_by(&:last).reverse.first(10).each { |key, value| puts "#{key} #{value}" }
puts "top 10 universities for zenefits employees:"
schools_agg.sort_by(&:last).reverse.first(10).each { |key, value| puts "#{key} #{value}" }
puts "top 10 former companies for zenefits employees:"
other_companies_agg.sort_by(&:last).reverse.first(10).each { |key, value| puts "#{key} #{value}" }
```
```ruby Ruby (SQL) expandable theme={null}
# See https://github.com/peopledatalabs/peopledatalabs-ruby
require 'peopledatalabs'
Peopledatalabs.api_key = 'YOUR API KEY'
PAGE_SIZE = 100
#runs search in batches with 6 second intervals
def get_all_pdl_records_sql(query)
all_records = []
batch = 1
scroll_token = {}
while batch == 1 || !scroll_token.nil?
response = Peopledatalabs::Search.people(searchType: 'sql', query: query, size: PAGE_SIZE, scroll_token: scroll_token, dataset: "all")
if response['status'] == 200
all_records += response['data']
scroll_token = response['scroll_token']
puts "batch #{batch} success!"
sleep(6)
batch += 1
else
puts "Unable to continue scrolling"
break
end
end
puts "done!"
return all_records
end
SQL_QUERY = \
"""
SELECT * FROM person
WHERE job_company_website='zenefits.com';
"""
all_zenefits_employees = get_all_pdl_records_sql(SQL_QUERY)
skills_agg = {}
titles_agg = {}
schools_agg = {}
other_companies_agg = {}
all_zenefits_employees.each do |record|
record['skills'].each do |skill|
skills_agg[skill] = skills_agg.fetch(skill, 0)
skills_agg[skill] += 1
end
if record.key?('job_title')
titles_agg[record['job_title']] = titles_agg.fetch(record['job_title'], 0)
titles_agg[record['job_title']] += 1
record['education'].each do |edu|
if edu.key?('school') && !edu['school'].nil? && edu['school']['type'] == "post-secondary institution"
schools_agg[edu['school']['name']] = schools_agg.fetch(edu['school']['name'], 0)
schools_agg[edu['school']['name']] += 1
end
end
end
record['experience'].each do |exp|
if exp.key?('company') && !exp['company'].nil? && exp['company']['name'] != 'zenefits'
other_companies_agg[exp['company']['name']] = other_companies_agg.fetch(exp['company']['name'], 0)
other_companies_agg[exp['company']['name']] += 1
end
end
end
puts "top 10 skills for zenefits employees:"
skills_agg.sort_by(&:last).reverse.first(10).each { |key, value| puts "#{key} #{value}" }
puts "top 10 titles for zenefits employees:"
titles_agg.sort_by(&:last).reverse.first(10).each { |key, value| puts "#{key} #{value}" }
puts "top 10 universities for zenefits employees:"
schools_agg.sort_by(&:last).reverse.first(10).each { |key, value| puts "#{key} #{value}" }
puts "top 10 former companies for zenefits employees:"
other_companies_agg.sort_by(&:last).reverse.first(10).each { |key, value| puts "#{key} #{value}" }
```
```python Python3 (Elasticsearch) expandable theme={null}
from time import sleep
import requests, json
API_KEY = #YOUR API KEY
PDL_URL = "https://api.peopledatalabs.com/v5/person/search"
H = {
'Content-Type': "application/json",
'X-api-key': API_KEY
}
def get_all_pdl_records_es(query):
#runs search in batches with 6 second intervals
PAGE_SIZE = 100
all_records = []
batch = 1
params = {
'query': json.dumps(query),
'size': PAGE_SIZE,
'dataset': "all"
}
while batch == 1 or params['scroll_token']:
response = requests.get(
PDL_URL,
headers=H,
params=params
).json()
if response['status'] == 200:
all_records.extend(response['data'])
params['scroll_token'] = response['scroll_token']
print(f"batch {batch} success!")
sleep(6)
batch += 1
else:
print("Unable to continue scrolling")
break
print("done!")
return all_records
if __name__ == '__main__':
ES_QUERY = {
"query": {
"term": {"job_company_website": "zenefits.com"}
}
}
all_zenefits_employees = get_all_pdl_records_es(ES_QUERY)
skills_agg = {}
titles_agg = {}
schools_agg = {}
other_companies_agg = {}
for record in all_zenefits_employees:
for skill in record['skills']:
skills_agg.setdefault(skill, 0)
skills_agg[skill] += 1
if record['job_title']:
titles_agg.setdefault(record['job_title'], 0)
titles_agg[record['job_title']] += 1
for edu in record['education']:
if edu['school'] and edu['school']['type'] == "post-secondary institution":
schools_agg.setdefault(edu['school']['name'], 0)
schools_agg[edu['school']['name']] += 1
for exp in record['experience']:
if exp['company'] and exp['company']['name'] != 'zenefits':
other_companies_agg.setdefault(exp['company']['name'], 0)
other_companies_agg[exp['company']['name']] += 1
print("top 10 skills for zenefits employees:")
for skill, count in sorted(
skills_agg.items(), key = lambda x: x[1], reverse=True
)[:10]:
print(count, skill)
print("top 10 titles for zenefits employees:")
for title, count in sorted(
titles_agg.items(), key = lambda x: x[1], reverse=True
)[:10]:
print(count, title)
print("top 10 universities for zenefits employees:")
for school, count in sorted(
schools_agg.items(), key = lambda x: x[1], reverse=True
)[:10]:
print(count, school)
print("top 10 former companies for zenefits employees:")
for company, count in sorted(
other_companies_agg.items(), key = lambda x: x[1], reverse=True
)[:10]:
print(count, company)
```
```python Python3 (SQL) expandable theme={null}
from time import sleep
import requests, json
API_KEY = #YOUR API KEY
PDL_URL = "https://api.peopledatalabs.com/v5/person/search"
H = {
'Content-Type': "application/json",
'X-api-key': API_KEY
}
def get_all_pdl_records_sql(query):
#runs search in batches with 6 second intervals
PAGE_SIZE = 100
all_records = []
batch = 1
params = {
'sql': query,
'size': PAGE_SIZE,
'dataset': "all"
}
while batch == 1 or params['scroll_token']:
response = requests.get(
PDL_URL,
headers=H,
params=params
).json()
if response['status'] == 200:
all_records.extend(response['data'])
params['scroll_token'] = response['scroll_token']
print(f"batch {batch} success!")
sleep(6)
batch += 1
else:
print("Unable to continue scrolling")
break
print("done!")
return all_records
SQL_QUERY = \
"""
SELECT * FROM person
WHERE job_company_website='zenefits.com';
"""
all_zenefits_employees = get_all_pdl_records_sql(SQL_QUERY)
skills_agg = {}
titles_agg = {}
schools_agg = {}
other_companies_agg = {}
for record in all_zenefits_employees:
for skill in record['skills']:
skills_agg.setdefault(skill, 0)
skills_agg[skill] += 1
if record['job_title']:
titles_agg.setdefault(record['job_title'], 0)
titles_agg[record['job_title']] += 1
for edu in record['education']:
if edu['school'] and edu['school']['type'] == "post-secondary institution":
schools_agg.setdefault(edu['school']['name'], 0)
schools_agg[edu['school']['name']] += 1
for exp in record['experience']:
if exp['company'] and exp['company']['name'] != 'zenefits':
other_companies_agg.setdefault(exp['company']['name'], 0)
other_companies_agg[exp['company']['name']] += 1
print("top 10 skills for zenefits employees:")
for skill, count in sorted(
skills_agg.items(), key = lambda x: x[1], reverse=True
)[:10]:
print(count, skill)
print("top 10 titles for zenefits employees:")
for title, count in sorted(
titles_agg.items(), key = lambda x: x[1], reverse=True
)[:10]:
print(count, title)
print("top 10 universities for zenefits employees:")
for school, count in sorted(
schools_agg.items(), key = lambda x: x[1], reverse=True
)[:10]:
print(count, school)
print("top 10 former companies for zenefits employees:")
for company, count in sorted(
other_companies_agg.items(), key = lambda x: x[1], reverse=True
)[:10]:
print(count, company)
```
## Advanced Examples
### Company Enrichment and Person Search
*I want to find X number of people at each company in my list*
```python Python SDK (Elasticsearch) expandable theme={null}
import json
# See https://github.com/peopledatalabs/peopledatalabs-python
from peopledatalabs import PDLPY
# Create a client, specifying an API key
client = PDLPY(
api_key="YOUR API KEY",
)
company_websites = [
"facebook.com",
"amazon.com",
"apple.com",
"netflix.com",
"google.com"
]
max_num_people = 100
# Enrich company then find people at that company:
for company_website in company_websites:
# Company Enrichment
querystring = { "website": company_website }
response = client.company.enrichment(**querystring).json()
if response['status'] == 200:
enriched_company = response
else:
enriched_company = {}
print(f"Company Enrichment Error for [{company_website}]: {response.text}")
# Person Search
company_employee_matches = {}
if enriched_company:
ES_QUERY = {
"query": {
"bool": {
"must": [
{"term": {"job_company_id": enriched_company["id"]}},
]
}
}
}
params = {
'query': ES_QUERY,
'size': max_num_people
}
response = client.person.search(**params).json()
if response['status'] == 200:
company_employee_matches = response['data']
else:
company_employee_matches = {}
print(f"Person Search Error for [{company_website}]: {response.text}")
print(f"Found {len(company_employee_matches)} employee profiles at {company_website}")
```
```python Python SDK (SQL) expandable theme={null}
import json
# See https://github.com/peopledatalabs/peopledatalabs-python
from peopledatalabs import PDLPY
# Create a client, specifying an API key
client = PDLPY(
api_key="YOUR API KEY",
)
company_websites = [
"facebook.com",
"amazon.com",
"apple.com",
"netflix.com",
"google.com"
]
max_num_people = 100
# Enrich company then find people at that company:
for company_website in company_websites:
# Company Enrichment
querystring = { "website": company_website }
response = client.company.enrichment(**querystring).json()
if response['status'] == 200:
enriched_company = response
else:
enriched_company = {}
print(f"Company Enrichment Error for [{company_website}]: {response.text}")
# Person Search
company_employee_matches = {}
if enriched_company:
SQL_QUERY = f"""
SELECT * FROM person
WHERE job_company_id = '{enriched_company['id']}'
"""
params = {
'sql': SQL_QUERY,
'size': max_num_people
}
response = client.person.search(**params).json()
if response['status'] == 200:
company_employee_matches = response['data']
else:
company_employee_matches = {}
print(f"Person Search Error for [{company_website}]: {response.text}")
print(f"Found {len(company_employee_matches)} employee profiles at {company_website}")
```
```javascript JavaScript (Elasticsearch) expandable theme={null}
// See https://github.com/peopledatalabs/peopledatalabs-js
import PDLJS from 'peopledatalabs';
const PDLJSClient = new PDLJS({ apiKey: "YOUR API KEY" });
const companyWebsites = [
"facebook.com",
"amazon.com",
"apple.com",
"netflix.com",
"google.com"
];
const maxMumPeople = 100;
// Enrich company then find people at that company:
for (let companyWebsite = 0; companyWebsite < companyWebsites.length; companyWebsite++) {
// Company Enrichment
let queryString = { "website": companyWebsites[companyWebsite] };
let enrichedCompany = {};
let companyEmployeeMatches = {};
PDLJSClient.company.enrichment(queryString).then((enrichedCompany) => {
// Person Search
let esQuery = {
query: {
bool: {
must:[
{term: {job_company_id: enrichedCompany.id}},
]
}
}
}
let params = {
searchQuery: esQuery,
size: maxMumPeople
}
PDLJSClient.person.search.elastic(params).then((data) => {
companyEmployeeMatches = data.data;
console.log("Found " + companyEmployeeMatches.length +
" employee profiles at " + companyWebsites[companyWebsite]);
}).catch((error) => {
console.log("Person Seach Error for " + companyWebsites[companyWebsite] +
": " + error);
});
}).catch((error) => {
console.log("Company Enrichment Error for " + companyWebsites[companyWebsite] +
": " + error);
});
}
```
```javascript JavaScript (SQL) expandable theme={null}
// See https://github.com/peopledatalabs/peopledatalabs-js
import PDLJS from 'peopledatalabs';
const PDLJSClient = new PDLJS({ apiKey: "YOUR API KEY" });
const companyWebsites = [
"facebook.com",
"amazon.com",
"apple.com",
"netflix.com",
"google.com"
];
const maxMumPeople = 100;
// Enrich company then find people at that company:
for (let companyWebsite = 0; companyWebsite < companyWebsites.length; companyWebsite++) {
// Company Enrichment
let queryString = { "website": companyWebsites[companyWebsite] };
let enrichedCompany = {};
let companyEmployeeMatches = {};
PDLJSClient.company.enrichment(queryString).then((enrichedCompany) => {
// Person Search
let sqlQuery = `SELECT * FROM person
WHERE job_company_id = '` + enrichedCompany.id + `';`;
let params = {
searchQuery: sqlQuery,
size: maxMumPeople
}
PDLJSClient.person.search.sql(params).then((data) => {
companyEmployeeMatches = data.data;
console.log("Found " + companyEmployeeMatches.length +
" employee profiles at " + companyWebsites[companyWebsite]);
}).catch((error) => {
console.log("Person Seach Error for " + companyWebsites[companyWebsite] +
": " + error);
});
}).catch((error) => {
console.log("Company Enrichment Error for " + companyWebsites[companyWebsite]
+ ": " + error);
});
}
```
```ruby Ruby (Elasticsearch) expandable theme={null}
# See https://github.com/peopledatalabs/peopledatalabs-ruby
require 'peopledatalabs'
Peopledatalabs.api_key = 'YOUR API KEY'
company_websites = [
"facebook.com",
"amazon.com",
"apple.com",
"netflix.com",
"google.com"
]
max_num_people = 100
# Enrich company then find people at that company:
company_websites.each do |company_website|
# Company Enrichment
querystring = { "website": company_website }
response = Peopledatalabs::Enrichment.company(params: querystring)
if response['status'] == 200
enriched_company = response
else
enriched_company = {}
puts "Company Enrichment Error for [#{company_website}]: #{response}"
end
# Person Search
company_employee_matches = {}
if !enriched_company.nil?
es_query = {
"query": {
"bool": {
"must": [
{"term": {"job_company_id": enriched_company["id"]}},
]
}
}
}
response = Peopledatalabs::Search.people(searchType: 'elastic', query: es_query, size: max_num_people)
if response['status'] == 200
company_employee_matches = response['data']
else
company_employee_matches = {}
puts "Person Search Error for [#{company_website}]: #{response}"
end
end
puts "Found #{company_employee_matches.length()} employee profiles at #{company_website}"
end
```
```ruby Ruby (SQL) expandable theme={null}
# See https://github.com/peopledatalabs/peopledatalabs-ruby
require 'peopledatalabs'
Peopledatalabs.api_key = 'YOUR API KEY'
company_websites = [
"facebook.com",
"amazon.com",
"apple.com",
"netflix.com",
"google.com"
]
max_num_people = 100
# Enrich company then find people at that company:
company_websites.each do |company_website|
# Company Enrichment
querystring = { "website": company_website }
response = Peopledatalabs::Enrichment.company(params: querystring)
if response['status'] == 200
enriched_company = response
else
enriched_company = {}
puts "Company Enrichment Error for [#{company_website}]: #{response}"
end
# Person Search
company_employee_matches = {}
if !enriched_company.nil?
sql_query = """
SELECT * FROM person
WHERE job_company_id = '#{enriched_company['id']}'
"""
response = Peopledatalabs::Search.people(searchType: 'sql', query: sql_query, size: max_num_people)
if response['status'] == 200
company_employee_matches = response['data']
else
company_employee_matches = {}
puts "Person Search Error for [#{company_website}]: #{response}"
end
end
puts "Found #{company_employee_matches.length()} employee profiles at #{company_website}"
end
```
```python Python3 (Elasticsearch) expandable theme={null}
import json
import requests
PDL_COMPANY_ENRICH_URL = "https://api.peopledatalabs.com/v5/company/enrich"
PDL_PERSON_SEARCH_URL = "https://api.peopledatalabs.com/v5/person/search"
API_KEY = "####" # Enter your api key here
company_websites = [
"facebook.com",
"amazon.com",
"apple.com",
"netflix.com",
"google.com"
]
max_num_people = 100
# Enrich company then find people at that company:
for company_website in company_websites:
# Company Enrichment
querystring = { "website": company_website }
headers = {
'accept': "application/json",
'content-type': "application/json",
'x-api-key': API_KEY
}
response = requests.request("GET", PDL_COMPANY_ENRICH_URL, headers=headers, params=querystring)
if response.status_code == 200:
enriched_company = response.json()
else:
enriched_company = {}
print(f"Company Enrichment Error for [{company_website}]: {response.text}")
# Person Search
company_employee_matches = {}
if enriched_company:
headers = {
'Content-Type': "application/json",
'X-api-key': API_KEY
}
ES_QUERY = {
"query": {
"bool": {
"must": [
{"term": {"job_company_id": enriched_company["id"]}},
]
}
}
}
params = {
'query': json.dumps(ES_QUERY),
'size': max_num_people
}
response = requests.get( PDL_PERSON_SEARCH_URL, headers=headers, params=params)
if response.status_code == 200:
company_employee_matches = response.json()['data']
else:
company_employee_matches = {}
print(f"Person Search Error for [{company_website}]: {response.text}")
print(f"Found {len(company_employee_matches)} employee profiles at {company_website}")
```
```python Python3 (SQL) expandable theme={null}
import json
import requests
PDL_COMPANY_ENRICH_URL = "https://api.peopledatalabs.com/v5/company/enrich"
PDL_PERSON_SEARCH_URL = "https://api.peopledatalabs.com/v5/person/search"
API_KEY = "####" # Enter your api key here
company_websites = [
"facebook.com",
"amazon.com",
"apple.com",
"netflix.com",
"google.com"
]
max_num_people = 100
# Enrich company then find people at that company:
for company_website in company_websites:
# Company Enrichment
querystring = { "website": company_website }
headers = {
'accept': "application/json",
'content-type': "application/json",
'x-api-key': API_KEY
}
response = requests.request("GET", PDL_COMPANY_ENRICH_URL, headers=headers, params=querystring)
if response.status_code == 200:
enriched_company = response.json()
else:
enriched_company = {}
print(f"Company Enrichment Error for [{company_website}]: {response.text}")
# Person Search
company_employee_matches = {}
if enriched_company:
headers = {
'Content-Type': "application/json",
'X-api-key': API_KEY
}
SQL_QUERY = f"""
SELECT * FROM person
WHERE job_company_id = '{enriched_company['id']}'
"""
params = {
'sql': SQL_QUERY,
'size': max_num_people
}
response = requests.get( PDL_PERSON_SEARCH_URL, headers=headers, params=params)
if response.status_code == 200:
company_employee_matches = response.json()['data']
else:
company_employee_matches = {}
print(f"Person Search Error for [{company_website}]: {response.text}")
print(f"Found {len(company_employee_matches)} employee profiles at {company_website}")
```
# Query Limitations
The following Elasticsearch query types will be accepted:
* [term](https://www.elastic.co/guide/en/elasticsearch/reference/7.7/query-dsl-term-query.html)
* [terms](https://www.elastic.co/guide/en/elasticsearch/reference/7.7/query-dsl-terms-query.html)
* [exists](https://www.elastic.co/guide/en/elasticsearch/reference/7.7/query-dsl-exists-query.html)
* [bool](https://www.elastic.co/guide/en/elasticsearch/reference/7.7/query-dsl-bool-query.html)
* [match](https://www.elastic.co/guide/en/elasticsearch/reference/7.7/query-dsl-match-query.html)
* [range](https://www.elastic.co/guide/en/elasticsearch/reference/7.7/query-dsl-range-query.html)
* [match\_phrase](https://www.elastic.co/guide/en/elasticsearch/reference/7.7/query-dsl-match-query-phrase.html)
* [wildcard](https://www.elastic.co/guide/en/elasticsearch/reference/7.7/query-dsl-wildcard-query.html)
* [prefix](https://www.elastic.co/guide/en/elasticsearch/reference/7.7/query-dsl-prefix-query.html)
* [match\_all](https://www.elastic.co/guide/en/elasticsearch/reference/7.7/query-dsl-match-all-query.html)
Most specialized options are disabled, such as boosting and custom scoring. No aggregations.
Any SQL query that translates to the above available query types via the [ES SQL translate API](https://www.elastic.co/guide/en/elasticsearch/reference/7.7/sql-translate.html) will be accepted. This means most basic SQL. No joins, groupbys, etc.
Any array found in the query (such as a `terms` array) will have a hard limit of 100 elements. Any query containing an array surpassing this limit will be rejected.
# Full Example Response
```json JSON expandable theme={null}
{
"status": 200,
"data": [
{
"id": "qEnOZ5Oh0poWnQ1luFBfVw_0000",
"full_name": "sean thorne",
"first_name": "sean",
"middle_initial": "f",
"middle_name": "fong",
"last_name": "thorne",
"sex": "male",
"birth_year": "1990",
"birth_date": null,
"linkedin_url": "linkedin.com/in/seanthorne",
"linkedin_username": "seanthorne",
"linkedin_id": "145991517",
"facebook_url": "facebook.com/deseanthorne",
"facebook_username": "deseanthorne",
"facebook_id": "1089351304",
"twitter_url": "twitter.com/seanthorne5",
"twitter_username": "seanthorne5",
"github_url": null,
"github_username": null,
"work_email": "sean@peopledatalabs.com",
"personal_emails": ["sean@gmail.com"],
"mobile_phone": "+14155688415",
"industry": "computer software",
"job_title": "co-founder and chief executive officer",
"job_title_role": null,
"job_title_sub_role": null,
"job_title_levels": [
"owner",
"cxo"
],
"job_company_id": "peopledatalabs",
"job_company_name": "people data labs",
"job_company_website": "peopledatalabs.com",
"job_company_size": "11-50",
"job_company_founded": "2015",
"job_company_industry": "computer software",
"job_company_linkedin_url": "linkedin.com/company/peopledatalabs",
"job_company_linkedin_id": "18170482",
"job_company_facebook_url": "facebook.com/peopledatalabs",
"job_company_twitter_url": "twitter.com/peopledatalabs",
"job_company_location_name": "san francisco, california, united states",
"job_company_location_locality": "san francisco",
"job_company_location_metro": "san francisco, california",
"job_company_location_region": "california",
"job_company_location_geo": "37.77,-122.41",
"job_company_location_street_address": "455 market street",
"job_company_location_address_line_2": "suite 1670",
"job_company_location_postal_code": "94105",
"job_company_location_country": "united states",
"job_company_location_continent": "north america",
"job_last_updated": "2020-12-01",
"job_start_date": "2015-03",
"location_name": "san francisco, california, united states",
"location_locality": "san francisco",
"location_metro": "san francisco, california",
"location_region": "california",
"location_country": "united states",
"location_continent": "north america",
"location_street_address": null,
"location_address_line_2": null,
"location_postal_code": null,
"location_geo": "37.77,-122.41",
"location_last_updated": "2020-12-01",
"phone_numbers": [
"+14155688415"
],
"emails": [
{
"address": "sthorne@uoregon.edu",
"type": null
},
{
"address": "sean@hallspot.com",
"type": "professional"
},
{
"address": "sean@talentiq.co",
"type": "professional"
},
{
"address": "sean.thorne@talentiq.co",
"type": "professional"
},
{
"address": "sean@peopledatalabs.com",
"type": "current_professional"
},
{
"address": "sthorne@peopledatalabs.com",
"type": "current_professional"
},
{
"address": "sean.thorne@peopledatalabs.com",
"type": "current_professional"
}
],
"interests": [
"location based services",
"mobile",
"social media",
"colleges",
"university students",
"consumer internet",
"college campuses"
],
"skills": [
"entrepreneurship",
"start ups",
"management",
"public speaking",
"strategic partnerships",
"strategy",
"fundraising",
"saas",
"enterprise technology sales",
"social networking"
],
"location_names": [
"san francisco, california, united states",
"albany, california, united states",
"portland, oregon, united states"
],
"regions": [
"california, united states",
"oregon, united states"
],
"countries": [
"united states"
],
"street_addresses": [],
"experience": [
{
"company": {
"name": "hallspot",
"size": "1-10",
"id": "hallspot",
"founded": "2013",
"industry": "computer software",
"location": {
"name": "portland, oregon, united states",
"locality": "portland",
"region": "oregon",
"metro": "portland, oregon",
"country": "united states",
"continent": "north america",
"street_address": "1231 northwest hoyt street",
"address_line_2": "suite 202",
"postal_code": "97209",
"geo": "45.52,-122.67"
},
"linkedin_url": "linkedin.com/company/hallspot",
"linkedin_id": "3019184",
"facebook_url": null,
"twitter_url": "twitter.com/hallspot",
"website": "hallspot.com"
},
"location_names": [],
"end_date": "2015-02",
"start_date": "2012-08",
"title": {
"name": "co-founder",
"role": null,
"sub_role": null,
"levels": [
"owner"
]
},
"is_primary": false
},
{
"company": {
"name": "people data labs",
"size": "11-50",
"id": "peopledatalabs",
"founded": "2015",
"industry": "computer software",
"location": {
"name": "san francisco, california, united states",
"locality": "san francisco",
"region": "california",
"metro": "san francisco, california",
"country": "united states",
"continent": "north america",
"street_address": "455 market street",
"address_line_2": "suite 1670",
"postal_code": "94105",
"geo": "37.77,-122.41"
},
"linkedin_url": "linkedin.com/company/peopledatalabs",
"linkedin_id": "18170482",
"facebook_url": "facebook.com/peopledatalabs",
"twitter_url": "twitter.com/peopledatalabs",
"website": "peopledatalabs.com"
},
"location_names": [],
"end_date": null,
"start_date": "2015-03",
"title": {
"name": "co-founder and chief executive officer",
"role": null,
"sub_role": null,
"levels": [
"owner",
"cxo"
]
},
"is_primary": true
}
],
"education": [
{
"school": {
"name": "university of oregon",
"type": "post-secondary institution",
"id": "64LkgfdwWYkCC2TjbldMDQ_0",
"location": {
"name": "eugene, oregon, united states",
"locality": "eugene",
"region": "oregon",
"country": "united states",
"continent": "north america"
},
"linkedin_url": "linkedin.com/school/university-of-oregon",
"facebook_url": "facebook.com/universityoforegon",
"twitter_url": "twitter.com/uoregon",
"linkedin_id": "19207",
"website": "uoregon.edu",
"domain": "uoregon.edu"
},
"end_date": "2014",
"start_date": "2010",
"gpa": null,
"degrees": [],
"majors": [
"entrepreneurship"
],
"minors": []
}
],
"profiles": [
{
"network": "linkedin",
"id": "145991517",
"url": "linkedin.com/in/seanthorne",
"username": "seanthorne"
},
{
"network": "facebook",
"id": "1089351304",
"url": "facebook.com/deseanthorne",
"username": "deseanthorne"
},
{
"network": "twitter",
"id": null,
"url": "twitter.com/seanthorne5",
"username": "seanthorne5"
},
{
"network": "linkedin",
"id": null,
"url": "linkedin.com/in/sean-thorne-9b9a8540",
"username": "sean-thorne-9b9a8540"
},
{
"network": "angellist",
"id": null,
"url": "angel.co/deseanthorne",
"username": "deseanthorne"
},
{
"network": "gravatar",
"id": null,
"url": "gravatar.com/seanthorne5",
"username": "seanthorne5"
},
{
"network": "klout",
"id": null,
"url": "klout.com/seanthorne5",
"username": "seanthorne5"
},
{
"network": "aboutme",
"id": null,
"url": "about.me/sean_thorne",
"username": "sean_thorne"
}
]
}
],
"scroll_token": "1117$12.176522"
"total": 94
}
```
# Full Field Mapping
This section contains the Elasticsearch mapping for our full Person Schema indicating which fields have been indexed and made searchable through our Person Search API, along with the corresponding data type for the field.
```json JSON expandable theme={null}
{
"_routing" : {
"required" : true
},
"date_detection" : false,
"properties" : {
"birth_date" : {
"type" : "keyword"
},
"birth_year" : {
"type" : "keyword"
},
"certifications" : {
"properties" : {
"end_date" : {
"type" : "keyword"
},
"name" : {
"type" : "keyword"
},
"organization" : {
"type" : "keyword"
},
"start_date" : {
"type" : "keyword"
}
}
},
"countries" : {
"type" : "keyword"
},
"datapull" : {
"type" : "keyword"
},
"education" : {
"properties" : {
"degrees" : {
"type" : "keyword"
},
"end_date" : {
"type" : "keyword"
},
"gpa" : {
"type" : "float",
"doc_values" : false
},
"majors" : {
"type" : "keyword"
},
"minors" : {
"type" : "keyword"
},
"raw" : {
"type" : "keyword",
"index" : false
},
"school" : {
"properties" : {
"domain" : {
"type" : "keyword"
},
"facebook_url" : {
"type" : "keyword"
},
"id" : {
"type" : "keyword"
},
"linkedin_id" : {
"type" : "keyword"
},
"linkedin_url" : {
"type" : "keyword"
},
"location" : {
"properties" : {
"continent" : {
"type" : "keyword"
},
"country" : {
"type" : "keyword"
},
"locality" : {
"type" : "keyword"
},
"name" : {
"type" : "keyword"
},
"region" : {
"type" : "keyword"
}
}
},
"name" : {
"type" : "keyword"
},
"raw" : {
"type" : "keyword"
},
"twitter_url" : {
"type" : "keyword"
},
"type" : {
"type" : "keyword"
},
"website" : {
"type" : "keyword"
}
}
},
"start_date" : {
"type" : "keyword"
},
"summary" : {
"type" : "keyword",
"index" : false
}
}
},
"email_hashes" : {
"type" : "keyword"
},
"emails" : {
"properties" : {
"address" : {
"type" : "keyword"
},
"type" : {
"type" : "keyword"
}
}
},
"experience" : {
"properties" : {
"company" : {
"properties" : {
"email_domains" : {
"type" : "keyword"
},
"facebook_url" : {
"type" : "keyword"
},
"founded" : {
"type" : "keyword"
},
"fuzzy_match" : {
"type" : "boolean",
"doc_values" : false
},
"id" : {
"type" : "keyword"
},
"industry" : {
"type" : "keyword"
},
"linkedin_id" : {
"type" : "keyword"
},
"linkedin_url" : {
"type" : "keyword"
},
"location" : {
"properties" : {
"address_line_2" : {
"type" : "keyword"
},
"continent" : {
"type" : "keyword"
},
"country" : {
"type" : "keyword"
},
"geo" : {
"type" : "geo_point",
"doc_values" : false
},
"locality" : {
"type" : "keyword"
},
"metro" : {
"type" : "keyword"
},
"name" : {
"type" : "keyword"
},
"postal_code" : {
"type" : "keyword"
},
"region" : {
"type" : "keyword"
},
"street_address" : {
"type" : "keyword"
}
}
},
"name" : {
"type" : "keyword"
},
"raw" : {
"type" : "keyword"
},
"size" : {
"type" : "keyword"
},
"ticker" : {
"type" : "keyword"
},
"twitter_url" : {
"type" : "keyword"
},
"type" : {
"type" : "keyword"
},
"website" : {
"type" : "keyword"
}
}
},
"end_date" : {
"type" : "keyword"
},
"is_primary" : {
"type" : "boolean",
"doc_values" : false
},
"location_names" : {
"type" : "keyword"
},
"start_date" : {
"type" : "keyword"
},
"summary" : {
"type" : "text"
},
"title" : {
"properties" : {
"functions" : {
"type" : "keyword"
},
"levels" : {
"type" : "keyword"
},
"name" : {
"type" : "keyword",
"fields" : {
"text" : {
"type" : "text"
}
},
"ignore_above" : 256
},
"raw" : {
"type" : "keyword"
},
"role" : {
"type" : "keyword"
},
"sub_role" : {
"type" : "keyword"
}
}
},
"type" : {
"type" : "keyword"
}
}
},
"facebook_id" : {
"type" : "keyword"
},
"facebook_url" : {
"type" : "keyword"
},
"facebook_username" : {
"type" : "keyword"
},
"first_name" : {
"type" : "keyword"
},
"full_name" : {
"type" : "keyword"
},
"gender" : {
"type" : "keyword"
},
"github_url" : {
"type" : "keyword"
},
"github_username" : {
"type" : "keyword"
},
"id" : {
"type" : "keyword",
"index" : false
},
"industry" : {
"type" : "keyword"
},
"inferred_location_names" : {
"type" : "keyword"
},
"inferred_salary" : {
"type" : "keyword"
},
"inferred_years_experience" : {
"type" : "integer",
"doc_values" : false
},
"interests" : {
"type" : "keyword"
},
"is_frankenstein" : {
"type" : "boolean",
"doc_values" : false
},
"job_company_facebook_url" : {
"type" : "keyword"
},
"job_company_founded" : {
"type" : "keyword"
},
"job_company_id" : {
"type" : "keyword"
},
"job_company_industry" : {
"type" : "keyword"
},
"job_company_linkedin_id" : {
"type" : "keyword"
},
"job_company_linkedin_url" : {
"type" : "keyword"
},
"job_company_location_address_line_2" : {
"type" : "keyword"
},
"job_company_location_continent" : {
"type" : "keyword"
},
"job_company_location_country" : {
"type" : "keyword"
},
"job_company_location_geo" : {
"type" : "geo_point",
"doc_values" : false
},
"job_company_location_locality" : {
"type" : "keyword"
},
"job_company_location_metro" : {
"type" : "keyword"
},
"job_company_location_name" : {
"type" : "keyword"
},
"job_company_location_postal_code" : {
"type" : "keyword"
},
"job_company_location_region" : {
"type" : "keyword"
},
"job_company_location_street_address" : {
"type" : "keyword"
},
"job_company_name" : {
"type" : "keyword"
},
"job_company_size" : {
"type" : "keyword"
},
"job_company_ticker" : {
"type" : "keyword"
},
"job_company_twitter_url" : {
"type" : "keyword"
},
"job_company_type" : {
"type" : "keyword"
},
"job_company_website" : {
"type" : "keyword"
},
"job_last_updated" : {
"type" : "keyword"
},
"job_start_date" : {
"type" : "keyword"
},
"job_summary" : {
"type" : "text"
},
"job_title" : {
"type" : "keyword",
"fields" : {
"text" : {
"type" : "text"
}
},
"ignore_above" : 256
},
"job_title_levels" : {
"type" : "keyword"
},
"job_title_role" : {
"type" : "keyword"
},
"job_title_sub_role" : {
"type" : "keyword"
},
"languages" : {
"properties" : {
"name" : {
"type" : "keyword"
},
"proficiency" : {
"type" : "integer",
"doc_values" : false
}
}
},
"last_name" : {
"type" : "keyword"
},
"linkedin_connections" : {
"type" : "integer",
"doc_values" : false
},
"linkedin_id" : {
"type" : "keyword"
},
"linkedin_url" : {
"type" : "keyword"
},
"linkedin_username" : {
"type" : "keyword"
},
"location_address_line_2" : {
"type" : "keyword"
},
"location_continent" : {
"type" : "keyword"
},
"location_country" : {
"type" : "keyword"
},
"location_full_address" : {
"type" : "keyword"
},
"location_geo" : {
"type" : "geo_point",
"doc_values" : false
},
"location_last_updated" : {
"type" : "keyword"
},
"location_locality" : {
"type" : "keyword"
},
"location_metro" : {
"type" : "keyword"
},
"location_name" : {
"type" : "keyword"
},
"location_names" : {
"type" : "keyword"
},
"location_postal_code" : {
"type" : "keyword"
},
"location_region" : {
"type" : "keyword"
},
"location_street_address" : {
"type" : "keyword"
},
"middle_initial" : {
"type" : "keyword"
},
"middle_name" : {
"type" : "keyword"
},
"mobile_phone" : {
"type" : "keyword"
},
"personal_emails" : {
"type" : "keyword"
},
"phone_numbers" : {
"type" : "keyword"
},
"profiles" : {
"properties" : {
"id" : {
"type" : "keyword"
},
"network" : {
"type" : "keyword"
},
"url" : {
"type" : "keyword"
},
"username" : {
"type" : "keyword"
}
}
},
"regions" : {
"type" : "keyword"
},
"skills" : {
"type" : "keyword"
},
"street_addresses" : {
"properties" : {
"address_line_2" : {
"type" : "keyword"
},
"continent" : {
"type" : "keyword"
},
"country" : {
"type" : "keyword"
},
"full_address" : {
"type" : "keyword"
},
"geo" : {
"type" : "geo_point",
"doc_values" : false
},
"locality" : {
"type" : "keyword"
},
"metro" : {
"type" : "keyword"
},
"name" : {
"type" : "keyword"
},
"postal_code" : {
"type" : "keyword"
},
"region" : {
"type" : "keyword"
},
"street_address" : {
"type" : "keyword"
}
}
},
"summary" : {
"type" : "text"
},
"twitter_url" : {
"type" : "keyword"
},
"twitter_username" : {
"type" : "keyword"
},
"version_status" : {
"properties" : {
"contains" : {
"type" : "keyword"
},
"current_version" : {
"type" : "keyword",
"index" : false
},
"previous_version" : {
"type" : "keyword",
"index" : false
},
"status" : {
"type" : "keyword"
}
}
},
"work_email" : {
"type" : "keyword"
}
}
}
```
# All Field Descriptions
[See this doc](/docs/fields).
## Related topics
- [Person Search API](/docs/person-search-api.md)
- [Quickstart - Person Search API](/docs/quickstart-person-search-api.md)
- [Reference - Person Search API](/docs/reference-person-search-api.md)
- [Examples - Person Search API](/docs/examples-person-search-api.md)
- [FAQs - Person Search API](/docs/faqs-person-search-api.md)