Searching by job_title

It can be daunting to tackle a dataset as large as the PDL Person dataset, and learning to properly query the job_title field is one of the trickier skills to hone. PDL standardizes the job titles it ingests, but the underlying values still come from messy, real-world human input.

Standardizing `job_title` values

PDL lowercases every job_title value and expands common acronyms:

Raw input	Standardized to
`CEO`	`chief executive officer`
`VP`	`vice president`
`Sr.`	`senior`
`Jr.`	`junior`

So for example, you don’t need to search job_title for the literal string "CEO", since it’s already been expanded to "chief executive officer" through our standardization processes.

Why `job_title` is hard to search

The biggest reason why searching on the job_title field can be challenging is the sheer variety of ways people describe their roles. The same role gets spelled out hundreds of different ways. The chart below groups every job title by how many person profiles share that exact string. For example, the “2-99” row is every title that shows up on somewhere between 2 and 99 profiles and compares that split two ways: as a share of unique titles, and as a share of person profiles. The two sides mirror each other: 85.5% of unique titles occur exactly once, but those one-off titles account for roughly 11.6% of profiles. Flipping it around: titles occurring 5,000+ times are only 0.01% of unique titles, yet they cover over 56.9% of profiles. The variety is real, but it’s concentrated in a long tail of rarely-seen titles; instead, most profiles land on a comparatively small set of common job titles.

See the exact counts behind the chart

`job_title` frequency	Person profiles	Unique job titles
1	65,000,000	65,000,000
2-99	68,000,000	10,775,000
100-999	53,000,000	190,000
1000-4999	55,000,000	26,000
5000+	318,000,000	9,000

Comparing search techniques

Let’s take a search for software engineers as an example. The cards below show that the way you phrase your search can significantly impact the number of results you get.

Each bar below is sized relative to the largest record count of the three (5.6M)

While exact and wildcard searches on job_title work, it’s usually easier to cast a wider net with PDL’s job_title_role, job_title_sub_role, and job_title_levels fields.

Using `job_title_role` and `job_title_sub_role`

PDL maps job titles to a canonicalized list of job title roles and job title subroles, using keywords in job_title and company industry to capture the many different titles that describe the same role. Over 60% of profiles with a listed job_title are tagged with a job_title_role. For example, here are some of the titles that fall under job_title_role = "engineering" and job_title_sub_role = "software":

software developer
system engineer
software test engineer

software development manager
software architect
ios developer

In most cases, PDL has already done the work of tagging job titles to a role and subrole, so you don’t need to guess every variation of a title a person might use. Instead, you can cast a wide net with the role/subrole taxonomy, then narrow in with the skills, job_summary, headline, or summary fields for a specific specialty. For example, to find someone who writes Ruby code:

Example: Searching for Ruby developers

from peopledatalabs import PDLPY

CLIENT = PDLPY(api_key="YOUR API KEY")

ES_QUERY = {
    "query": {
        "bool": {
            "must": [
                {"term": {"job_title_role": "engineering"}},
                {"term": {"job_title_sub_role": "software"}},
            ],
            "should": [
                {"term": {"skills": "ruby"}},
                {"wildcard": {"job_summary": "*ruby*"}},
                {"wildcard": {"headline": "*ruby*"}},
                {"wildcard": {"summary": "*ruby*"}},
            ],
            "minimum_should_match": 1,
        }
    },
    "size": 100,
}

response = CLIENT.person.search(query=ES_QUERY, size=100).json()

curl -X POST \
  'https://api.peopledatalabs.com/v5/person/search' \
  -H 'X-Api-Key: YOUR API KEY' \
  -H 'Content-Type: application/json' \
  -d '{
    "query": {
      "bool": {
        "must": [
          {"term": {"job_title_role": "engineering"}},
          {"term": {"job_title_sub_role": "software"}}
        ],
        "should": [
          {"term": {"skills": "ruby"}},
          {"wildcard": {"job_summary": "*ruby*"}},
          {"wildcard": {"headline": "*ruby*"}},
          {"wildcard": {"summary": "*ruby*"}}
        ],
        "minimum_should_match": 1
      }
    },
    "size": 100
  }'

PDL has some blind spots in its job title tagging. Relying only on job_title_role and job_title_sub_role is not a viable strategy for very specialized or niche roles — pair it with a wildcard or free-text search for those cases.

Using the Job Title Enrichment API

The Job Title Enrichment API is useful for expanding a search when you don’t have a good sense of the title variations you’re looking for. It returns a cleaned title plus a list of contextually-similar titles and skills as shown below, but it comes with some important limitations as well.

Example Response from Job Title Enrichment API

JSON

{
  "status": 200,
  "data": {
    "cleaned_job_title": "pediatric nurse",
    "similar_job_titles": [
      "pediatric registered nurse",
      "pediatric nurse practitioner",
      "pediatric licensed practical nurse",
      "nursery nurse",
      "pediatric intensive care nurse"
    ],
    "relevant_skills": [
      "pediatric nursing",
      "nursing",
      "pediatrics",
      "patient safety",
      "patient education"
    ]
  }
}

The important caveat to keep in mind here is that the API only returns titles that appear in PDL’s global resume data at least 100 times. So if you rely solely on the values it returns, you’re missing out on roughly 24% of profiles, and the long tail of the rarer title variations described above. Here’s how a search for "pediatric nurse" plays out across each technique. Each bar below is sized relative to the largest match count of the four (56,000):

Wildcards are powerful, but they slow queries down. PDL allows a maximum of 20 wildcards per search.

Below is a further breakdown of the top titles in behind that last, broadest search (Role + subrole + wildcard), where you can see both the gain and the cost of this approach:

Top titles Role + subrole + wildcard search

“Other” is every remaining title the search over-matched into, not a single specific one.

The Job Title Enrichment API beats a plain wildcard search, but it still misses some real variations. The broadest, role/subrole-plus-wildcard search returns nearly double the results of the API-driven search, but includes some irrelevant profiles along with it.

Choosing an approach

Which technique to reach for depends on how familiar you are with the titles you’re targeting, and your tolerance for over- or under-matching:

PDL’s RecommendationDefault to combining the role/subrole taxonomy with a targeted wildcard or skills search (see example above). It’s the best balance of recall and precision, as long as you spot-check for over-matched profiles.

Exact match

Best for: you know the precise, common phrasingTrade-off: misses every variation and synonym

Wildcard match

Best for: you know a distinctive substringTrade-off: misses titles without that substring; counts against the 20-wildcard limit

Role + subrole

Best for: broad recall with minimal guessworkTrade-off: blind spots for niche or highly specialized roles

Job Title Enrichment API

Best for: you don’t know the title variations up frontTrade-off: only surfaces titles seen 100+ times, missing ~24% of profiles

Guides

Best Practices

Recipes

Tutorials

Standardizing `job_title` values

Why `job_title` is hard to search

Comparing search techniques

Using `job_title_role` and `job_title_sub_role`

Example: Searching for Ruby developers

Using the Job Title Enrichment API

Example Response from Job Title Enrichment API

Top titles Role + subrole + wildcard search

Choosing an approach

Exact match

Wildcard match

Role + subrole

Job Title Enrichment API

​Standardizing job_title values

​Why job_title is hard to search

​Comparing search techniques

​Using job_title_role and job_title_sub_role

Example: Searching for Ruby developers

​Using the Job Title Enrichment API

Example Response from Job Title Enrichment API

Top titles Role + subrole + wildcard search

​Choosing an approach

Exact match

Wildcard match

Role + subrole

Job Title Enrichment API

Standardizing `job_title` values

Why `job_title` is hard to search

Comparing search techniques

Using `job_title_role` and `job_title_sub_role`

Using the Job Title Enrichment API

Choosing an approach