Matching Niche B2B Expertise with Natural Language Processing (NLP)
A peek into the technology that helps us match our clients with experts
The expert network business is, at its core, very simple. Clients need insights that are in-depth, specific, and otherwise difficult to access. We deliver that insight by connecting them with niche experts, whom we source and match for each specific project.
But finding the best experts to match with each project isn’t easy. Traditionally there’s a huge amount of manual searching, vetting, outreach, and human-to-human interaction involved. As a result, it’s historically a white-glove service, where expert networks often keep as much as seventy percent of the price paid by the client.
Today, expert networks deliver this service with some blend of manual human effort and software. For some providers, it’s perhaps 90% manual and 10% software-driven. In these instances, the expert network manages a database of experts and has a team of people sifting and searching to find the right expert for each project.
For DeepBench, software plays a much larger role — allowing our team to be much more efficient and effective in their work. By pairing natural language processing (NLP) systems with our team of Client Associates, we are able to deliver better fit experts more quickly with less overhead.
Our matching process is broken down into three steps:
- Collecting project information from the client.
- Delivering a recommended pool of candidate experts.
- Refining/extending that recommended pool.
Collecting Project Information
The client first opens a new project by describing what they’re hoping to learn. The more specificity the client can provide, the better. Granular topics, key areas of expertise, geographies, screening questions, and work experience all help us make more accurate matches in less time.
Once we have the project information, the machine learning system gets to work. It starts by scrubbing the consolidated text provided by the client, removing common words (such as ‘the’, ‘is’, ‘and’, etc.), and saving the remaining keywords in a “bag”.
However, not all keywords are equally descriptive of the project. We leverage the bags of words against a larger pool of words derived from thousands of other projects run to date to give each keyword an importance weighting — the less frequently a keyword appears in other texts, the more unique and important it is to the project at hand.
We compare how similar two pieces of text are by some function of the overlap between the words in each piece of text and the associated weights from our larger pool of words. The higher that number is, the more similar the two pieces of text are. Given the peculiarities of language, we also take into account redundancies based on word stems (e.g., noting overlaps between ‘tall’ vs. ‘taller’ vs. ‘tallest’, etc.).
For any given project, we can check if we have similar projects using the method above. If we find a project that is similar, the advisors on it may be good candidates for the project at hand.
With the same techniques mentioned above, we can also match project descriptions directly to DeepBench profiles that contain titles, organizations, and job descriptions.
Delivering a Pool of Expert Candidates
Concurrent with the NLP processes working in the background, we have a team of Client Associates augmenting the software’s output. Including a human-in-the-loop allows us to catch potential errors in expert sourcing, optimize for desired client outcomes, and clarify any nuanced details to experts and clients as needed.
With this, we assemble a curated pool of candidate experts and hand them off to the client. The client may take one of the following actions: upvote/downvote the profile, message the expert or request a consultation with them. We use these actions to identify which experts were strong matches. This helps us refine our suggestions by finding other experts that have similar characteristics to the strong matches.
Extending The Recommended Pool
The algorithm powering this process becomes more effective over time. With each new project created, and with each new expert registered, the amount of data the algorithm has access to continues to grow. This in turn expands its capability for comparison and analysis, thereby allowing us to deliver better expert suggestions.
One of the exciting features we are exploring at DeepBench is dynamically building out our advisor profiles. If a client starts a project that is tagged with ‘Sales & Marketing’ and upvotes or requests a meeting with an advisor that does not have that tag, that advisor is tagged with that topic and this mechanism could lead another client with a similar project focus to be connected with the advisor.
The expert network space is an incredibly exciting industry to bring commercial applications of artificial intelligence to market.
DeepBench is thrilled to continue expanding the capabilities of this market and delivering exceptional experiences to clients and experts alike.
