How I helped a ML focused product team go from underperforming to over achieving

I worked onsite at Facebook headquarters in Menlo Park, CA for 7 months in 2018. I embedded within a team that builds internal recruiting tools. As a consultant, my job was to uncover the reason for low adoption of their first machine learning feature. I would help revise product strategy and work with the team to build and execute on a new roadmap.

The challenge: Engineering-driven product team had launched a ML feature that was met with low enthusiasm and adoption.

Step 1: Understand

• The team’s priorities and strategy: the team was measured on their ability to increase claims and recruiter efficiency.
• The product and how it worked: I worked with the cross functional team to do a deep dive on the existing workflow, features and user experience.
• The users: I partnered with user research to design a series of research studies to model user workflow and pain points.

User research was key to understanding the problem. We had plenty of data on how users engaged with the web application. But we lacked a comprehensive picture of the sourcing workflow, especially tasks that were happening offline and in other channels. This generative research provided new insights on recruiter pain points and priorities.

Step 2: Identify the gaps

A clear picture began to emerge about why ML was unsuccessful:

• Trust and users’ mental model were the primary barriers to adoption.
• How ML worked was unclear. The product required users to choose between search and ML. Since they were familiar with search and unsure how ML worked, they stuck with what they knew.
• User intent was unknown. For ML to be effective, we need a problem statement — a clear signal as to what the user is trying to do.

Step 3: Revise product strategy

We revised our product strategy to address these unmet needs.

• I led multiple design workshops to synthesize our findings and to brainstorm solutions.
• Through roadmapping and creative briefs I helped focus the team’s efforts in new directions.
• We used prototypes and concept testing to validate a new approach.
• I presented the results to executives, creatives and cross functional teams, working across product silos to create a unified experience.

Results: We launched our first user centric feature, using more iterative and research based methods, and saw dramatic results.

• 130% increase in claims within 30 days
• By year’s end, we had a 10X increase in claims from ML features
• The team exceeded it’s yearly goal by 9%, despite a rocky start