AI Expo 2019: Tim Jurka (LinkedIn, Director Feed AI) - Part 4 of 4
LinkedIn: A look behind the AI that powers the LI feed
Tim Jurka (Dir. Feed AI)
The talk was focused on the objectives of LinkedIn's Feed. The talk was focused to a high level (exec) audience. While I was familiar with the space, the objective function formulation and presentation was interesting:
The recommendation problem for LinkedIn is maximizing Like/Comment/Share CTR + downstream network activation (virals) + encouraging new creators.
P(click) + P(viral) * (alpha_downstream + alpha_creator * e ^ (- decay * E[num_response_to_creator])
alpha_downstream accounts for downstream effects; alpha_creator penalizes popular creators to induce diversity.
General approaches (Toolbox):
Multi Objective Optimization (ads vs organic content).
Logistic Regression: Features, Embeddings and Decision Trees (XGBoost for Feature Importance), occasional pruning
Auto tuning of weights of the MOO to correct for drifts in accuracy of the component models (to meet a product goal).
Running the Team:
The goal of the team is to maximize: Successful Experiments / Total Experiments Run
Two approaches: maximize successful experiments, minimize unsuccessful experiments.
Maximize successful experiments:
1. Hire the best talent
2. Increase the total number of experiments being run online.
3. Automate deployments, parameter tuning, retraining, rebasing, ramping to maximize developer throughput.
Minimize unsuccessful experiments:
1. Replay models over historical data to figure out whether they would perform better than the current model before moving to online.
2. Compute actual business metrics, Determine precision @ 1, precision @ top3/top5 over a randomized sample of data.
3. Use bandits to figure out how to be intelligent about collecting data and exploiting the current model.
4. AI for Ai: Auto retrain and evaluate models. Identify promising features and ramp online. Find optimizations for existing models automatically. Highlight promising variants to engineers.