AI Expo 2019: Tim Jurka (LinkedIn, Director Feed AI) - Part 4 of 4

I recently attended the AI Expo 2019 at the Santa Clara Convention Center. Notes are from my understanding of the talk. Any errors are mine and mine alone.

LinkedIn: A look behind the AI that powers the LI feed

Tim Jurka (Dir. Feed AI)


The talk was focused on the objectives of LinkedIn's Feed. The talk was focused to a high level (exec) audience. While I was familiar with the space, the objective function formulation and presentation was interesting:


The recommendation problem for LinkedIn is maximizing Like/Comment/Share CTR + downstream network activation (virals) + encouraging new creators.

Problem Formulation:

P(click) + P(viral) * (alpha_downstream + alpha_creator * e ^ (- decay * E[num_response_to_creator])

alpha_downstream accounts for downstream effects; alpha_creator penalizes popular creators to induce diversity.


General approaches (Toolbox):

Multi Objective Optimization (ads vs organic content).

Logistic Regression: Features, Embeddings and Decision Trees (XGBoost for Feature Importance), occasional pruning 

Auto tuning of weights of the MOO to correct for drifts in accuracy of the component models (to meet a product goal).


Running the Team:

The goal of the team is to maximize: Successful Experiments / Total Experiments Run

Two approaches: maximize successful experiments, minimize unsuccessful experiments.


Maximize successful experiments:

1. Hire the best talent

2. Increase the total number of experiments being run online.

3. Automate deployments, parameter tuning, retraining, rebasing, ramping to maximize developer throughput.


Minimize unsuccessful experiments:

1. Replay models over historical data to figure out whether they would perform better than the current model before moving to online.

2. Compute actual business metrics, Determine precision @ 1, precision @ top3/top5 over a randomized sample of data.

3. Use bandits to figure out how to be intelligent about collecting data and exploiting the current model.

4. AI for Ai: Auto retrain and evaluate models. Identify promising features and ramp online. Find optimizations for existing models automatically. Highlight promising variants to engineers.

Comments

Popular posts from this blog

AJAX और हिंदी

Coke Studio: Madari English Meaning and Lyrics

Sadi Gali - Punjabi Lyrics and Meaning (in English) - Tanu Weds Manu

Solved? LaTeX Error: pdf file is damaged - attempting to reconstruct xref table

Tune Meri Jaana Kabhi Nahin Jaana - Lonely (Emptiness) - IIT Guwahati - Rohan Rathore