AI Expo 2019: Tim Jurka (LinkedIn, Director Feed AI) - Part 4 of 4

I recently attended the AI Expo 2019 at the Santa Clara Convention Center. Notes are from my understanding of the talk. Any errors are mine and mine alone.

LinkedIn: A look behind the AI that powers the LI feed

Tim Jurka (Dir. Feed AI)

The talk was focused on the objectives of LinkedIn's Feed. The talk was focused to a high level (exec) audience. While I was familiar with the space, the objective function formulation and presentation was interesting:

The recommendation problem for LinkedIn is maximizing Like/Comment/Share CTR + downstream network activation (virals) + encouraging new creators.

Problem Formulation:

P(click) + P(viral) * (alpha_downstream + alpha_creator * e ^ (- decay * E[num_response_to_creator])

alpha_downstream accounts for downstream effects; alpha_creator penalizes popular creators to induce diversity.

General approaches (Toolbox):

Multi Objective Optimization (ads vs organic content).

Logistic Regression: Features, Embeddings and Decision Trees (XGBoost for Feature Importance), occasional pruning 

Auto tuning of weights of the MOO to correct for drifts in accuracy of the component models (to meet a product goal).

Running the Team:

The goal of the team is to maximize: Successful Experiments / Total Experiments Run

Two approaches: maximize successful experiments, minimize unsuccessful experiments.

Maximize successful experiments:

1. Hire the best talent

2. Increase the total number of experiments being run online.

3. Automate deployments, parameter tuning, retraining, rebasing, ramping to maximize developer throughput.

Minimize unsuccessful experiments:

1. Replay models over historical data to figure out whether they would perform better than the current model before moving to online.

2. Compute actual business metrics, Determine precision @ 1, precision @ top3/top5 over a randomized sample of data.

3. Use bandits to figure out how to be intelligent about collecting data and exploiting the current model.

4. AI for Ai: Auto retrain and evaluate models. Identify promising features and ramp online. Find optimizations for existing models automatically. Highlight promising variants to engineers.