Strange Loop

Machine Learning for Relevance and Serendipity

Careful use of well-designed machine learning systems can transform products by providing highly personalized user experiences. Unlike hand-tuned or heuristic-based personalization systems, machine learning allows for the use of millions of different potential indicators when making a decision, and is robust to many types of noise. In this talk, I will discuss our deeply-integrated use of machine learning and natural language processing for content discovery at Prismatic. Our real-time personalization engine is designed to give our users not just the content they expect, but also a healthy dose of targeted serendipity, all based on relevance models learned from users’ interactions with the site. We use sophisticated machine learning techniques for topical classification of stories, to determine story similarity, make topic suggestions, rate the value of different social connections, and ultimately to determine the relevance of a particular story for a particular user. I will go into detail describing our personalized relevance model, starting with a description of our problem formulation, then discussing feature design, model design, evaluation metrics, and our experimental setup which allows quick offline proto-typing without forcing users into the role of guinea pig. Our model’s combination of social cues, topical classification, publisher information, and analysis of the user’s prior interactions produces highly-relevant and often delightfully serendipitous content for our users to consume.

Jenny Finkel

Prismatic

Jenny Finkel is Chief Software Architect at Prismatic, a ground-breaking news discovery app that is rapidly growing into its second year of expansion. She studied for an undergraduate degree in Computer Science at Columbia, followed that up with an MS and PhD in Computer Science at Stanford, and then did a post-doctoral fellowship at MIT and Columbia. During the course of her academic career she published seventeen papers on machine learning and natural language processing, and created the widely-used Stanford Named Entity Recognition software package. She segued directly from education into her role at Prismatic as one of the original members of a burgeoning team that has multiplied in numbers in the relatively short time since the company was founded.