Strange Loop

Building and Deploying Scaleable NLP Model Services

In this workshop, we'll walk through the process of deploying state-of-the-art NLP models as scaleable, fault tolerant API microservices using kubernetes and the Seldon Core framework. We'll begin by wrapping an existing model artifact in a Seldon Core microservice, including model loading and prediction, and configuring REST and GRPC endpoint services. We'll then discuss Seldon core deployment configuration options for kubernetes deployments while deploying a model service on a local kubernetes test environment. We'll finish up by using kubernetes tooling for monitoring resource utilization for the model service.

This workshop will make use of the following packages and frameworks, providing a brief introduction to each: * Huggingface transformers * Seldon Core * Kind (Kubernettes in Docker) * K9s

Zak Brown

Balto Speech and Language Processing

A Physicist by education, Zak has been focusing on interesting problems in natural language processing and information retrieval since 2015. He's worked on a variety of projects ranging from ML-based ranking for search to real-time entity extraction for conversational AI systems. In his spare time, he enjoys playing music, and camping with his family