Shreya Shankar is a computer scientist, PhD student in databases at UC Berkeley, and co-author of “Operationalizing Machine Learning: An Interview Study”, an ethnographic interview study with 18 machine learning engineers across a variety of industries on their experience deploying and maintaining ML pipelines in production.

Shreya explains the high-level findings of “Operationalizing Machine Learning”; variables that indicate a successful deployment (velocity, validation, and versioning), common pain points, and a grouping of the MLOps tool stack into four layers. Shreya and Lukas also discuss examples of data challenges in production, Jupyter Notebooks, and reproducibility.

Show notes (transcript and links): http://wandb.me/gd-shreya

—
⏳ *Timestamps: *
0:00 Intro
0:53 Shreya’s background in industry and academia
8:59 Findings from “Operationalizing Machine Learning”
19:05 Examples of data challenges in production
27:13 Shreya’s thoughts on Jupyter Notebooks
34:14 Shreya’s thoughts on ML tooling
40:42 ML tooling layers, tech stacks, and workflows
51:33 Provenance and evaluating different solutions
54:12 Outro

—

📝 *Links*
📍 “Operationalizing Machine Learning: An Interview Study” (Shankar et al., 2022)”, an interview study on deploying and maintaining ML production pipelines – https://arxiv.org/abs/2209.09125

“Understanding Black-box Predictions via Influence Functions” (Koh and Liang, 2017), on using influence functions to better understand the output of a model – https://arxiv.org/abs/1703.04730

—

*Connect with Sarah:*
📍 Sarah on Twitter: https://twitter.com/sarahcat21
📍 Sarah’s Amplify Partners profile: https://www.amplifypartners.com/inves…

—

💬 *Host:* Lukas Biewald

—

*Subscribe and listen to Gradient Dissent today!*
👉 Apple Podcasts: http://wandb.me/apple-podcasts​​
👉 Google Podcasts: http://wandb.me/google-podcasts​
👉 Spotify: http://wandb.me/spotify​

source