Reinforcement learning (RL) is a powerful general-purpose technique for automatically selecting actions that maximize rewards. At a surface level, RL seems like a perfect fit for solving several problems in data management, such as query optimization ("choose a plan that minimizes latency"), execution engines ("pick the right operator for my context"), and workload management ("schedule my queries such that SLA violations are minimized"). Unfortunately, putting RL into core database components in practice has several drawbacks, such as sample inefficiency and exploration overhead. This talk will begin with a case study in "lessons learned" from deploying learned query optimizers at Microsoft and Meta, and will then present our lab's current cookbook for successfully integrating RL into data systems. This cookbook boils down to two core principles: (1) move expensive exploration offline, and (2) ensure the overhead of the RL technique being used matches the potential gains from good decision-making.
Bio: Ryan Marcus is an assistant professor at the University of Pennsylvania, where he works on building next-generation database management systems that automatically adapt to new hardware and user workloads, invent novel processing strategies, and understand user intention. Before joining Penn, Ryan was a postdoctoral researcher at MIT, and received his PhD from Brandeis University. His work has received multiple awards, including a Google ML and Systems Junior Faculty Award, a SIGMOD Best Paper Award, and recognition in SIGMOD Research Highlights. You can read more about Ryan on his website: https://rmarcus.info
Zoom: https://cam-ac-uk.zoom.us/j/2644961338?pwd=PJrfDcVwjXyE4PNpSjznN9YYQYzqre.1&omn=83089748605
