Strange Loop

A Commerce-centric Approach to Queuing Fairly at High Throughput

When is a throttle more than a rate limiter? Historically, Shopify mitigated write-heavy traffic bursts up to 5x our baseline throughput via rate-limiting scripted in Nginx Lua modules at ingress on our load balancers. That solution served us well for years in scaling for some of the world's largest E-commerce flash sales. It also had drawbacks. Edge Tier overload protection divorced from Application Tier business logic meant inflexibility in testing, maintainability, & improving waiting room UX. High traffic on one shop could be throttled disproportionately from one load balancer to another. Users could wait 30 minutes, only to discover that their cart's inventory had gone out-of-stock 20 minutes prior. Lessons learned in moving from "off-the-shelf rate limiting" to "business-aware user queueing" broadly apply to any domain where traffic bursts could trigger a waiting room. This talk also covers our load testing & migration strategy in moving throttling away from the edge to our Rails monolith application tier.

Logan Martel

Logan Martel - Staff Developer - Checkout Performance @ Shopify

Logan is a staff developer at Shopify, where he works on scaling checkout performance to host some of the world's largest flash-sale thundering herds. Growing up in a cold remote-town in "true northwest" British Columbia (Kitimat), Logan decided he couldn't get enough of the cold when he moved to Montreal to earn a BSc. (Hons) Software Engineering at McGill University in Montréal, QC (and later, remotely study his M.S. CS from Georgia Tech) where he remains as of 2022. When he's not chasing down cache stampede and stray N+1 SQL calls (or context-switching to Data Science), Logan enjoys frigidly reading non-fiction at coffee shops and being "pretty good" (but not as good as his girlfriend) at Beat Saber.