OPEN TALK: How to Define Your Limits Proactively and Achieve Stable and Observable Services

OPEN Expo Discovery Stage
Join on Hopin

Will Morgan
Foursquare, Director of Engineering

In his role as Director of Engineering, Will serves as the technical lead for Foursquare's suite of Targeting products. He also leads design for the architecture of a petabyte-scale data processing workflow and related microservices. 

While modern architectural patterns encourage us to declare services in a scale-agnostic way, the fact remains that our software will always have limits. If we choose to define, document, and enforce reasonable limits based on the problem domain, then our users will benefit in diverse ways. Developers deliver services that are more stable, predictable, and easy to use. The developers will benefit directly as well, because the process of imposing constraints, such as resource quotas or rate limits, gives natural scaffolding for observability.

This talk draws from lessons learned over seven years running petabyte-scale data pipelines and their associated microservices at Foursquare.