We incorporate the endogenous feedback loop at the core of the seminal Schelling (1969) model of segregation into a dynamic model of neighborhood choice and use it to study the forces that shaped racial and income segregation in the San Francisco Bay area from 1990 to 2004. Such an analysis requires causal identification of households' responses to the socioeconomic composition of their neighbors. We achieve this with novel instrumental variables that can be rationalized with a dynamic choice model with frictions. These IVs have potentially broad application: studying sorting along any observable demographic dimension, estimating the effects of neighborhood composition on outcomes such as house prices, or identifying other network externalities. We find that discriminatory (taste‐based and statistical) sorting by race and by income is widespread and complex: almost all households respond positively to similar neighbors and negatively to different neighbors, although at varying degrees of intensity. In spite of these discriminatory responses, frictions—moving costs and uncertainty—mitigate their impact on segregation. This implies that sorting on the basis of other neighborhood amenities may have a large impact on segregation and may justify place‐based desegregation policies. Because of these frictions, there is also scope for desegregation policies based on the reallocation of households to succeed.