We document that racial disparities in COVID-19 in New York City stem from patterns of commuting and housing crowding. During the initial wave of the pandemic, we find that out-of-home activity related to commuting is strongly associated with COVID-19 cases at the ZIP Code level and hospitalization at an individual level. After layoffs of essential workers decreased commuting, we find case growth continued through household crowding. A larger share of individuals in crowded housing or commuting to essential work are Black, Hispanic, and lower-income. As a result, structural inequalities, rather than population density, play a role in determining the cross-section of COVID-19 risk exposure in urban areas.