We examine the determinants of COVID-19 risk exposure in the context of the initial wave in New York City. During the beginning of the first wave of the pandemic, out-of-home activity related to commuting was strongly associated with COVID-19 cases at the ZIP code level and hospitalization at an individual level. After layoffs of workers decreased commuting, case growth continued through household crowding. A larger share of individuals in crowded housing, or commuting to essential and frontline work, are Black, Hispanic, and lower-income—which contributes to disparities in disease risk. As a result, our paper shows that structural socio-economic inequalities help determine the cross-section of COVID-19 risk exposure in urban areas.