People of color and the poor bear an inordinate amount of negative consequences related to environmental degradation, pollution, natural and technological disasters (Harlan et al. 2015; Pellow and Brehm 2013; Roberts and Parks 2007). This fact is problematic for the concept of resilience, which is based upon the desire to maintain socio-ecological systems in balance after having faced external shocks. An important underpinning assumption of ecological resilience theory states that systems are adaptable and can return to an environmental state of balance, making them resilient. We argue in this chapter that the application of such an assumption to social resilience requires us to ask: resilient for whom? A social resilience theory has to acknowledge and address the inherent inequalities that are structured into current socio-ecological systems. Urbanization is particularly problematic for resilience, because it is taking place so quickly in many places, exerting pressure on cities to accommodate the growth without fully comprehending its consequences. As a result, cities – places well-known to be bastions of inequality (UN-HABITAT 2010; UNICEF Bangladesh 2010) – often adapt to external shocks in ways that actually reinforce inequality and structure vulnerability in ways that are difficult to reverse. Our goal in this chapter is to unpack resilience theory to highlight its key weaknesses in the face of environmental injustice. We argue that an infusion of critical theory is necessary to adapt what was originally an ecological theory to the human resilience context, which often results in severe exploitation of the many for the benefit of increasingly fewer elites.