This article examines how second-generation Filipinos understand their panethnic identity, given their historical connection with both Asians and Latinos, two of the largest panethnic groups in the USA. While previous studies show panethnicity to be a function of shared political interests or class status, I argue that the cultural residuals of historical colonialism in the Philippines, by both Spain and the USA, shape how Filipinos negotiate panethnic boundaries with Asians and Latinos, albeit in different ways. Filipinos cite the cultural remnants of US colonialism as a reason to racially demarcate themselves from Asians, and they allude to the legacies of Spanish colonialism to blur boundaries with Latinos. While the colonial history of Filipinos is unique, these findings have implications for better understanding racialization in an increasingly multiethnic society – namely, how historical legacies in sending societies interact with new racial contexts to influence panethnic identity development.