The Ribeira (riverside) area represents the most symbolic place of colonialism in Lisbon. It served as the literal point of departure for Portuguese overseas conquests, the establishment of the oldest European colonial empire and the enslavement of c. six million Africans. The consequences of this difficult past deeply mark the character of the Authorized Heritage Discourse and still dominate the Portuguese mythscape. By contrast, decolonizing initiatives promoting counter-narratives call for a plural history that can lead us to recognize subaltern memories and help construct more just societies. In this context, life stories can reveal the entanglement between memory and cognitive justice related to social and political inequalities and racism in today’s Portugal. This chapter analyses five life stories collected in Lisbon in January 2020 and reflects on how the colonial past and related narratives have an impact on the present. It elaborates on the challenge of embracing a critical understanding of the history and culture of Africa in Portuguese academia, society and politics. The resulting analytical matrix demonstrates that, together with the ‘decolonization of sight’, counter-narratives enable what we define as the ‘decolonization of hearing’.