This chapter discusses the gender violence and what is termed sexual diversities. It outlines the fundamental distinctions between the two theoretical fields, the core concepts of psychoanalysis are infantile sexuality and the Oedipus/castration complex, which ground the concept of sexual difference. The chapter analyses whether there is any guiding concept that determines interpretations of “difference”. It highlights S. Freud’s works the presence of growing complexity in relation to sexual and gender difference since both the concept of bisexuality and the complete Oedipus complex represent a decentring from the position that considers the positive Oedipus the ideal solution. The chapter proposes to find other paths for thinking about sexual difference beyond the risks presented by extreme culturalism or pure biological explanations. It explains that there is masculine/feminine gender difference “previous to” access to sexual difference. The chapter discusses that dichotomous polarities do not encompass the complexity of processes of construction of sexed subjectivity since positions may be mixed or multiple.