Raphael's parents were Richard Thompson Semmes, of French origin, and Catherine Middleton, the daughter of a wealthy southern Maryland family. Uncle Raphael enthralled his nephew stories of his heroic naval service during the War of 1812. Young Raphael later attended the prestigious Charlotte Hall Military Academy in St. Mary's City, a school famous for its training of naval officers. He became a midshipman in the United States Navy in 1826. During Mexican War, Semmes served ably and came to attention of higher-ups. In Washington, his outgoing wife thrived, the argumentative Semmes fell out with Matthew Fontaine Maury of the Oceanographic and Geodetic Service but made a favorable impression on Secretary of War Jefferson Davis On February 14, 1861, ex-Senator Clay asked to come to Montgomery, Alabama, and on February 15, he resigned from the US Navy. The European governments complained that the Union Navy could not protect their vessels. Still, President Abraham Lincoln continued to trade on the Atlantic.