Semi-autonomous cars are already on the road, and highly autonomous cars will soon be with us. Little is understood about how drivers will adapt to the changing relationship with their vehicle, but to ensure safety and consumer acceptance, this insight is vital. To this end, an on-road study in a semi-autonomous vehicle was undertaken with six UK drivers. The ‘think-aloud’ technique was employed, and video and audio footage of their interaction with the vehicle was captured. Neisser’s (1976) Perceptual Cycle Model (PCM) was used to analyse the data, and three case studies are presented to highlight how poor synergy between driver and semi-autonomous vehicles can occur from the lens of schema, action, or world information. Seven key design considerations are proposed to ensure a more positive and safer interaction between driver and autonomous vehicle to guide focus by manufacturers. Further evidence for the existence of a ‘counter cycle’ (Plant and Stanton, 2015) within the PCM is found, and how this relates to the challenges of using verbal protocols expressed during a fast-moving dynamic task is discussed.