Groundwater use has been proposed as a solution to increase water security in semi-arid areas worldwide. However, little information is available on the role of groundwater in supporting agricultural livelihoods in many countries in semi-arid areas of sub-Saharan Africa, and the opportunities to expand this role in the future. Here, we provide an overview of the benefits of using groundwater to improve water security in Tanzania. The analysis employs data generated under the GroFutures project, involving 90 households between 2015 and 2017 in three villages in the Usangu Plains of the Upper Great Ruaha River Catchment in southern Tanzania. We adopt a sequential exploratory research design to collect quantitative and qualitative data. The analysis shows that most smallholder farmers depend on groundwater and that investment in groundwater had a positive cost-benefit ratio, mainly due to the ability to produce an additional dry-season crop. Performance decreased with well depth but remained positive at 100 m for irrigation of 1 ha. This outcome calls for increased efforts by local and national governments along with development partners to unlock the potential of groundwater to improve livelihoods and food security.