Nature conservation and protection of biodiversity meant fencing off pieces of nature to mitigate human influence. This approach neglected the role of local farming communities in shaping many unique ecosystems and maintaining a wide range of species and varieties of animals and plants. In the Cuchillas del Toa MaB reserve, the interface between natural ecosystems and agricultural landscapes has not been sufficiently investigated yet. Through 37 household surveys and participatory farmer workshops, it was investigated why farmers grow certain varieties and why the cultivation of others is being abandoned. The results show that the number of species and varieties maintained by farmers has decreased in recent years. This for different reasons. Lack of access to education and medical care foster rural migration. A diminishing work force and a poor road infrastructure decrease the options to commercialize farm products, focusing only on a few crops to sell. Thus, the rapid erosion of farmers’ traditional knowledge is also accelerating the reduction of diversity, in a feedback process. To counteract this the development of basic infrastructure needs to be improved, and the farmer’s contribution to the conservation of agrobiodiversity be recognized and valued by policy makers and markets.