Low and stagnating crop yields are attributed to low soil moisture and nutrient reserves and poor soil structure, as well as the overall low fertility of soils in sub-Saharan Africa. Nutrient cycling and improved fallow management can enhance soil fertility and also improve soil physical properties through integrated soil fertility management. The management of soil physical properties and processes is essential to improving soil chemical fertility and biological attributes. The proportion of macro-aggregates decreased significantly in soil that was under crop cultivation, and especially so with the removal of crop residues. The process of soil physical degradation is set in motion by the decline in soil structure and the proportion, size, and strength of aggregates. A system-based conservation tillage is an appropriate option of sustainable management of the fragile soils under the harsh climate of sub-Saharan Africa. The decline in aggregation and the increase in soil bulk density and penetration resistance also affect water retention and transmission pores.