Haavelmo’s probability approach is predicated on a division between a priori deductive model building by economists and a posteriori model estimation and testing by econometricians. This chapter shows this division is unattainable. What the reality tells us is that practically useful econometric models can only be learnt via iterative processes involving both deductive and inductive reasoning. Now, the theory of machine learning, specifically the principle of probably approximately correct (PAC) learning, offers a viable basis for such learning processes. The basics of machine learning theory are described, and how they can enlighten us on econometric model formulation issues is also briefly discussed.