ABSTRACT
We investigate the number of dichotomies realizable with multi-layer neural networks. For n-real-input two-hidden-layer neural networks with h 1 units in the first hidden layer and h 2 in the second (h ≡ h 1 + h 2), where the units use the linear threshold function (Heaviside function), we show that the networks can implement between ∑ i = 0 ( h 1 h 2 + n h 2 ) / 2 ( N i ) https://s3-euw1-ap-pe-df-pch-content-public-p.s3.eu-west-1.amazonaws.com/9781315784076/744dcdac-ef6b-4709-bdaa-84955f763c9b/content/eq1000.tif"/> and ∑ i = 0 ( h 1 h 2 + n h ) ( log h + 2 log log h ) ( N i ) https://s3-euw1-ap-pe-df-pch-content-public-p.s3.eu-west-1.amazonaws.com/9781315784076/744dcdac-ef6b-4709-bdaa-84955f763c9b/content/eq1001.tif"/> dichotomies for almost any set of N input points but they can implement between 2 ∑ i = 0 h 1 h 2 + n h 2 ( N − 1 i ) https://s3-euw1-ap-pe-df-pch-content-public-p.s3.eu-west-1.amazonaws.com/9781315784076/744dcdac-ef6b-4709-bdaa-84955f763c9b/content/eq1002.tif"/> and 2 ∑ i = 0 h 1 h 2 + 2 n h ( N − 1 i ) https://s3-euw1-ap-pe-df-pch-content-public-p.s3.eu-west-1.amazonaws.com/9781315784076/744dcdac-ef6b-4709-bdaa-84955f763c9b/content/eq1003.tif"/> dichotomies for some set of N input points with infinite Lebesgue measure.
