This chapter discusses the hardware design of type classification in the computer-aided diagnosis (CAD) system for colorectal endoscopic images with narrow-band imaging magnification is proposed for real-time processing of full high-definition image. A pyramid-style image segmentation with support vector machines for multi-size scan windows, which can be implemented on a field-programmable gate array with small circuit area and achieve high accuracy, is proposed for actual complex colorectal endoscopic images. The CAD system for colorectal endoscopic images with narrow-band imaging magnification has already been proposed. The decision function calculation module is implemented using 2-pipelined stage architecture. The proposed pyramid-style classifier with SVMs for multi-size and multi-layer scan windows enables smaller circuit area and higher throughput with higher classification accuracy than the conventional methods for actual complex colorectal endoscopic images. The sum-of-product calculation in the decision function is computed by sum-of-product calculator blocks which are orange-color in parallel.