This tex t describes a physical approach to color image understanding. It uses an intrinsic reflection m odel th a t accounts for the effects of shading and highlight reflection in color images. Com bining th is model w ith a cam era m odel and w ith a geom etric analysis of the relationship between object shapes and color cluster shapes, this work is able to dem onstrate th a t a theoretical, physical model can be transform ed into a very successful com putational procedure for analyzing color images. T he resulting color image in terpretation system segm ents color images along m aterial edges - ignoring color changes due to highlights and shading - and separates the images into their intrinsic reflection com ponents.