It is the goal of artificial intelligence and robotics to develop au tom atic system s (robots) th a t are able to ac t in and in teract w ith a com plex en vironm ent and dem onstrate a perform ance th a t is a t least com parable to th a t of hum ans. One of the im portan t subtasks in this field is to gather inform ation abou t the environm ent dynam ically through sensing devices and to interpret this inform ation so th a t the robot can (re)act accordingly. In th is area, com puter vision has concentrated on gathering and in terp ret ing visual inform ation. Im age understanding involves interpreting two dim ensional images which are projections of three-dim ensional scenes. The goal is to ex tract useful inform ation from the images to plan (re)actions. Such inform ation includes a description of the scene as a collection of shiny and m atte surfaces, sm ooth and rough, in teracting w ith light, shape, and shadow. U nfortunately, light interacts in m any com plex ways w ith m a t ter, producing optical effects such as shadow casting, object shading and highlights, and due to the inform ation lost during the projection process, com puter vision has not yet been successful a t deriving an appropriate de scription of such surface and illum ination properties from an image. The m ain reason has been a lack of models rich enough to relate pixels and pixel-aggregates to scene characteristics. Furtherm ore, m ost im age anal ysis concentrates on interpreting only intensity d a ta in black-and-w hite images. Such images do not provide enough information to model optical
effects. T his book presents an approach to com puter vision th a t uses color inform ation to in terpret the effects of shading and highlights in a scene. It exploits a physical reflection m odel which relates shading and highlights to color variation in the im age.