Abstract: “Nobel prize winner M. G. Lippmann described his dream of an ideal display as a “window into the world.” “While the current most perfect photographic print only shows one aspect of reality, reduced to a single image fixed in a plane, the direct view of reality offers, as we know, infinitely more variety.” Changing the observer’s viewpoint reveals perspective changes in object size and location. Moreover, changing the environment lighting can vary appearance substantially by shifting highlights and cast shadows. These effects are extremely important for shape and material perception, and thus for realism. Despite significant recent advances in multiscopic display technology, the majority of these displays are still limited in an important respect: one can only display a scene under the same illumination conditions in which it was captured. If the illumination in the observers environment changes during playback, there is no corresponding effect on the shading, highlight positions, or cast shadows that they witness on the display. In this talk I will survey a sequence of projects we carried in the last years, constructing several light-sensitive displays with different levels of complexities. Our ultimate goal is to build a 3D light-sensitive display, capable of presenting viewpoint-sensitive depth content as well as spatially-varying material reflectance properties that will accurately react to the interaction between environment lighting and scene geometry.”