Full text of DOI:10.1109/38.814562

Vision 2000
Im ag inat ion
Am p lificat ion
improvements have resulted in images such as Figure 1
(right), a Florence Courtyard.
Since their inception, ray tracing and radiosity meth-
ods have not changed fundamentally in their basic algo-
rithm, although improvements in the 1990’s have made
them much faster and more widely applicable. Radios-
ity methods have given way to radiance methods that
t would be easy to gaze into the distant also capture nondiffuse interreflection. Stochastic and
future and speculate what image synthesis hybrid methods have brought together the best of both
I
technology may be. Since Star Trek with its Holodeck families of algorithms. These changes, along with the
has already done this far better than I can, prognosti- steady increase in computational power, bring global
cating will not be my main focus here. Instead, I hope illumination methods to a point where they could be
to relate some of my own inspiration for getting used on a wide scale. However, they aren’t.
involved with computer graphics, an inspiration that is
still an unrealized dream, one that I’m uncertain how to Th e frust rat ion
fully achieve.
Why aren’t global illumination algorithms used rou-
My interest in computer graphics can best be summed tinely? The most obvious answer is that even with an
up as a desire to create tools for imagination amplifica- unbelievably inexpensive modern graphics card, they’re
tion. I suspect I did not invent the term imagination still too slow. We still cannot render scenes of interest-
amplification, but I don’t remember the source.
ing complexity at interactive rates.
We all have thoughts about things that don’t exist.
Computational costs aren’t the only barriers to wider
These may range from simple musings, to architectural acceptance of global illumination algorithms. As algo-
designs, to industrial design problems. We all have rithms become more sophisticated, the requirements
drawn on the backs of envelopes or napkins to try to con- imposed on the models to be rendered increase. Surface
vey a sense of our thought process. Drawing not only geometry and surface properties need to be more fully
communicates, it also helps to clarify thinking. This and more carefully defined. Radiosity methods often
process of creating images from ideas I call imagination require subdividing surfaces in particular ways to make
amplification. Computers can help.
the algorithm produce accurate images. Both ray trac-
The computer can, in the form of an algorithm, be ing and radiosity methods break when slight errors are
given information about how light reflects from objects, included in the geometry, leading to ugly artifacts. Lights
how materials bend and bounce, how ink flows on a represented as point sources are no longer adequate. In
piece of paper, how cameras create images from light, practice, those designing virtual sets for games or movies
and more. Thus, from only partial information, the com- often prefer the simpler control possible when lighting
puter can simulate (to some approximation) the physics isn’t realistic. Shadows may not even be a desirable fea-
of materials, paint, and light to generate images we ture. We cannot expect the majority of computer graph-
couldn’t create on our own. The computer can show us ics users to go to the trouble to create accurate models.
how things will change when the input changes.
Michael F.
Cohen
Microsoft
Research
Realistic image synthesis has made tremendous A t em p orary refug e
progress over the past three decades. The first major
Recent years have seen the rise of a new paradigm for
body of work on raster images of synthetic objects came rendering images known as image-based rendering. IBR
from the University of Utah in the 1970s. To understand represents a marriage of sorts between computer vision
just how indebted we are to this time and place in his- and computer graphics. While computer vision tries to
tory, see http://www.cs.utah.edu/dept/history/. If the create a machine that can look at the world and form an
1970s can be considered the decade of local illumina- accurate mental model of it, computer graphics tries to
tion, the 1980s were the decade of global illumination. create a machine capable of taking such mental models
Ray tracing and radiosity algorithms provide the abili- and turning them into realistic images.
ty to capture the subtleties of interreflection and shad-
Unfortunately, if a state-of-the-art computer vision
ows. Figure 1 (left) shows a radiosity image I helped system is shown a complex scene, it will at best deliver
create 10 or so years ago at Cornell University.1 Recent a very coarse approximation (for example, it would not
54
January/ February 2000
0272-1716/00/$10.00 © 2000 IEEE

1 The left image known as the Cornell Box1 demon-
strates the color bleeding effects of interreflection
faithfully simulated by the radiosity method. Andrzej
Zarzycki modeled the right image of the Florence
Courtyard (Piazza ss. Annunziata) rendered using
Lightscape radiosity software.
2 Sunflowers and stream. These images are snapshots
from layered depth images.2 The viewer can, in real
time, render this and nearby views. The LDIs were
created and rendered by Jonathan Shade. The data was
supplied courtesy of Oliver Deussen.
capture hairs on a stuffed animal). Even if it did, image
The accelerating progression of ever-higher compu-
synthesis algorithms wouldn’t be able to deal with such tational and rendering speeds at ever-lower costs will, in
rich detail in anything close to real time. the not too distant future, bring real-time ray tracing
IBR combines these computer vision and graphics within reach of an average PC user. This will funda-
technologies by asking the question, what can I do to mentally change the look we expect from computer
render new images directly from a set of images plus graphics.
limited geometric information that computer vision
I also hope this will inspire a newly energized effort at
algorithms can supply? In this way, we are freed from developing tools for physical simulation. The power of
having to create detailed mathematical representations procedural modeling and physically based animation
of the world, yet we can potentially create very realistic and rendering will help expand our ability to explore
images. Lumigraphs2 are one example, constructed from new ideas. However, none of these advances will put the
multiple images. The Lumigraph, much like a digital power of imagination amplification in the hands of non-
hologram, is a unified representation of what an object specialists. Better, easier, more intuitive tools need to be
looks like from all directions.
built to help us express our imagination. One of my
The sunflower and stream images shown in Figure 2 favorite Siggraph papers of recent years is Igarashi et
were rendered in real time from another data structure al.’s, “Teddy.”4 This paper presents the type of intuitive
referred to as a layered depth image.3 An LDI falls interface to modeling that, if extended more broadly,
somewhere between a Lumigraph and an image. It dif- will bring the ability to create imaginary worlds much
fers from a simple image in that it contains, for each closer to everyone.
pixel, a list of color values plus depth. The depth value
In addition to new technologies, people need to be
provides the means to induce parallax as the viewpoint empowered to participate in the use of new tools. Edu-
changes. As you move your head from side to side, things cational curricula should provide more focus on the
far away will stay still, while nearby objects will move kinds of skills needed to leverage this new computa-
back and forth in your visual field. Multiple, per-pixel, tional power. The focus on traditional reading and writ-
color-depth pairs fill in the gaps created when an occlud- ing skills should be expanded to include formats that
ed surface becomes visible.
are interactive and make increasing use of multimedia.
Given all the interesting recent work in IBR, why title We don’t even have verbs to fully describe either the
this section “a temporary refuge”? IBR provides a tan- process of authoring or viewing such rich documents.
talizing means to capture very complex real-world Whatever the name we choose for these processes, we
geometry plus lighting and a way to leverage slow glob- need to teach these skills.
al illumination methods. However, IBR provides only
■
limited help with imagination amplification. IBR by its
nature doesn’t help us ask what-if questions. Having to
start from images to create new images puts us in a References
chicken-and-egg bind.
1. M. Cohen and D. Greenberg, “The Hemicube,” Proc. Sig-
graph, ACM Press, New York, 1985, pp. 31-40.
2. S. Gortler et al., “The Lumigraph,” Proc. Siggraph, ACM
Press, New York, 1996, pp. 43-54.
3. J. Shade et al., “Layered Depth Images,” Proc. Siggraph,
ACM Press, 1998, pp. 231-242.
4. T. Igarashi et al., “Teddy: A Sketching Interface for 3D
Freeform Design,” Proc. Siggraph, ACM Press, New York,
1999, pp. 409-416.
Th e h op e
Despite the frustration just expressed, I’m still both
amazed about the progress computer graphics has made
to date and very hopeful for its future. Recent movies,
such as Jurassic Park, Titanic, and Star Wars Episode One,
have relied heavily on computer graphics rendering with
great success. I would challenge anyone to spot when
the dinosaurs are pure rendering and not filmed physi-
cal models.
Contact Cohen at mcohen@microsoft.com.
IEEE Computer Graphics and Applications
55