October 15, 2009
The rippling muscles of Mr. Incredible in The Incredibles, the exquisitely detailed surfaces of Ratatouille, the effortlessly draped garb of Monsters, Inc., the expressive robots of WALL-E, and the sparkling splashes and eerie lighting of Finding Nemo all owe their power to mathematical innovation.
"There is indeed a lot of mathematics behind the scenes," computer scientist Tony DeRose of Pixar Animation Studios noted in his recent talk on "Math in the Movies." In each of these animated films, constructed entirely on computers, trigonometry helps rotate and move characters, algebra creates the special effects that make images shine and sparkle, and integral calculus helps light the scenes.
DeRose's talk was part of the MAA's Distinguished Lecture Series, which is supported by the National Security Agency.
Using examples from Pixar productions to illustrate his points, DeRose started by giving his audience a glimpse of how computer-generated animated films are made. The process can take up to four years from start to finish, with about half of the time spent on refining the story.
"It all begins with the story," DeRose emphasized. Before computers come into play, artists first create thousands of hand-drawn pen-and-paper sketches or, more recently, Photoshop images and quick animations to flesh out the story.
"As the story starts to gel, the art department starts to fire up . . . to create characters and designs for the worlds the characters live in," DeRose continued. These blueprints and models become the basis for building the digital characters, a process akin to constructing physical marionettes with strings to govern their motions. The digital strings serve as animation controls, taking the form of dials that allow animators to define the motions of the characters, from bending an elbow to raising an eyebrow.
"We typically end up with about 300 degrees of freedom in these characters and perhaps 700 controls," DeRose said. "Imagine a marionette with 700 strings; that's what our animators are dealing with."
Subsequent steps produce the highly detailed sets, color and light the scenes, and generate the final rendering.
DeRose went on to offer several case studies of where different kinds of mathematics are used for different aspects of a film, from the simulation of how light bounces around in an environment (integral calculus) to obtaining smooth surfaces efficiently (subdivision surfaces) and making characters move in a realistic fashion (harmonic coordinates).
"It shouldn't be at all surprising that mathematics plays some role," DeRose said. The resulting images are all digital. Three numbers define each pixel, one each for the amount of red, green, and blue. "Since we're getting a big bucket of numbers out, and all we have basically is numbers in—positions of geometry, values of degrees of freedom," DeRose said, "there's got to be some big computation in the middle."
Global illumination involves simulating how light bounces around in an environment. "Mathematically, for every point y and for every other point z, we need to compute how much light is traveling from y toward z, and we have to do that for every pair of points in the environment," DeRose said. In a continuous setting, this turns into an integral—the rendering equation.
Approximating the integral equation as a set of linear equations—one equation for every piece, with typically 1 million to 10 million pieces—we have that many equations to solve for every frame of the film, DeRose said.
The geometric technique of subdivision surfaces allows artists to represent complex surfaces and complicated shapes efficiently on a computer, while maintaining an illusion of smoothness. Under many iterations of this split-and-average process, a faceted model converges to a smooth surface that can be easily animated.
"There's a lot of flexibility in this general framework," DeRose said. "It's new mathematics inspired by the film industry." The technique of subdivision surfaces was used for the first time in animation to create the character Geri in the short film Geri's Game (1999), which won an Academy Award.
To create and control volume deformations used to articulate characters, Pixar has recently turned to harmonic coordinates. DeRose indicated that harmonic coordinates are generalized barycentric coordinates that can be extended to any dimension; they describe how interior points move within polygons in the plane or polyhedra in space.
"You can reformulate barycentric coordinates as a boundary-value problem for Laplace's equation," DeRose said. "Solve the differential equation and you get coordinates that behave just as you want. As far as I know, it's a novel use of Laplace's equation."
This technique greatly simplifies the controls necessary to animate a character, DeRose added. "This has the potential to really reduce the time and effort that [animators] expend moving things around."
Tony DeRose is a Senior Scientist and head of the Research Group at Pixar Animation Studios. He received a B.S. in physics in from the University of California, Davis, and a Ph.D. in computer science from the University of California, Berkeley. DeRose was a professor of computer science and engineering at the University of Washington.
An Interview with Tony DeRose
"Math in the Movies" (Discoveries and Breakthroughs Inside Science)