October 15, 2009
Tony DeRose is currently a Senior Scientist and head of the Research Group at Pixar Animation Studios. He received a B.S. in physics in from the University of California, Davis, and a Ph.D. in computer science from the University of California, Berkeley. From 1986 to 1995, DeRose was a professor of computer science and engineering at the University of Washington. In 1998 he was a major contributor to the Oscar-winning short film Geri's Game. In 1999 he received the ACM SIGGRAPH Computer Graphics Achievement Award, and in 2006 he received a Scientific and Technical Academy Award for his work on surface representations.
Ivars Peterson: What do you do at Pixar?
Tony DeRose: I lead the Research Group at Pixar. It was founded about five years ago. Lots of people innovate in the company, but we're the only group that's supposed to innovate all the time. Most of the people who are innovating in the company are working on particular movies, so they work on new technology, algorithms, mathematics that might be particularly appropriate to that film. But there are a lot of problems that aren't appropriate for them to work on because the time horizon for the problem is too long or the risk is too high, and a producer may not want to bet their budget on that particular project working out. So, we look for those kinds of opportunities.
IP: What is the research group like?
TD: The group is small. It's six Ph.D.s—they're the full-time researchers—plus about five interns. They're very senior, very self-directed. So the amount of time I spend managing is very small. Mostly I try to keep up with what they're doing. I can spend 60 or 70 percent of my time on my own research.
IP: What particular areas are you researching?
TD: Traditionally, I've done geometry—constructive differential geometry. But many of those issues are pretty well understood for most of what we want to do now. It turns out there are some bigger opportunities in other areas. I've been starting to look at human-computer interaction problems. This is interface work, bringing in novel input devices to see if we can make movie-making tools more fun and flexible, going beyond mice.
The basic question we're asking ourselves is: We've been making movies with traditional workstations for decades, with mouse and keyboard. But that's unlikely to be the ideal movie-making workstation. So, what is it? We have a unique opportunity at Pixar because we write virtually all our own software, and all our users are in house. If we can identify really innovative and effective workstation designs, we can build the software to use them and deploy them in the company.
IP: How often do you get to speak about what you're doing to audiences outside Pixar?
TD: Generally speaking, the whole group tries to get out to talk a reasonable amount. We encourage publishing and being productive members of the research community, serving on program committees, and so on. In addition, I've been interested in finding ways that Pixar and Pixar and Disney together can add a little bit of excitement, energy, and relevance to math and science education, particularly at the middle and high school levels.
I give a few talks a year along those lines, and I've been brainstorming with a lot of people trying to identify initiatives that we might undertake.
IP: You grew up in Pacific Grove, California. When you were young, was there anything that you did, liked, or knew about that would have pointed to what you are doing now?
TD: Only in the broadest terms. Ever since I can remember, I enjoyed science. It was only in junior college that I really discovered that I love math. I had been big into physics and chemistry before that. I ended up majoring in physics as an undergrad at Davis, partly because I was looking at it more as applied math than anything else. I had always known somehow that I wanted to do research, so grad school was a given.
When I was thinking about what I wanted to do for grad school, the two things I loved most at that point were physics and, because of a summer internship that had introduced me to the very beginnings of computer graphics, computer science. I was left with the choice of improving on 400 years of brilliant physicists or jumping in essentially on the ground floor in computer graphics to see what I could do. I chose computer graphics and went to Berkeley. Happily, my advisor, Brian Barsky, had just arrived that same fall; he was a fresh Ph.D., and he was working on surface representations, splines, that sort of thing. We really hit it off. That's where I really found the beauty of doing constructive mathematics, and computer graphics was a way to make it visual.
IP: What sparked you interest in mathematics in college?
TD: It was the incredible power of calculus. I just had no idea that you could mathematically analyze such incredible situations. That really fascinated me.
IP: Was there an aesthetic component to your interest in computer graphics?
TD: It was more algorithmic. I never felt that I ever had an aesthetic sense. I always considered myself more of a technologist. So I never anticipated that I would be making movies. That connection came about because LucasFilm was just across the bay and was really just beginning at the point—about 1981. George Lucas had hired Ed Catmull to start a graphics group at LucasFilm, and that was right about the same time that Brian and I got to Berkeley, and Ed agreed to serve on my Ph.D. committee. I got to know him as part of that, and we kept in touch.
So, about 10 years after I graduated, Pixar was just about to release Toy Story, and it looked like it was going to be successful. I was at a point in my career that, if ever I was going to try something else, that was a good time. I had never heard of anything that was nearly as interesting as trying to help Ed and the others at Pixar figure out how to make films.
I was at the University of Washington at that point, building a research group and doing the standard computer science professor thing. My research was mostly surface representation and other geometric problems motivated by computer graphics.
When I left Washington and joined Pixar, the first project was a short film called Geri’s Game. This was right after the release of Toy Story. Ed realized that a lot of aspects of Toy Story worked really well. The story was great; the toy characters were believable, compelling. But anytime a human came on screen, it was kind of jarring. It was clear we were going to have to improve our ability to tell stories with humans.
I worked with Michael Kass, a physical simulation expert. He worked on Geri's clothing, and I worked on Geri's skin and his body forms. Then Jan Pinkava was brought in as the director. That was about a two-year project. It really offered the opportunity to develop subdivision surfaces, which is the technique I had been working on most recently at Washington, and really make that work in a production environment. When that film was released, it won an Academy Award, which was very nice.
The next four years I spent building the software system internal to Pixar that artists use to create our characters. As the characters move and deform, the shapes are changing in very complex ways, so the challenge there was to build an interactive system that artists—nonmathematicians—could use to create these complex behaviors of animated characters.
After that, I spent a few years working on the underpinnings of the system. I got really interested in software engineering at that point, then moved to the research group. That's when it started up.
Prior to that time, we knew how to do so little of the movie-making process that the productions were really aggressive about accepting new technology. But as time went on, we learned how to do more and more, budgets were going up, the risk of failure was going up, so the productions were naturally becoming more conservative technically than they had been earlier. And that was appropriate. So it was becoming more and more difficult to have the same kind of fundamental technology shifts in the context of a production. It was just too risky. When that trend really took root, that was the time to create the research group.
Our job is to look at these risky problems and develop them to the point where we understand them well enough that we can reasonably, safely put them into production. Or sometimes we decide that this won't be ready anytime soon, so let's put it on the shelf.
IP: Pixar films seem built on innovation.
TD: The first leap was being able to make a computer graphics movie at all. That was Toy Story. That was probably the most incredible leap. A Bug's Life was challenging because it was the first story that we told with lush outdoor environments, so there was a lot of scale and complexity that hadn't been present before. Monsters, Inc. was challenging because it was the first full-length film in which we used simulated clothing. And there was also a lot of hair. Finding Nemo was challenging because of the water, both the simulation of the water at the surface but also the surge and swell that you feel in the reef, the caustics, the lighting effects. The Incredibles was challenging because there were 80 human characters. Some of the people who are in our group now were responsible for really making the muscle systems work far better, which is particularly important for characters like Mr. Incredible, who has a lot of muscle mass. Making all that move in a credible fashion was a big challenge.
IP: How important is the physics in this kind of computer graphics work for representing physical things in a physical world?
TD: We typically don't do Newtonian physics. We do Pixarian physics. Physics is a starting point for us, and then we make all kinds of approximations in order to make the final algorithms and techniques more directable rather than physically accurate. One of our team members is a former professor of atmospheric sciences. He's a wonderful physicist. He really understands dynamical systems, particularly water and air. He's got a lot of insight about the fundamental physics, and is able to translate that into algorithms that aren't physically accurate but capture the interesting visible attributes and are typically more directable.
There have been insights from subdivision surfaces, for instance, that have filtered back into the mathematics community. It turns out the subdivision and wavelets are intimately connected. Anytime you have one, you have the other. That insight and connection has propelled both disciplines forward.
IP: What do you see 10 years from now?
TD: I ask myself that question a lot. My crystal ball is cloudy for that kind of time horizon. The thing that drives me the most is the sense of discovery. Whatever it is that I'm doing, I hope it gives me the opportunity to have interesting intellectual insights myself rather than managing a group to help others to do it.
Graphics is a lot more mature than it was when I started. The big gaps between what we can imagine and what we can actually accomplish have shrunk considerably in the last 20 years. I'm starting to look more broadly at some of the problems that are occurring in the Disney Corporation. Disney acquired us a few years ago, and one of the things that has happened since the acquisition is that we've started to build up research groups in other parts of Disney. If you look at other business units in Disney, there's a lot of low-hanging fruit. Pixar has enjoyed 25 or 30 years of concentrated, consistent technical development. That's true to a lesser extent in other parts of Disney. That's why the research groups are starting.
The intent is to have a lot of collaborative projects between people at Pixar and people at Imagineering, for instance. Disney also owns ESPN, ABC, a bunch of games studios, so the range of problems that we can start looking at now is much broader. There are going to be a lot of interesting developments both within some of those disciplines and also at the seams.
We had one project that was collaborative between our group and a group at Imagineering and another group at Pixar to build a physical WALL-E robot. He was by far the most expressive robot that we've seen because of the Pixar understanding of what it means to animate something, together with some control algorithms that our folks provided.
IP: What are you involved with in trying to promote math and science education?
TD: Right now, doing talks, such as "Math in the Movies," for teacher organizations and other groups. We're hoping to make the talk available on the Web and build some curricular activities for use in classrooms. We've also hosted a few math festivals. It does seem like there's an opportunity here, so I've been trying to talk to as many people as I can find about other things we might try to do.
One thing I would like to see in the middle and high schools, and this carries into calculus education as well, there is such an emphasis on calculation at all the levels, and very little emphasis on what I think of as mathematics, which is pattern matching, looking for patterns, forming hypotheses, developing problem-solving strategies. That doesn't seem to be taught much at all. When I took calculus, there was a huge emphasis on calculation and not so much on concept.
Read about Tony DeRose's Distinguished Lecture
Papers by Tony DeRose