Kevin Rafferty is a computer graphics supervisor at ILM whose team worked extensively on Draco. We asked him to explain how ILM goes about bringing a mythical being to life.
CG Supervisor Kevin Rafferty Discusses the Making of Draco
"About the same amount of data as four t-rexes from Jurassic Park, made up the data for Draco's wing."Kevin Rafferty
Long before Jurassic Park, Marin-based Industrial Light & Magic had been doing great special effects for the movies. In this summer's crop of movies, ILM's been responsible for the hurricane in Twister, scenes in Mission Impossible that it contractually can't talk about, and a good deal of other cool stuff. From a technical standpoint, perhaps the most impressive recent accomplishment was the creation of Draco, the last of the dragons, the object of Dennis Quaid's knightly quest in Dragonheart. Draco is a virtual character who interacts with the real ones, and incidentally has the voice and mannerisms of Sean Connery. He is a being many generations removed from a cheesy plastic Godzilla smashing scale-model Tokyo. Kevin Rafferty is a computer graphics supervisor at ILM whose team worked extensively on Draco. We asked him to explain how ILM goes about bringing a mythical being to life.
Mary: What was your mission, and what were the technical challenges that came with it?
Kevin: Our mission was the greatest challenge, in that we wanted to create a character that was not only believable but had subtle nuances of the human actor who was providing the voice.
Mary: How well-defined was the character on paper before you started?
Kevin: We started out with a sculpture, a scale model with moving armature so we could spread the wing out and move the arms and legs and open the mouth. It was a pretty detailed study of what they wanted to see, and it went through quite a few iterations of how they wanted Draco to look in pre-production. At first he had a longer snout than he ended up having...
Mary: In the movie he almost looks like a Persian cat!
Kevin: Right, because we wanted to go away from the normal idea of what a dragon was, and most of the time you do see this longer, wolfish kind of snout on a dragon. We thought shortening the nose would help distinguish the different character and help us get him to emote better.
Then as we developed the lip-sync software, the in-house software that we developed for the film, we started making minor adjustments so we could get that little Sean Connery crook of the eyebrow and what have you.
The main design of the character did come early.
Mary: You just had to make it real.
Mary: So how did you go about that?
Kevin: It's a very long process.
We wanted to add as much geometric information as possible on Draco, to have actual real nooks and crannies around his forehead, and veins and things like that. Normally this would be done with texture mapping, which is projecting a picture on the geometry to gain certain feelings of rising and falling on the skin.
We wanted to add in the actual geometry, to give it one step more towards realism.
Once we have the 3D model that's animated, we light it. It's almost like we're virtual set lighters in that we bring Draco into the scene, and put key lights and fill lights on him, and make sure the leathery skin texture maps are working correctly, to make it fit in with the live action. We go away at night and let the computers render frame by frame all of the animation. It scans out each frame based on the work we've done that day, tweaking all the lights... We have the horsepower of the SGIs-we had 12-and 16-processor Challenges as our big render engines, and we also used everybody's workstation at night. We run takes of all of our shots that are in progress every night, and we have dailies every morning. So each night we would have to our avail about 150 CPUS.
And it was never enough!
The size of the data of Draco was amazing-about the same amount of data as four T-Rexes from Jurassic Park made up the data for maybe Draco's wing.
Mary: So you start out with analog video, or film, or what?
Kevin: We start out with film. We get a background plate of film, scan that in frame by frame, 24 frames a second. Then basically we take the models we've built-we have a team of character animators who would choreograph Draco's movements. They would start out by choreographing his gross body movements, making his actual body language fit with the live action actors, using SoftImage on SGIs.
The character animators would give that choreography information, along with the model data, to technical directors, the digital lighting artists, to merge the choreography with the material characteristics that Draco should have to fit seamlessly into the background plate. Each night we send off that render, scan out the lighting information and choreography of Draco, and then composite it into the background.
The character animators will also use our proprietary software we call Caricature to put in all the subtle expressions-not only make the lips move to the words spoken by Sean Connery, but also nudge in all the subtle nuances of facial expression.
All the character animators had character sheets of Sean Connery, pictures from all the movies he's made and all the different emotions he's had. They had a sheet full of his faces in anger, and surprise, and delight, and love. That type of thing. So they always would have visual reference. It was up to the talent of the artist more than any type of scanning and morphing to get those nuances that brought Draco and Draco's face to life. it was an incredibly talented team of people working with really good hardware from SGI and software from other companies as well as inhouse. We used Alias for model-building, SoftImage for animation. Then
We used Renderman for lighting and our own in-house software to embellish SoftImage's animation with our lip-sync software, Caricature. It's actually emotion, lip-sync, shape-building-it's a big megillah! (laughs). And then we have our own in-house compositing software.
For production of movies like Dragonheart, we use pretty much exclusively SGI hardware.
Mary: You compared the amount of data in Draco's wing to the T-Rexes in Jurassic Park. How has the technology evolved since that time?
Kevin: Actually, things have changed in both the hardware and software arenas. SGI, since Jurassic Park, has provided companies like ILM with faster machines....
Mary: But it's never enough!
Kevin: That was going to be my next point. The workstations as well as the rendering engines, the big servers, are faster and faster, but that just gives us the opportunity to put more and more complexity and realism into the motion. No matter how powerful the machine is, we'll find a way to bring it to its knees! [laughter]
Mary: Meanwhile, down in Mountain View, they're taking this as a challenge!
Kevin: Well, yeah! Exactly!
Also, software-wise, with our proprietary stuff, we find it a challenge to say, "Okay, this is too much data for the machines to handle all at once; let's split it up into different tiles." At some point when we had really, really complex scenes, we would have four processors working on one frame, rather than one frame to one processor, which is the normal way we do things.
Mary: I remember from some presentations I've seen on the Jurassic Park dinosaurs that a huge amount of work goes into things like anatomically accurate movement and correct physics and things like that-how the leg moves when the dinosaur hits the ground with all its weight, and things like that. When you're dealing with an animal that never existed, what issues like that apply?
Kevin: That's a very interesting question, and very poignant to Draco, in that the physics applied were the physics the director, Rob Cohen, wanted, because it was a mythical creature.
It depended on the emotion of the shot, the feeling of the shot. At times he wanted Draco to be a magical mythical character who was made not of flesh and bone but of light. And so in that respect, and in those shots, we took the idea of taking suspension of disbelief one step further, seeing this massive creature flit about like he had no weight.
At other times, when it called for him to have a strong presence, and be in, like, a fight scene, we kind of said, "Okay, we wanted him to be made of light, and as such normal gravity and physics do not apply, but in these shots he needs to have strength, and he needs to have mass, and he needs to have gravity." And so in those respects we had to put those things in.
Once again we used our Caricature software to manipulate shapes for, say, the biceps and where the legs meet, the chest area, so when he'd slam his claw down there'd be a little bit of vibration and movement. That took us into another realm of research, which Draco didn't directly benefit from, but he was the catalyst for future research into skin-over-bone techniques-kind of automating that type of jiggle with gravity, and making it work in a character animation world, where the animators still want control. They don't want it really automated, because sometimes when you put in the true physics and stuff it just doesn't perform like you want it to-it's too mechanical. So the character animators still would like the final say, to be able to go in there and tweak things. Those are the type of tools were trying to develop for them.
It's funny that we had to approach both sides of the coin with Draco, and it also spurred on this whole research team working on a series of software items for doing just that.
Mary: So when the characters in the movie are interacting with Draco, are they actually interacting with a bluescreen?
Kevin: There was actually very little bluescreen going on. Most of the time they were out in Slovakia on location, in the middle of a field or what have you, and they would be talking to ping-pong balls. (laughs)
We would have a series of ping-pong balls out in the live-action shot where Draco was going to be, to be able to triangulate and match the live-action camera to the computer camera. We used those things to triangulate data, and then we didn't have to take them out of the plate because we were going to put Draco right on top of it. It served a double purpose in that Dennis Quaid and all the actors who were interacting with Draco could focus on something.
Mary: They didn't have to imagine everything.
Kevin: Exactly. Exactly. And then for times when there were close-ups and stuff, they had a cutout of a profile and a front view of Draco's head. "This is where he's going to be-set your eye-gaze here." Then it would be up to us to make Draco fit into that scene.
If there were objects in the foreground, our rotoscoping team would go in...
Mary: What's rotoscoping?
Kevin: It's actually the digital form of an old technique. It's been around in the optical world for a long time. Say there's a log between the camera and Draco. Since this wasn't done bluescreen or anything, and Draco needs to be behind that log, the rotoscope team will go in and digitally outline that log, frame by frame, and create a matte, so at composite time we can bring in the live action plate, put Draco over the live action plate, and put the log back over Draco.
Mary: And do they also deal with issues like since the sun is in east in the scene, the log has to be lit correctly?
Kevin: No, there was no lighting involved, since it was a live-action log, it was part of the plate. We had to take that into account when we lit Draco-we would have all that information on our camera sheets, we knew what time of day it was, and we had these reference pictures, a man standing in front of a camera with the same camera setup as the scene, with a huge white styrofoam ball that showed us visually where the sun was and how the bounced light was acting. We always had visual references as well as real numbers, because usually neither works well, you have to merge the two together to make something really believable.
Mary: What do you think could have been done better if you'd had infinitely perfect tools?
Kevin: Infinitely perfect tools!
Our saying always is, 'It's never done, you're just out of time.' (laughs)
What I wish we'd had more time doing was there was supposed to be a magical quality to Draco's wings, and we were developing a really beautiful iridescence for his wings. It never worked correctly twice.
Mary: It was beautiful a million times., but never-the same twice?
Mary: Now you just have to turn that into a feature.
Kevin: Exactly! Some might contend, "Well, he's a magical creature, so what?" But when you were cutting from shot to shot, and it was supposed to be there in both shots, it just didn't quite look right. So I wish we'd had more time developing that to make it really, really good. There are other things. We achieved the way the eyes work quite well, but I wish we'd had more time developing how the eyes dilate and possibly get the little musculature in the iris to work more like a real eye would. Those are little things that you'd probably see the fifth or sixth time you saw the show. Those are the things that I myself personally would have loved to have more time working on. I was thinking of the wet look, when Draco was in the lake or the rain and stuff-I think we achieved that quite nicely.
Mary: So what exactly do you do at ILM?
Kevin: On the credits I'm called a Digital Effects Supervisor. At ILM my title is Computer Graphics Supervisor.
What I do is I supervise a team of the technical directors, the people who do all the lighting and bring everything together, bring the mattes from the rotoscope people, bring the choreography and the lip-sync from the animators, and bring the live-action plate in and make them all merge together seamlessly, and put any effects on top that need to happen, like if he's breathing fire or any of those things. That's basically what a technical director does, and I supervise part of the technical direction team, along with two other supervisors.
Mary: What did you do before Draco?
Kevin: I was a CG supervisor on Casper I supervised a lot of the technical directors and helped develop how the ghosts looked on a shot-to-shot basis.
Casper was a magical being too-did he present different challenges from Draco? He was a magical being. However, we could take a lot more liberties with Casper, because he was a ghost, and he was a familiar character. It was a double-edged sword-we had liberties because he was a ghost and amorphous and this cute little blob that ,as flying around in space. But also we had to be true to this cartoon character-he had to be recognized and loved as Casper.
The challenges were different there-how to make something from a comic book come to life in 3D, and be believable and recognizable. And then we had all the amorphous qualities that ghosts have-going through walls, going invisible and still disturbing the air a little bit it was a completely different set of challenges. The one similar challenge was the lip-sync. The lip-sync stuff from Casper was totally different than what Draco had. It was one of those things like Draco and the skin-over-bone-just like Draco was the caralyst for our skin-over-bone research, Casper was the catalyst for the lip-sync program that we used on Dragonheart. It's kind of a nice evolutionary thing-the realized needs of one film will come to full realization, probably, in the next film.
Mary: So what are you working on now?
Kevin: I'm working on pre-production on Jurassic Park II The Lost World.
Mary: Aha! Did you work the first One?
Kevin: No, I came to ILM just as they were finishing the last shots of Jurassic Park I
Mary: Well, there's plenty of legacy stuff from that! [laughter] And on the other hand, the world has moved on quite a bit, Draco's wing suggests-what will you be able to do technically that you couldn't before, without giving away any big secrets?
Kevin: I can say this-our biggest challenge is to be better than the first one!