)
digital head replacement
fxg: Eric, congratulations on The Curious Case of Benjamin Button. You must be very pleased how it all turned out. It's a stunning film.
Eric Barba: I actually am. It's taken a long time to get to that point where I can relax and be happy with it. Now that people are really responding warmly to it, it's great.
fxg: Eric, before we talk about how you created the stunning shots for this movie, can I ask you to take us back to your initial discussion with David Fincher about Benjamin Button. How did you think you would be able to achieve the visual effects in this film?
)
Director David Fincher
At that point David was pushing the digital workflow and we were using the Viper camera. We got really excited about Benjamin Button in about 2004. We were paid to do a test which we had a really short amount of time to do. We built a maquette and it turned out to be the scene where you first see Benjamin Button as a kid sitting at the dining table banging his spoon. For the test we did a head replacement with another actor in about 4-5 weeks, and people thought we were crazy to attempt this. But because he did not have to talk or emote too much we could build it specifically for the shot.
The test went pretty well and it was received very warmly by the studio. I remember showing it to producer Frank Marshall and he through it was the most exciting thing he'd seen since the first dinosaur test walk for Jurassic Park. So then we knew we were onto something. But the movie wasn't greenlit at that point so I went back to doing music videos with David and ultimately Zodiac. When Zodiac finished we found that Benjamin wasn't dead and these commercials came up that were the perfect test bed for Benjamin Button. "
(These were three ads for iconic US Popcorn brand Orville Redenbacher in 2007.The Colonel Sanders of the Popcorn world, Orville was famous for his direct to camera promotional spots that aired on US television in the 70s and 80s. After his death, Fincher used CGI techniques to bring him back for a spot in 2007).
)
Brad Pitt on set
They were using similar techniques that we had used to created the test scene for Benjamin Button in 2004 but these guys had a really good handle on how it all worked. Our use of Mova Contour on the Orville Redenbacher commercials didn't get the warmest reception. Part of the reason was we were doing a guy who was already dead so there was no way to capture a performance from him. We cast a voice that was similar and then different actors to do the face and the body, which ended up as three very disparate performances.
Another problem was that as you art direct a commercial you start to get away from the original performance and you tend to move into that "Uncanny Valley" territory. Doing that commercial was a great learning experience and it also humbled my team that had previously worked with David Fincher to put out some pretty amazing work. Up until that point we thought we had a handle on it and then we did Orville and realized we still had a lot to do to make it all work.
fxg: What was the difference between the approach you developed for the test shot in 2004 and how you achieved the character in Benjamin Button?
)
The first 52 minutes feature no live Brad Pitt performances
In the 2004 test, because we did not have the character doing a major performance, the rigging system was fairly primitive. We knew we'd have to build a much more elaborate system. When we started Orville our character supervisor rigged and animated Orville's performance. We were going to use the Mova Contour data to help us but we had some technical problems and we still needed to do a lot of homework on it. For Benjamin Button we knew what we needed to get out of that system and they knew how to get it all working. Six months later when we got Brad into the rig we were able to get everything we needed, which was basically a series of facial expressions.
The Mova system is designed to get 24fps data and then that can be used to identically recreate that performance on a CGI version of the character. But because we were going to be using Brad Pitt's performance to drive older versions of his physiology, basically retargetting the performance to an older version of himself, that wasn't really going to work for us. And because the Mova capture system requires that the actor is seated and you can't move very much, and yet we needed to put him in an environment where he is moving around and interacting with other actors, there were going to be some limitations on what we could use that data for. So we used the Mova rig to capture Brad's facial shapes and we then built systems to use Brad's performance to drive Benjamin.
)
Brad Pitt performances were achieved with what DD called emotional capture done with the Mova contour capture
Eric Barba: The Mova Contour Capture rig is like a bunch of speed rails that are arrayed for 150 degrees around and the actor sitting in a chair. It holds 23 cameras and they are all aimed at the face which is covered with phosphorescent green make-up. This allows for frame by frame tracking of patterns and each point can be tracked in 3D space.
This is the first system to truly capture someone's face moving in realtime and provide a moving mesh that can be subdivided, rebuilt and then retargeted to another mesh to drive a CGI performance. Since Orville Redenbacher, Mova have been constantly working to refine the Contour capture system and we had been working with them on this. It was a collaborative effort. They were invited in to collaborate on Benjamin Button and like all of the vendors were incredibly excited.
)
Cate Blancett on set
)
Daisy also required digital work to help her dance and age - much of which was done by Hydraulx and Lola visual effects
fxg: So where does the digital Brad begin and end in this film?
Eric Barba: The baby was a mixture of a live action robotic maquette which was enhanced by Hydraulx I believe. But for the first 52 minutes of the film it's a full 3D head, there were 352 individual shots. There's no projection, there's no 2D techniques. Once our work stops about 52 minutes in, Brad takes over in makeup. Ultimately as he gets younger he wears less and less makeup until its just Brad.
As he gets very young Lola did some touchup work on his physical makeup then when he gets back to the Dance Studio Lola takes over doing the younger version of Brad. Ultimately there's a couple of child actors and a baby at the end. The bulk of the work in head replacement happens in the first 52 minutes. We prevized the U-boat battle and I planned it out with David, and I had planned on doing it and we built a bunch of the assets but ultimately we handed that off to Asylum because of time and economics, and I think they did a pretty good job of it.
"The first "digital head" shot is the one we did the test on, where there's a long dolly and pan until the audience sees Benjamin sitting at the table banging his spoon. That's the first body double for Ben in his 80s, as he grows younger we have another body double take over for him in his 70s, when he goes out on the tugboat with Cap'n Mike and goes to the bar. The bulk of our work is the "Ben70" character, and the "Ben60" when he leaves home. One of our last shots is when he is reading the letter from Daisy on the back of the tugboat. The line where he tells the Captain, "Well you do drink a lot", that's where the real Brad takes over.
)
)
fxg: Did the number of head replacement shots grow during the filmmaking process, or was it always intended to be that many?
Eric Barba: David always said that he didn't want to shoot this movie around the fact that it was a CG character. He wanted to shoot it like he was shooting an actor. That means we don't shy away from difficult shots such as seeing him naked in the bathtub or getting a haircut or getting drunk and stumbling. It allowed David to tell the story the way he wanted to tell it which I think is the right way to do it.
fxg: Were there many challenges in the job of attaching the digital Brad to the various bodies used during the live shoot?
Eric Barba: The body doubles had different neck lengths and shoulder sizes and even the arch of the neck was different, so there was a lot of massaging that had to go on to make those heads feel like they belonged. We had to create a tracking system that was far more robust than traditional tracking because traditional tracking doesn't give you accurate Z depth. It doesn't give you a spine which we needed to have so the head would follow the body exactly as it moved around. During the shoot we would have four of the principal Viper cameras running in perfect sync. We devised the software to take any of the different cameras and, once the plates were locked, we could triangulate and retrack to get the accuracy we needed. This was a huge undertaking but we learned with Orville that there wasn't anything out there that could do the tracking the way we needed to have it, so that you buy the illusion every time.
)
)
fxg: In many of the shots the Brad Pitt character is wearing a hat or glasses? Did that present additional challenges?
Eric Barba: My intention when we started shooting was to always shoot Benjamin with the actor wearing a hat so that the body was shadowed correctly. My character supervisor was really worried about that approach but I thought that it would help with the illusion, so the audience would not know where the plate stopped and Benjamin started. Most of the hats in the movie are the live hat.
There were some issues with lineups where Brad's head proportion didn't exactly match the body double. We didn't want to distort Brad's head to fit the hat because we didn't think that was the right way to go. So sometimes we would just add in a CG hat and hope the audience wouldn't notice, but I thought they needed to be in the shots so that Claudio Miranda (Director of Photography) and David could frame their shots. It did help us sell the illusion ultimately. However glasses were a huge problem.
Even back in 2004 when we were doing the first test scene David said he wanted these glasses to be as big as coke bottles. He wanted the audience to be able to see the eyes enlarged through refraction so they got the sense that he cannot see. At that stage I was going "Uh, oh" because refraction adds problems,. the main one being that when we are looking through refracted glasses with the thicker lens it often distorts our perception of where that person is looking. So maintaining focus and an eyeline was very challenging.
)
)
cleanbreak
fxg: Was there a lot of work in compositing as well?
Eric Barba: It was massive. One of the things that we learned from the initial test scene and Orville, where we had 15 shots in the three spots we did for Orville, was that even with some really good compositors (and I had my A-team on that project) there's only a handful of people that have the right sensitivity that once they get these head renders and put em in a scene, that they have the sensitivity to colour so that the skin tones don't come apart, or look ashen or too pink or too red. One of my compositing supervisors for the Orville spot had that sensitivity, although the spots didn't end up working that well for other reasons.
So part of our plan going into Benjamin Button was to build a lighting system that would give us as close to perfect as possible, out of the box, then we needed to build a compositing template system that allowed us to hand off shots to 20 other compositors and get the same results . That's a really tall order with normal rendered CG but for Benjamin Button it was off the meter in terms of difficulty. Hats off to my lighting supervisors who did an amazing job.
We used a lot of common techniques such as HDR for set reconstruction. Our lighting system was different from the normal workflow. Traditionally your tracking team hands over their data, the lighting team will render off the pieces and they will get handed off to the compositing team, so they come in last. In this show our compositing actually came in first and they really enjoyed that. The integration team that was on set with us for seven months, basically surveyed every piece of equipment and set and we took HDRs for all the different head positions as we were shooting, and then all that information was catalogued as we were shooting it.
We then built a system in 3D that would ingest that data, build the HDRs. We then had modelers rebuild the set from the survey data. We built shader systems that could take HDRs and project them back onto the set geometry, Because we had the tracking of the head, we could relight it within Nuke and we could reproject those HDRs onto the CG head, to recreate a corrected HDR for the head position.
)
)
Digital Domain emotion Capture Approach
- Eric Barba: " Since our goal was not to create Benjamin's performance in animation, but rather to 'xerox' Brad Pitt's performance onto this CG head, we had to develop a brand new process that we call emotion capture."
The overall process included:
1. Working from life-casts of Brad Pitt and body actors to create three photo-real maquettes representing Benjamin in his 80s, 70s and 60s, then shooting them in different lighting conditions using a light stage. (Rick Baker and Kazu Tsuji created the maquettes).
2. Creating 3D computer scans of each of the three maquettes.
3. Shooting scenes on set with body actors in blue hoods.
4. Creating computer-based lighting to match the on-set lighting for every frame where Benjamin appears.
5. Having Brad perform facial expressions while being volumetrically captured (with Mova/Contour), and creating a library of 'micro-expressions.'
6. Shooting Brad in high definition performing the role, from four camera angles, and using image analysis technology data to get animation curves and timings.
7. Matching the library of expressions to Brad's live performance of Benjamin.
8. Re-targeting the performance and expression data to the digital models of Benjamin (created from scanning the maquettes) at the specific age required in the shot
9. Finessing the performance to match current-Brad expressions to old-Benjamin physiology using hand animation.
10. Creating software systems for hair, eyes, skin, teeth, and all elements that make up Benjamin.
11. Creating software to track the exact movements of the body actor and the camera, to integrate the CG head precisely with the body.
12. Compositing all of Benjamin's elements to integrate animation, lighting, and create the final shot.
No comments:
Post a Comment