While Cameron and the actors cavort around the set, technicians
sit at the far end of the room, pulling the digital strings. This
is Avatar's "brain bar" - three tiers of computer monitors
manned by 20 or so people: video specialists on the first level,
mo-cap technicians from Giant Studios and real-time CG operators
from Cameron's Lightstorm Entertainment on the second, and
visual-effects supervisors and facial-capture experts from Peter
Jackson's Weta Digital up top.
On the monitors, the brain-bar crew is watching various
iterations of the fan lizard scene as it's being captured. One
screen maps skeletal data points on a grid. Others toggle from
Worthington and Saldana on the risers to their animated characters
in the jungle. The brain bar was built to accommodate
Avatar's digital production pipeline. Several months
before, Cameron wrapped the bulk of the film's hi-def 3D shots in
New Zealand, most of them against a green screen. Now, thanks to
hardware called a simulcam, the production team can mix
performance-capture from Playa Vista, live-action footage from New
Zealand, and CG backdrops to conjure up a real-time composite. It
compresses into seconds a job that would normally take weeks. On
the fly, the brain bar can transmute hundreds of gigabytes of raw
data into a rough approximation of the fantastic universe that, for
the past 15 years, has existed only in Cameron's mind. "You can
call it what you want," says Glenn Derry, Avatar's
virtual-production supervisor. "But it's just a big database."
For Cameron, the brain bar is the ultimate control mechanism.
"It started with Jim going, 'What if I could do this?'" says Richie
Baneham, the production's animation director. But beyond that,
there was no master plan. "It just kind of evolved," Derry says.
"We kept adding features and adding features, and then we got to
the point where we couldn't keep track of them all. Now we bring in
people from outside and they go, 'Whoa!'" The foundation of Derry's
virtual-production system is Motion- Builder, a 3D
character-animation program that's been used on everything from
Rock Band to Beowulf. Giant Studios was brought
in for its real-time "solve" - that is, the ability to interpret
performance-capture data in real time. They ended up with a highly
evolved form of previsualisation, a computer-animation technique
designed to show what a scene might look like with live-action
sequences and CGI meshed together.
When Derry started on Avatar in 2005, Steven Spielberg
and Industrial Light & Magic were pushing the limits on
previsualisation for War of the Worlds. As Spielberg was
shooting an action scene, he knew where the alien war machine would
go when ILM rendered it months later. But he couldn't actually see
the tripod crashing into cars and buildings during his shoot -
which is what you'd get with Cameron's setup.
Meanwhile, Baneham had to figure out how to overcome the
"uncanny valley", that unnerving gulf between human and not-quite-
human that made the performance-capture figures in Robert
Zemeckis's Beowulf and Polar Express feel creepy.
Zemeckis, Cameron says, "used the same marker system we use for the
body and captured the facial performance with those mo-cap cameras.
Really a bonehead idea." Baneham's solution required the production
crew to make a plaster cast of each performer's head, then build a
custom harness to hold a tiny videocam that would be aimed back at
the actor's face from just a few centimetres away. The wide-angle
lenses would record every twitch, every blink, every frown.
But it would take more than that to make Avatar look
real. Before he joined this production, Baneham led the effort to
bring Gollum to life in The Lord of the Rings. At that
time, when digital animators wanted to render a smile they would
just dial the mouth a little wider, which invariably looked fake.
So Baneham and his team studied physiology: what muscles fire to
produce a smile, and in what order? How much light does the skin
absorb, and how much does it reflect? How deep is the pupil, the
opening at the centre of the eye? The facial details that made
Gollum so convincing were not provided by performance-capture but
by painstaking, frame-by-frame CG work. Avatar's system
streamlines that process: animators can work from the head-rig
videos, which supply a complete visual record they can map to each
Even as Derry was reinventing virtual production and Baneham was
rethinking facial animation, Cameron was throwing out the rules of
shooting 3D. With camera designer Vincent Pace, he developed a
system with twin lenses that could mimic human vision. Wearing
polarised glasses, viewers would get the same 3D effect they do
with CG films like last summer's Monsters vs Aliens or the
forthcoming Toy Story 3. But shooting live-action 3D is
more complicated than rendering computer animation in 3D.
The conventional method relies on a series of cumbersome
mathematical formulae designed to preserve the "screen plane" - the
surface on which the movie appears. In 2D that's the screen itself,
but in 3D it's an imaginary point somewhere in front of you." The
viewer doesn't think there's a screen plane," Cameron explains.
"There's only a perceptual window, and that perceptual window
exists somewhere around arm's length from us. That's why I say
everything that's ever been written about stereography is
To Cameron, eliminating the screen plane is crucial. "The screen
plane has always been this subconscious barrier," says Jon Landau,
his long-time producer. By removing it, Cameron hopes to create an
all-encompassing cinematic experience. That's the reason for the
virtual camera and the simulcam, the hi-def head rigs, the 3D
camera system: total immersion. "This is not just a movie," Landau
says. "It's a world. The film industry has not created an original
universe since Star Wars. When one comes along so seldom,
you want to realise it to the fullest possible extent."
That's one reason to do all this - the other reason is more
personal. "I've made a bunch of movies, won a bunch of awards, made
a bunch of money," Cameron declares. "None of those are interesting
to me. They never were. It was never about the awards, never about
the money, only about the films. But what's challenging? I look for
the new thing. And with this film, we've really loaded it up."
Want more Wired UK magazine? Get your copy every
month by subscribing online today