Many a parent has surely looked at a tween and wondered what was going on in that child’s head. But few can spin that thought into an animated feature film. Pixar’s Pete Docter did just that.
In a joyful sequence in Pixar’s film Up, for which Docter received an Oscar, we see Carl Fredricksen and his wife, Ellie, as children. Ellie is an exuberant little girl, full of life. Docter’s young daughter Ellie provided that little girl’s voice. And then, at age 11, Ellie Docter changed.
So, too, 11-year-old Riley in Disney/Pixar’s Inside Out. Written and directed by Docter and co-directed by Ronnie Del Carmen, who was story supervisor on Up, the film follows the Riley her parents and the outside world see, and the emotional Riley inside her mind that only we, the audience, can see.
“I feel like it’s the most personal film I’ve dealt with,” Docter says. “Riley became her own character, but the story is based on what it is to grow up. It’s difficult for the kid and for the parent. You love your kid, and then their personality changes, and that’s difficult. Relationships go beyond having good times with people. We share intense sadness. We’re scared for people, angry at them. Things don’t feel positive at times. I wanted to dig deeper and have something substantive. We set the film where we feel the deepest connection.”
As the title suggests, there are two parallel story lines in this film, with each affecting the other. Riley, her mother, and her father inhabit the real world, the outside world, a recognizable world set in Minnesota and San Francisco. Inside Riley’s mind an ensemble cast of personified emotions cavorts and kvetches in a bright, candy-colored world.
“Of all the films I’ve worked on, this one has changed the least in terms of overall tone and feeling,” Docter says. “Monsters, Inc. and Up were hugely different. It’s been great going to a place everyone has been to in their minds but have never seen on screen.”
To generate extreme emotions in 11-year-old Riley, the story sends her from a happy, hockey-playing life in Minnesota to a bare bedroom in a dank San Francisco Victorian and a formidable new school. Her father has uprooted the family to take a new job. Inside Riley’s mind, her emotions gather in headquarters and try to help.
But, which emotions? Docter and the crew did research and consulted experts, including Dr. Paul Ekman, a psychology professor emeritus at the University of California, San Francisco, who has studied and classified facial expressions and emotions. Among animation and visual effects studios, Ekman is known for the Facial Action Coding System (FACS), an anatomically-based tool for objectively describing observable facial movement, which he developed with Wallace Friesen, published in 1978, and revised in 2003 with J Hager as a third author.
“At the beginning, I had listed optimism, hope, pride, love, and many other things,” Docter says. “We talked about love with Paul Ekman, and he argued that love is not an emotion. Technically, the way you define an emotion is as a short, three- to five-minute reaction. Love is a state of being that transcends a burst. Anger and fear were obvious, and even joy, which I called optimism and he called happiness. He brought up disgust, and I realized that could be interesting. We ended up with 16 emotions, but that would have been melee in headquarters, so we narrowed them down.”
Ekman lists seven emotions that have universal signals: anger, fear, sadness, disgust, contempt, surprise, and happiness. The Pixar team eliminated contempt and surprise, kept anger, fear, sadness, disgust, and happiness, and renamed happiness “joy.” All the human characters have the same emotions inside. Mom’s emotions wear glasses; Dad’s have a mustache. But Riley’s Joy is the central character.
Mental Energy
“Pete said that young kids are very happy,” says Co-director Del Carmen. “So, the emotion that must be driving them is joy. Joy would be our lead emotion, our main character. At first, we had Joy be happy all the time, and Anger be angry all the time. The characters said the same things over and over. It was annoying. We had to learn. They’re not people, they’re emotions. But they needed to have a range of emotions.”
And then, the personified emotions raised other questions. Do the emotions know one another? Do they care about one another? What do they talk about? What do they sound like? They live in headquarters, but what do they do during the day?
“The punch line was when we put all the emotions in a booth in a Chinese restaurant,” says Producer Jonas Rivera. “Do they eat? Where do they get food? Do they all like Chinese food? Will they have to go to the bathroom? This brand-new world was a challenge. As soon as we made up a rule, we ended up breaking it.”
Anger required special rigging controls to open his mouth cartoon-wide.
The team continued to do research into emotions and their role in our lives. They concluded that the emotions’ goal is to make Riley happy.
The desaturated colors underscore the family’s not-so-happy move to San Francisco.
“Emotions are there as guardians and advisers,” Del Carmen says. “All the characters try to find out how to best serve whatever challenges Riley at the moment. They don’t always choose correctly, but they take their best shot. Life is changing for Riley. Joy, the lead emotion, has to work hard to keep things running the way the emotions want. She’s right in front of every other emotion, and we watch Joy being challenged by all the other emotions. Joy is dedicated to micromanaging everyone’s job. That’s what causes a lot of trouble.”
The character designers gave each emotion colors that reflect their state of Riley’s inner mind. Sadness is blue. Anger, red. Fear, fuchsia. Disgust, green. And Joy is a golden yellow. All the characters have a soft glow around them in varying amounts. Anger glows the least and is the most solid, then Fear, Disgust, and Sadness. Joy has the most glow.
Lightbulb
To capture Joy’s spirit, Production Designer Ralph Eggleston gave Joy effervescence. “Like champagne bubbles,” he says. “And the first words out of John [Lasseter’s] mouth were, ‘Great. Let’s put it on all the characters.’ We almost didn’t. It was so expensive.”
A team of 35 lighting artists led by Director of Photography and Lighting Kim White lit the scenes and added the glows. The effervescence is a particle simulation created by character effects artists.
“The particles close to the skin are procedural, created in the character department,” says Lighting Artist Angelique Reisch. “We tested them in lighting, and it took a lot of testing. Sometimes they’d poke through the mouth bag and come out of an eye.”
The lighting artists had an additional challenge with Joy: The happiness personified character glows from within.
“Joy is a volume,” Reisch says. “Our first volume character. We wanted softness around Joy’s face. We wanted her to feel like energy, not like she had particles on top. Her wrists are transparent. We needed a volume approach.”
A geometric representation of the character served as a stand-in for shading and lighting – and for animators, as well.
“That made it fast to iterate,” Reisch says. “It was the same with Sully [in Monsters University]. We turned off his fur and lit his geometry so we didn’t have the expense of interactive rendering.”
However, the environment had little effect on Joy because she’s one of the most unique CG characters to star in a film: Joy is a light source.
“How do we light a lightbulb?” Reisch asks. “It was a huge challenge. Typically, we rely heavily on value, the lightness or darkness, to shape a character. We look at images independent of color to be sure we get a good tonal range of brights, middle tones, and darks. However, Joy is so bright overall, we lost that range of value. Yet, we still had to shape her.”
Rather than using value, the lighting artists used color for Joy.
“On the key side, we used a cool and desaturated kick light,” Reisch explains. “A blue light for the yellow character. Her soft key is neutral with no saturation change. And then on her off-key side, we used warm, saturated color. For her inner glow light along her face and body, we used desaturated lights on the key side and saturated pink on the off-key side. We don’t shadow her.”
In addition to lighting Joy directly, the artists needed to have Joy cast light.
“We wanted to see her light casting in detail,” Reisch says. “We wanted to see light between her fingers.”
Fortunately, the Pixar RenderMan team developed a geometry light with which artists can specify a model and turn it into a light source.
“They gave me Joy’s geometry light in December 2013,” Reisch says. “Our guys worked with them to get it working, and in March 2014, we started using it in production. It’s in every shot in the show. As Joy walks, you see her light spill on the floor with good specular reflection. It changes with her animation.”
Animation
Directing Animator Jamie Roe, Supervising Animators Shawn Krause and Victor Navone, and a team of 45 animators performed the cast of characters in the real world and the emotions inside Riley and her parents’ minds. In addition, a separate team created a 2D animation for an abstract sequence in the film, and a crowds team handled hockey players and other groups of characters.
The entire production for Inside Out took three years, with half that time needed for animation. (Animators at Pixar produce approximately three seconds of animation per animator per week.)
The humans in the outside world had familiar characteristics and a design that placed them between the realistic characters in Toy Story 3 and the stylized ones in Up. “Initially, we animated them very naturalistic to provide contrast with the mind characters,” Navone says. “But, they felt boring and unappealing, so we loosened up.”
The mind characters, however, are among the most cartoony that Pixar animators have worked on.
“We could break some of the rules we have here in terms of realism and physical boundaries,” Docter says. “It was our chance to reach back to Tex Avery and Chuck Jones. From the get-go, I imagined this film to be a movie with a lot of fun, ensemble comedy.
Each of the supervising animators picked a mind character to focus on, and worked with a character TD to design appropriate rigging. Krause tested Joy, Navone took fear, Roe managed sadness, and Story Artist Tony Fucile, who worked with the animators, helped with Disgust. As the animators worked with the characters, Fucile would use a new sketching tool built into Presto, Pixar’s animation system, to draw over the poses.
“Each emotion character is unique in design and style,” Roe says. “Anger is a short block, really stiff. Certain angles look good for him, so we had to cheat for camera angles. Fear was difficult. We had to craft a lot of poses and work with his eyes and his hair to get him to look like a character. His nose looked wrong when we saw him straight on, and his eyes were on top of each other.” Also, his legs were short and skinny.
The subtle Sadness was simpler to perform because she didn’t move much. “We had to cheat perspective a lot with her mouth,” Roe says. “She had a big mouth that could go small.” Similarly, Disgust stays consistently disgusted.
The challenge for Joy was in rigging her eyes.
“She’s the first character we’ve had with eyes taller than round,” Navone says. “It was hard to get the lid shape right. Tony [Fucile] would draw over the poses, but our current controls wouldn’t let us do that. The design worked when her eyes were wide open or shut, but the in-between subtleties were difficult. Trying to do things that Tony made simple with drawing was hard to do with a CG rig. She had to go through an entire spectrum of emotions.”
Although each character started with the same master rig, Sadness and Anger were so short, the riggers had to remove a segment of the spine. And, they gave Fear and Joy extra controls for the eyes. Anger had a rig that allowed the animators to open his mouth extra wide.
The process of creating a final image (from top left): storyboard, concept art, character design, set design, layout and character blocking, animation and simulation, and the final with lighting
“The character design was cartoony, so we gravitated toward a Warner Bros./Tex Avery/Alice in Wonderland style,” Navone says. “We wanted to have fun, but we needed to keep a sense of peril involved. They could do anything and survive. We wanted people to relate to them – particularly Joy and Sadness. We needed to have the audience feel the characters could be hurt or the audience wouldn’t be invested.”
Believable Imaginary Worlds
In the same way that the human characters tended to be realistic and the mind characters colorful and cartoony, so, too, are the worlds they inhabited.
“In the real world, we have a lot of texture but not a lot of reflection,” says Eggleston. “It’s high key, low contrast. There aren’t a lot of dark shadows. The mind world is high contrast, saturated, and has a lot of translucency and light-emitting surfaces. It was by far a gazillion times the hardest film ever. We had to do two films at once – the real world and the mind world. And in the mind world, we take a journey, so that made it two and a half films. Any changes to the mind world changed the real world and vice versa. I felt like I was roller-skating on marbles.”
All the characters in Inside Out glow, but Joy is the only character that is a volume and a light source
The film begins in Minnesota, but the Midwestern pastels became desaturated when the family reaches San Francisco.
“In Minnesota, they had an acre of land,” Eggleston says. “In San Francisco, they have a little place. There are lots of tracks, lots of wires.”
The mind world, by contrast, is filled with color and light.
We first see the emotions in their headquarters. The walls and ramps are curved and purple, and glowing. Outside the tall windows, memory banks extend into the distance, walls of shelves filled with millions of colored, semi-transparent globes, each holding a memory. Joy waves to the train of thought as it whistles past the window.
“Headquarters was a major set and the central focus of Act 1,” Eggleston says. “We used elements of the brain, but we wanted the characters in the mind, not the brain. I probably did 200 designs. We have cellular structures, and for the long-term memory, used folds of the brain.”
The inside world – which includes the long-term memory, the mind pit, personality islands, abstract thought, imagination land, dream productions, and, of course, the train of thought running through it all – is huge.
“The mind pit is 2.3 miles across in virtual space,” Eggle-ston says. “Joy is 4.2 inches tall. We couldn’t cheat. All this stuff had to be rendered. It was scary. There are so many one-offs.”
Artist Dan Holland did 284 designs for the trains before finding the one that worked.
“John [Lasseter] came up with the idea of an electric train because of the electrochemical brain,” Eggleston says. “The track appears and disappears behind the train, and no matter where it went, if it stopped, it came to a train station.”
Filming Inside and Out
The contrast between “Inside” and “Out” is heightened further with the cinematic style chosen for each. Director of Photography for Camera Patrick Lin led a team of 10 layout artists who choreographed the camera moves, starting with an initial character-blocking pass.
“What we do is visual storytelling,” Lin says. “We created a visual language that defined each world, created contrast, and kept the two separate.”
I
n the outside world, the camera is realistic and flawed. Inside the mind, the world is virtual and perfect. Outside, the camera has imperfect focus. Inside, spot-on focus. Outside, the camera is handheld or a Steadicam. Inside, the camera is on a mechanical dolly, a track, or a boom crane. Outside, an operator in a motion-capture room moves the camera. Inside, the camera is CG.
Pixar first used motion capture to film a scene with a virtual camera for a home video sequence in Toy Story 3. The short film “Blue Umbrella” was next.
"We expanded from there,” Lin says, “made the technique more polished, and made a pipeline. In the motion-capture room, the camera is locked to a virtual set. Every time the camera moves in the outside world, we did it with the camera capture.”
In Act 1, the “outside” camera moves with motion captured from a Steadicam on a tripod as the DP films Riley in Minnesota; in Act 2, an unlocked camera is still on a tripod. “Riley’s emotions are more unstable; she’s locked down,” Lin says. “So, we only use pan and tilt. In Act 3, the camera being captured is handheld.”
The camera in the mind world, however, is virtual.
“In the mind world, the progressions are more subtle,” Lin says. “Act 1 is ’30s-style mechanical: slow, deliberate, as if the camera were really big. All the emotions are working well and everything is under control. Late in Act 2 and in Act 3, we move into a more contemporary style, as if the virtual camera were smaller and easier to handle. The pan and tilt is more active, and the speed of the crane is faster.”
The simple idea of wondering what is going on in a child’s mind has resulted in a complex, highly designed film – two films, actually, as the crew points out. Each film has its own production design, color and lighting, animation style, and photography, and each affects the other. Few studios other than Pixar could choreograph such an intricate dance.
“I think this is a profound film in many ways,” Del Carmen says. “Even though the characters have friendly, child-like shapes, their potential for being visual tools for conversations about feelings and emotions is enormous. Making this movie has helped me understand that you are not your emotions, but you have a way of enriching your emotional response to the world. Joy isn’t the solution to everything. Nor is anger. Sadness is empathy. Fear will make sure you’re not in danger, but it could be irrational. That kind of character development and storytelling is fun.”