Toying Around

If  you thought Pixar closed the lid once and for all on the toy box following Toy Story 3 in 2010, well, think again. The beloved characters that broke new technological ground and paved the way forward for 3D computer animation when they made their debut in 1995 have returned once again to the big screen in Toy Story 4.

When we last saw Buzz, Woody, and the gang, they were settling in with their new kid, the sweet, young Bonnie, following a harrowing ordeal at Sunshine Daycare after accidentally getting discarded when the teenage Andy was packing up before leaving home for college. There they met Bonnie, and after some adventures and misadventures, the toys found a new home with her, and she promised Andy that she will care for them just as he did.

"Before we finished Toy Story 3, Andrew Stanton, who was the godfather of all the Toy Story films, had been playing around with the treatment for Toy Story 4 but hadn't really shared that with anyone," says Josh Cooley, Toy Story 4 director. "We all loved the ending of Toy Story 3. It felt like the end of the trilogy, the end of the story."

Absent from Toy Story 3 was Bo Peep. And Stanton's treatment for a sequel had Bo back in Woody's life. Cooley began playing around with that concept, "to see if there was anything there worth visiting."

"The more we played around with it, the more we discovered that [Toy Story 3] was not the end of Woody's story, it's the end of Woody's time with Andy. There was actually more of Woody's story to complete his arc as a character," Cooley says. "We kind of realized that there's more because of where he is now at the end of Toy Story 3 - he's with a kid, in a new bedroom, and with new toys. And that is a situation we've never seen this character in before. As we started talking about it, it was really fascinating to see how the story just kept going. And, we came up with a story that we just had to make."

Cooley describes the film in terms of a romantic comedy, from Woody's point of view, while Bo is a major catalyst in his change as a character.

Toy Story 4 opens on a reluctant Bonnie as she is getting ready for her kindergarten orientation. Shy and afraid, Bonnie and the other students are instructed to make an art project, but Bonnie is left at a table, alone, with no art supplies. Woody, who stowed away in her backpack to make sure things go smoothly for his new kid, secretly retrieves some items from the trash for her to use. She proceeds to make Forky, a plastic spork with pipe cleaner arms, feet fashioned from a wooden stick, and a face constructed of odds and ends.

Bonnie loves her new toy. Only, the anxiety-ridden Forky stubbornly maintains he is trash, not a toy, and rebels against that notion. Woody, who had been getting less playtime with Bonnie even before Forky's arrival, nevertheless tries to make the craft project feel welcome and accepting of its new status as a toy - all because Bonnie loves Forky, and Woody loves Bonnie.

With kindergarten a week away, Bonnie's parents take her on a road trip, during which Woody tries to keep Forky from escaping into the trash where he feels safe and comfortable. Eventually Forky accomplishes his goal, but Woody follows in an attempt to bring him back to Bonnie. The two talk, and Forky begins to accept his new role, and the two head to the campground to meet up with Bonnie and the other toys. Nearly there, Woody spots the lamp belonging to the long-lost Bo Peep in an antique mall window and wanders inside looking for her. He and Forky are then captured by the resident Gabby Gabby doll and her ventriloquist dummy gang. Meanwhile, Buzz sets out to find Woody and Forky and becomes an uncooperative prize at a carnival across the street from the antique mall. Woody finds Bo and they escape, and Buzz escapes the prize rack along with two stuffed toys, Ducky and Bunny. They all unite to save Forky.

The effects team spent the majority of its time and budget creating the rainstorm in the film’s prologue.

When It Rains…

So, what happened to Bo all those years before? We find out in the opening of minutes of Toy Story 4, in a flashback. There is a big rainstorm and the toys are attempting a daring rescue of the toy car RC, which is trapped in a gutter and about to be washed away. After the mission is accomplished, Woody finds that Bo Peep, part of the lamp in Andy's sister's room, is missing. Andy's sister outgrew her. Woody tries to help her escape her fate, but Bo accepts it and is driven away in a box.

Cooley estimates that the majority of the effects team's time and budget were spent in that prologue.  

"The opening of the film is huge," says Cooley. "It's a downpour, making it as hard as possible for the toys to rescue RC. It's like a flash flood to them. And as RC is being swept away, there's leaves, dirt, sticks, and debris in the runoff along the side of the driveway. It's all mixing with the rain and interacting with the toys. It's pretty amazing."

Bill Reeves, global technical supervisor, agrees. "It was an impressive, effects-heavy sequence that we worked really hard on to make it look authentic," he says. "When Woody climbs down to help RC, the rain is really big from his perspective. He's a toy, maybe 15 inches tall. It took a lot to make that scene work."

As Bob Moyer, supervising technical director, points out, the artists used SideFX's Houdini Engine to make the rain feel authentic at both a human level for scenes involving the human characters as well as a toy level. "It had to feel like the rain was a real threat to the toys, and the effects team had to make the splashes threatening and the gutter feel like a raging river to a toy," he adds.

Woody tries to help Bonnie’s new craft project accept his role as a toy.

Moyer explains that the effects artists worked out a palette of effects they could dial in depending on the shot. Initially, they planned to have human shots and then toy shots. But as they progressed, they realized that each shot would have its own special blend of both types, so they developed techniques and qualities they could push forward or backward depending on the requirements of the shot.

Interestingly, the original film also contains a rainstorm, when Buzz was strapped to a rocket in Sid's room. "John Lasseter wanted the scene to be moody and dark, and the pitch was to do a rainstorm," says Reeves. "Back then, that was really hard, so we just did some particles zipping through the air in one exterior shot, and then we cut inside and did drips on the window for the rest of the storm. We had a matte painting outside with clouds, but we never showed the actual rainstorm other than those simple effects. It was all we could do at the time."

Flash-forward to 2019, and the water is rushing down the gutter, hitting and pushing RC. He's spinning his wheels, and mud and debris are flying into the air. Indeed, time and technology do march on. And, Reeves should know, as he, along with production designer Bob Pauley, worked on all four Toy Story films, starting with the first one back in 1995, through 1999 with Toy Story 2, again in 2010 with Toy Story 3, and now the current sequel.

"1995 was sort of the Stone Age of computer animation. The technology and software was really primitive when we were working on the original Toy Story, and we managed to cobble it together and made a movie," recalls Reeves. "To this day, we continue to make a movie with the technology we've got at that point in time."

As Reeves points out, each Pixar film builds upon the previous one  in terms of overall knowledge, technology, and general know-how. "As films come along, they feed off each other. Some are radically different, and others just have small changes. Toy Story 4 is sort of in the middle of that spectrum," he says.

Changes in Environments

The film features two sets that were particularly complicated. The antique mall contained a significant amount of geometry that built up quickly, along with a lot of glass and reflective material, while the carnival had to feel alive, and its nighttime lighting enchanting.

There are some major visual changes between Toy Story 3 and 4, and while the latest release is a continuation of the original story and world, the imagery is noticeably richer and the environments have a higher degree of realism due to global illumination. "We've gone through a whole new renderer overhaul that you first saw in Finding Dory, but I think it has really come into its own for Toy Story 4. You're going to see that the global illumination and lighting on this film is absolutely gorgeous," says Moyer of using Pixar's RenderMan 21 for the film. "We spent a lot of time calibrating every lightbulb and light source in the film so they all interact with the set in a way that is believable and effective for extended dynamic range."

Adds Reeves, "With the renderer, we could do some pretty realistic environments, where the light is bouncing off all the glass and surfaces - bouncing, not just reflective. It's very subtle, but it makes [the scene] rich and believable."

Whereas the original look of Toy Story was stylized shapes, realistic materials, and theatrical lighting, the crew on Toy Story 4 took that even further. "We made sure the shading felt as detailed as possible, and our lighting department gave it the theatricality we were looking for - bright colors and a lot of atmosphere," Moyer explains. "We hesitate to call it 'realism' though, because we're still making an animated movie. But we talked about how to push things back using techniques like depth of field and atmosphere so [audiences] could focus on the action as opposed to how many objects were in the scene."

Second Chance Antiques is a massive set with thousands of objects.

Like the rainstorm sequence, scale - at the toy level - was important in these environments. "We really dug in on characters, sets, and effects to make the world at this scale a tactile, beautiful quality that you wouldn't see at the 6-foot-tall human scale," he adds.

At the Antique Mall

Nearly immediately, the crew began work on the antique mall. This included visits to antique stores, where they observed all types of nooks and crannies just the right size for toys to explore - behind shelving units, underneath low tables, and so forth - that would be interesting from a toy's point of view. That's what we see in the film. Cooley also observed that such stores have a history of ownership; no antique store is constructed specifically for that purpose. In the film, the mall is an enormous open warehouse filled with tons and tons of various pops.

A large portion of the film takes place here, and it quickly became one of the filmmakers' biggest technical challenges. The store contains approximately 10,000 items, all hand-built and hand-dressed. Not to mention, the set is full of glass, resulting in reflections, and has a multitude of chandeliers and lights reflecting off various types of objects: metal, wood, cloth. "Every item was reflecting off something else," says Cooley. "We realized we could render it if we did some cheats. Then we filled it with 10,000 unique items, and added dust and spiderwebs on top of that. And, it actually worked!"

According to Moyer, the team built the space into a hierarchy of subsets, where roughly each booth was a separate set, and then those were all pieced together into areas, and those were used to build out the mall. "We could activate, deactivate, and cut out pieces, sometimes even down to the individual geometric primitive level of visibility," Moyer adds.

To move the sequences through the pipeline more easily, the crew used USD (Universal Scene Description), which baked down all the set and prop information into a simple scene description so animators and lighters could work within these larger segments and not bog down their machines.

"There are sometimes millions and even hundreds of millions of curves in a scene, and we had to smartly move all that geometry through the pipeline and into lighting without killing our lights or our renderer," says Moyer. To that end, an automatic pruning system was devised that was able to work with all the refraction and reflection, as well as, on the flip side, turn off light sources and hide some reflections as needed. Sometimes that task was done by hand, sometimes it was automated.

At left shows the dust detail at full strength; at right is the final frame with adjusted dust detail.

In fact, the artists spent a lot of time with the RenderMan team, realizing that glass, lighting sources and practical illumination were going to be especially challenging. In addition, they used a machine-learning denoiser from Disney Research Studio in Zurich that automatically removes the noise from a partially resolved global illumination render. "The denoiser does such a beautiful job that we were actually able to cut our render times to almost half, with the same or better level of quality than we had in prior films," says Moyer. "It unlocked all these workflows for us that we hadn't been able to have before."

The denoiser, which has not yet been released, was written in two versions: one for the GPU and one for the CPU. Pixar used the latter.

Another level of difficulty in the antique mall resulted from the large amount of dust and spiderwebs. For the webs, the sets department used Houdini to create a simulation that was effectively digital spiders infused with AI that built curves to form a web. As for the dust, the artists physically covered the ground with little curves of dust and chunks of debris to give it a texture, as opposed to simply adding a shading layer of dust over the objects. "A group from effects made sure there was always particles subtly floating in the air," says Moyer. "So everywhere you looked in the mall, you feel a sense of age."

Fun at the Carnival

Just across the street from the antique mall, which is static and old, is a dynamic, bright, colorful pop-up carnival. Again, the team visited county fairs and the like to get a sense of how the rides move. They also people-watched and discovered toy-size spaces, particularly underneath the rides and games.

The simulation department rigged and animated the rides with controls to move appropriately. Then, within Houdini, the lighters created a half dozen to a dozen different light cycles per ride that the director or production designer could choose from for each sequence, depending whether they wanted a calmer or more active look.

New to the cast is Gabby Gabby and her ventriloquist henchmen.

In terms of lighting, "there are no cheats," Cooley notes. "During the day, carnivals are kind of gross, but at nighttime when they're all lit up, they're stunning and beautiful." Thus, lights were important. Cooley does not recall how many thousands of lights were used in the environment, "but they all work, they all blink, they all form patterns."

When constructing the carnival as well as the other locales in the film, the modelers added lightbulbs, and included metadata that contained wattage and lumens information, which were then integrated into the lighting system and the renderer. As a result, all the lights illuminated automatically. "In the antique mall, we only had to turn the lights on or off, but we had to animate them at the carnival," says Reeves. "But, the carnival lighting benefited from that upfront infrastructure we built into each lightbulb. Once we had everything built and assembled, it made things a lot easier to get those scenes up and running."

And like all the film's environments, the carnival set had to work on a both a human and toy scale.

Character Updates

The film has many returning characters, from Buzz and Woody, to Hamm, Jessie, Rex, Mr. and Mrs. Potato Head, the child Bonnie, and others. There is even an appearance by Tin Toy, the star of Pixar's 1988 short. Among the newcomers are daredevil Duke Caboom, the villain Gabby Gabby, her ventriloquist dummy henchmen, and the comedic duo of stuffed prizes Ducky and Bunny. And, making her reappearance is Bo Peep.

Even though the overall design of the returning cast did not change, they again were reworked to take advantage of current technology. "It's not because we want to art-direct them any differently. In fact, we want Woody to look like Woody. We don't want to change him," says Reeves. "But, this is nine years after the last Toy Story, and that version of Woody wouldn't work with our software that's running today or in our renderer. So, we had to rebuild him and the others."

The toys were aged slightly to reflect the passage of time since the last film, with subtle scratches on the hard-surface toys and fuzz on the cloth toys.

While the geometry stayed pretty much the same, the shading was overhauled to an unprecedented level of detail to where tiny scratches are visible on Woody's face and little pieces of fuzz can be seen on his vest. "With the new renderer and denoiser, the details hold up," says Moyer. Artists also added subsurfacing and fine detail to Woody's exposed skin, making it look more plastic-like.

Ducky and Bunny’s neon plush fur required artists to fine-tune the shaders.

The lighting and character departments also added subsurface and higher illumination to Buzz (and most of the plastic toys) to make him more translucent. Buzz also has some subtle glow and illumination coming from his buttons, thanks to the new renderer. "Buzz's helmet has always been a reflection cheat to ensure that the reflection didn't cover his face, and with the new renderer, his helmet is fully refractive," Moyer says.

For Bo, the changes are more significant. "The last time we saw her was in Toy Story 2. She's always been porcelain, but until now, we never had the technology to make her really look like porcelain," says Cooley. "We spent a lot of time researching porcelain, how it's affected by light, how it ages, the way it crackles underneath the surface. Then we incorporated that into Bo; she is shiny and reflective but has the depth of real porcelain. When light enters the glaze on her skin, it bounces in the tiny cracks and creates little extra shadows and brighter patches."

Back in Toy Story 1, Bo was a secondary character, and artists used a plastic shader that was tweaked to resemble porcelain. Now, with a new shader and subsurface work to achieve a crazing aesthetic, she is practically a new character - and in more ways than one.

Bo has ditched her bonnet and pretty dress for more practical clothing, but retains her shepherd's hook, using it Indiana Jones-style. She also has a cape, which animation and simulation worked to get the flow to feel realistic but stylized.

Bo's new friend, Giggles McDimples, is a tiny toy, about half an inch tall, with a face that is painted as opposed to comprising features (such as eyes that open and close), like the other characters. For this, artists devised a system that would create geometry in the animation software that the animators could see, and that would get projected onto the surface for the actual renders. For extreme close-ups, they could make the paint appear as if it had worn off in some spots.

Newcomers Ducky and Bunny have neon plush fur. "Neon in a photorealistic renderer can blow things up. Light hits the character, which is super bright, and it starts throwing what we call fireflies, which are bad pixels, everywhere," explains Moyer. This required the team to fine-tune the shaders to curb this problem.

Bo Peep is now shiny and reflective but has the depth of real porcelain.

Toys Will Be Toys

When Cooley was asked which character was his favorite, he replied, "Woody." Because he was the most difficult. "This movie is all about him, so for me, the hardest part of the movie was finding the right way to push Woody's story forward and not retread on anything that we've done before, to keep things completely new and different," he adds. "But, I love them all!"

In picking up this beloved franchise for another go, Cooley was left with many a sleepless night: "I just wanted to get it right because I know how much these characters mean, not only to the audience, but to Pixar as well, and our crew."

Throughout the film, Woody shows what a true friend he is and always will be, even as life presents new changes and adventures.

Is this really the end of this toy story? Cooley says he is happy with the way the film ended, and he feels they have completed Woody's arc. "So, if this is the last one, I'd be totally fine with that," he says. "I feel like people would agree. But, at the same time, at Pixar we never say 'never.'"

Wattage and lumens data were added into the carnival lighting system, so they illuminate automatically.