When a potential tent-pole film debuts, you can expect to see a range of licensed merchandise associated with that property popping up on store shelves. We’ve come to expect the various toys—video games, board games, and action figures—and even the fast-food accompaniments from McDonald’s and Burger King, for example. But more and more, movie characters are turning up just about everywhere—from aboard cruises to grocery aisles. Why? Kids love these characters, and parents are eager to satisfy their kids. And for companies looking for an added edge, it’s a marketing match made in reality—or possibly, virtual reality.
Eager to cash in on this trend, Nestle approached French film director/writer/ producer Luc Besson, co-founder of EuropaCorp movie studio, about a licensing agreement connected to his then-upcoming Arthur movie—Arthur and the Vengeance of Maltazard, the second in the series of CG animated films that Besson wrote, produced, and directed. In particular, Nestle was interested in licensing the popular movie franchise to use in a marketing campaign for its breakfast-cereal business in France.
Indeed, cereal companies have been using popular entertainment properties as a marketing vehicle for decades—for example, a small plastic toy inside the box or 45 rpm record (and later, CD) on the outside. For the Arthur campaign, Nestle chose an out-of-the-box approach, albeit one that is made for the current generation of kids: an augmented reality game created with cutting-edge CG technology from Dassault Systèmes, which makes 3D design and PLM products.
According to Mehdi Tayoubi, director of interactive strategy at Dassault Systèmes, the company was approached by Nestle, which was interested in designing an innovative campaign that would change the way the food distributor interacts with its top consumers: kids. Nestle then worked with a team from Dassault Systèmes, headed by Tayoubi. Using software from Dassault Systèmes’ 3DVIA division, they created an augmented-reality game in which consumers—using a webcam and Internet connection—could bring one of the movie characters to life by having it pop out from the cereal box.
As Tayoubi points out, the game appeared on two brands of cereal (Chocapic and Nesquik) in France only. Despite those limitations, the campaign was successful—at one point, increasing market share by approximately 1.6 points.
“It helped push cereal boxes off shelves,” says 3DVIA marketing manager Emmy Jonassen. “It is a pretty effective tool.”
One year later, in 2010, Dassault Systèmes once again teamed up with Nestle and Besson for another French campaign built around the director’s next Arthur film, The War of the Two Worlds. This time, the content-creation team decided to up the innovation ante, delivering a virtual-reality experience for which consumers would cut a pair of 3D anaglyph (red/blue) glasses from the box and again use their webcam and Internet connection for a novel interactive experience. The glasses contained a tag, which, when detected by the webcam, acted as the game interface, eliminating the need for a mouse, keyboard, or joystick.
“People could go online and ride a motorcycle around a course just by moving their head or body,” says Tayoubi. “It was the first time that customers could use the cereal box for interaction between, and 3D vision between, their movements and the immersive world.”
During the nine weeks of the campaign, Nestle shipped more than four million packs of Chocapic and Chocapic Duo, Chocapic Pepites, Nesquik, Cookie Crisp, Cheerios, and Chokela cereals featuring the 3D game to various stores in France. This time, consumers could collect three different augmented-reality tagged packages, each unlocking a different racetrack reflective of various locales in the film. The more industrious users also could build their own racetracks from parts found in the different universes.
Most impressive is the fact that the campaign featuring this interactivity launched on cereal boxes prior to Microsoft launching the Kinect gesture controller for the Xbox 360 console. As an added enhancement, the VR game was compatible with 3D televisions, making it the purported first online game to work with 3D TVs.
In the spring, Nestle and the Dassault Systèmes group partnered with Twentieth Century Fox/Blue Sky to create another augmented-reality game, this one based on the stereo 3D animated feature Rio (see “Stepping Out,” April 2011). Now, Nestle was ready to go global, rolling out the application on 26 different types of cereal boxes in 53 countries (but not the US). In all, more than 26 million boxes were part of the interactive promotion.
“We wanted to make it easy for any kid to access [the characters] Blu or Jewel with some interaction, and show they could have the same type of experience available on a video game console, only we were delivering it on a cereal box,” Tayoubi says.
According to Tayoubi, Nestle added a black-and-white augmented-reality tag on the back of the packaging, with simple directions to cut out the tag, go to the dedicated Web site (www.nestle-rio.com), and launch the 3D application. By moving the tag (depicted on screen as a cup filled with cereal) in front of the webcam, users can leave a virtual trail of cereal pieces to lead the parrot along the designated path to a full bowl of cereal.
Kids, games, and breakfast cereal: Nestle sees this as a winning combination. To this end, the company is offering various games, accessible online via an AR tag on its breakfast cereal packaging.
“There’s a game for the PS3 called EyePet [where you can interact with a virtual pet using gestures in real time]. Here, you are doing a similar thing, but you don’t need any complicated hardware. It’s all about the democratization of those technologies,” says Tayoubi. “We put this on a simple cereal box, and it works.”
Recipe for Success
Once the domain of science and the military, augmented reality in recent years has been moving slowly but steadily into the mainstream. Recognizing the widespread viability of AR and VR, Dassault Systèmes—which had been offering augmented and virtual realty to its traditional industrial customers since the early 1980s—invested in new forms of the technology coming out of the game industry. Subsequently, Dassault Systèmes purchased French company Virtools, whose development environment was used to create 3D applications across video games and corporate projects that could benefit from real-time interactivity. Dassault Systèmes’ focus, however, was not on the game industry, but rather on inventing new kinds of experiential projects by merging innovations from both the gaming and engineering market segments.
“My team took existing technology within the company and tried to push the functions of those technologies by applying them to new problems,” says Tayoubi. “With these [cereal] campaigns, we could demonstrate that the market has matured and the technologies are ready—now it’s only a question of using your imagination.”
A number of companies have been releasing interactive marketing projects in 2D, using Adobe’s Flash, for instance. “But with 3D, we can go beyond that in a different, more engaging direction,” Tayoubi points out.
Game On
When the Dassault Systèmes group created the first application, its main concern was that the application had to work on any computer with an Internet connection and a Web camera. “The first application was difficult to do,” says Tayoubi. “The second one was also a technical challenge because we wanted to push the limit beyond augmented reality and show something in virtual reality. The third campaign was similar to the first, so we focused more on the creative aspect. We were coming from the real-time world, but the content had to meet the expectations of the movie studio, and we wanted the imagery to improve [on our end] and meet those expectations.”
For the two Arthur games, the Dassault Systèmes group worked from a studio style guide to create the imagery from scratch. When it came to the Rio game, Fox/Blue Sky provided the artists with high-polygonal Autodesk Maya film models of Blu and Jewel. The models were basic, says Tayoubi—they were not textured, skinned, or rigged, and only contained a rough mesh. Nevertheless, they were still between 50,000 and 100,000 triangles—too large for real-time interactivity.
“We reworked all the models [using Autodesk’s 3ds Max] and adapted them for online real-time 3D, preparing them for integration with our 3DVIA Virtools interactive development platform,” says Tayoubi. “We made them lighter and decided that each character could not be more than 10,000 triangles.” The group also generated rich 3D environments. And to ensure maximum online compatibility, the group limited themselves to using vertex shader 2.0 and pixel shader 2.0, and adapted the online experience to the user’s connection speed with a high- and low-bandwidth version.
VR V. AR
What is the difference between virtual reality and augmented reality?
Virtual reality (VR) involves the creation of a computer-simulated environment with which a person interacts. The person is totally immersed within the synthetic environment, thus heightening the feeling of reality. A subset of virtual reality, augmented reality (AR) is the merging of virtual imagery and real life.
For virtual reality to be convincing to the user, the synthetic world must be created in painstaking detail. With augmented reality, the CG imagery must appear photoreal in order to seamlessly integrate with the real-world images. However, the computer augmentation has to occur in real time so that the experience is not disrupted.
“Augmented reality is part of the big family of virtual reality,” says Mehdi Tayoubi, director of interactive strategy at Dassault Systèmes. “They use the same type of technology, but AR is a dedicated usage where you mix virtual worlds with an input coming from the real world, such as a photo or a webcam.” |
To get a feel for the characters and how they were supposed to move, the artists reviewed unfinished sequences of the movie and a trailer. They also met weekly with the Fox/Blue Sky team to validate the animations they were creating and to ensure they were capturing the overall spirit of the property. “It was an iterative process,” says Tayoubi. The global gameplay was validated in the beginning of the project following a number of iterations made with 3DVIA software, which enabled the artists to create interactive experiences quickly and easily, as well as test those concepts through interactive prototyping.
In all, it took the Dassault Systèmes group approximately three months to create the augmented-reality Rio experience, which it did using the company’s software straight out of the box. While it is rare for Dassault Systèmes to act as a service provider, Tayoubi says that his creative team will do so, especially to help a partner, such as Nestle, with a challenging and innovative project.
“These campaigns really pushed the limit of the technology and enabled Nestle to communicate with kids on their level,” says Tayoubi. “A plastic toy inside the box isn’t going to do it anymore. This generation Y is complex—its members are constantly playing with their iPhones and PlayStations; they are interacting with their Kinects and other devices. They are expecting their preferred cereal brand to speak to them in a language they know and in a universe they are familiar with.”
While exact figures are not available yet from the Rio project, Tayoubi is optimistic about its success, given the number of visitors to the game site.
And just recently, the company announced that it is again teaming with Nestlé, and this time, DreamWorks to launch a campaign around Kung Fu Panda 2 with a new innovative and immersive augmented-reality gameplay. For the new application, the group followed the same content creation process with the team at DreamWorks as it did with Fox and Blue Sky for Rio.
Dassault Systèmes, using software from its 3DVIA division, teamed with Nestle and Twentieth Century Fox/Blue Sky Studios on an AR game built around the characters from the animated film Rio.
The gameplay for this new application is intuitive, promises Tayoubi: Using the AR marker, the user catches rice bowls attached on floating lights, all the while avoiding the fireworks with a character attached to a chain. The level is complete once the user has caught all the rice bowls, at which time the next level is unlocked, with another character attached to the chain.
According to Tayoubi, the creation team went a step further than before by integrating some physical components in the 3D engine to provide a realistic feeling that one is playing with an elastic yo-yo-like movie-character chain. To accomplish this, the artists used the same embedded physics engine to manage the characters chain as it did in the first Arthur game.
“We had to optimize the physics behaviors for the Web because we are using complex behaviors and animations,” Tayoubi notes.
The animators added physics to the 3D frames so they didn’t have to contend with the mesh topologies and the creation of the collision hull. By managing the damping translation and object rotation, they can avoid instability of the simulation.
“We can then create or delete constraints of the two-way springs between the 3D frames, thus creating the chain to which the characters will be attached,” explains Tayoubi. “Globally, we had to make sure that the parameters of the 3D frames and the springs are well balanced, resulting in a ‘nervous’ behavior, not an explosion of the calculated strengths.”
As far as performances are concerned, the calculation is done quickly—there is no collision detection. The physics are calculated at 100 to 120 hz minimum so that the chain is rigid enough but without the characters becoming stretched beyond reason. “On a good desktop PC, the application runs at 100 to 200 frames per second, including the rendering of the animations, but the game needs to be played on any PC configuration,” says Tayoubi. “Therefore, we needed to optimize the physics capabilities so that the chain doesn’t break too easily.”
In total, the game creation crew re-created five characters from the film: Po, Monkey, Tigress, Viper, and Crane, adapting the size of the characters for real-time and Web constraints. For instance, Viper is 3300 triangles, while Mantis—with its two wings, six legs, and two antennae) contains 6400. They, like the other characters, were built using the same tools and techniques as the other game characters, with film models from DreamWorks serving as reference.
For the more recent Kung Fu Panda 2 game, Dassault incorporated physics into the 3D engine to get a feeling of elasticity in the character chain, an integral part of the gameplay.
And, once again, the team focused on improving the quality of the animations and the visual effects. “For example, we added slow motion in real time when Po is doing is kung fu movements,” Tayoubi says, “and with the same technical constraints we had before.”
The most difficult character to animate, however, was Viper. “She has less bones (39) but she is still the most complicated to animate because we could only manipulate one spinal column,” says Tayoubi. “We had to set up complex rigs for each animation.” The animators used a derivative of Viper’s setup for Monkey’s tail. The animations later were combined and validated “live” within the physics engine, and re-adapted as necessary.
For the Panda 2 campaign, Nestle went even bigger and better as well: The rollout is worldwide and includes more countries than the Rio project.
Interactive Future
So, what is the biggest lesson Tayoubi has learned from these projects? It’s that the market is ready for new kinds of interactive 3D experiences, he notes.
Tayoubi compares the trend toward augmented and virtual realities as a marketing vehicle to the trend of going online in 2000. “Corporations were questioning whether they needed to go online for their brand messaging, or whether they should stick to traditional communication and marketing,” he explains, “and many of the traditional agencies did not have the skills or competency to use the Internet in this way. It’s the same challenge now—you just have to find the right partner with the right technologies to talk to consumers in a way they can understand.”
And, these Nestle campaigns are just a glimpse of what can be done to connect and engage with consumers. In just a short span of time, Dassault Systèmes evolved the application from augmented reality to virtual reality with body tracking and 3D vision. Now the group is contemplating the addition of a social networking dimension to those augmented- and virtual-reality experiences.
According to Tayoubi, Dassault Systèmes is also looking at the creation of trans-support experiences: “For instance, you can begin your 3D experience at home and share things on your social networks; then when you go to a digital theater, you can interact with other people in a fun way with the content you created at home.”
And in the middle of all these new, exciting applications will be augmented and virtual reality.
“3D TVs, digital theaters, 3D personal projectors, tracking cameras (a la Kinect), gyroscopes, and so forth are becoming mainstream, and 3D viewing and tracking are becoming standard functionalities that you can chose whether to activate,” says Tayoubi. “The interactive world between the world of 3D engineering and the world of hard-core gaming is huge. More and more, interactive 3D in the form of serious games for many purposes will be produced. Consumers are ready, the hardware is ready, and our 3D development platform is ready. It’s now a question of imagination and scenario.”
Are you ready to be engaged?
Karen Moltenbrey is the chief editor at Computer Graphics World.