Mean Streets

When Christopher Nolan’s Dark Knight exploded onto the cultural landscape in 2008, and Rocksteady Studios’ Arkham Asylum arrived on its heels in 2009, the groundbreaking film and equally groundbreaking video game (still lauded by The Guinness Book of World Records as the “best superhero game ever”) were a dynamic duo that set collective imaginations on fire, transcended genre, broke sales records, and established new, almost unreachably high standards for comic-book art in their respective mediums.

Now, that dynamic duo is poised to return with a one-two punch that culminates with Nolan’s Dark Knight
Rises in 2012 and begins with Rocksteady’s Batman: Arkham City, the eagerly awaited follow-up to Arkham Asylum that sold a staggering two million copies in its first three weeks alone.

Written again by Paul Dini, directed by Sefton Hill, and art-directed by David Hego, the sequel eclipses the scope and scale of its predecessor in almost every way, lifting Batman out of the claustrophobic confines of Arkham Asylum and releasing him onto the mean streets of Gotham, an environment that’s more than five times bigger. Scaling up the playing field meant scaling up the cast of villains and thugs—a population explosion that has the city overrun with almost every super-villain from the Batman mythos. As a result, Rocksteady had to adapt Batman’s gameplay to the massive Gothic sprawl. They gave him a Power Dive to glide between buildings and the ability to chain attacks to contend with the relentless gang assaults, sometimes comprising as many as 30 or more assailants—a far cry from the one-on-one combat of Arkham Asylum (see “Dark Matter,” October 2009).

The sequel is set one year after the original. Batman has foiled the Joker’s plans to poison Gotham’s water supply with the zombie-making Titan chemical, but Quincy Sharp, former warden of Arkham Asylum, has taken credit for the collar. Parlaying his notoriety into a successful bid for mayor, Sharp’s first act is to buy out a large section of the slum-infested North Gotham to house the burgeoning inmate population, creating a makeshift prison-city policed by a private military contractor called Tyger.

To oversee the so-called Arkham City, Sharp hires psychotic psychiatrist Hugo Strange, who not only has a hidden agenda for the city, but also knows Batman’s true identity, “leaving him vulnerable and exposed in a way he’s never been before,” says Hill. Surveying the open city from atop his gargoyle perch, watching from a distance as it factionalizes under each villain vying for rule, Batman is eventually forced into the city when Two-Face kidnaps Catwoman, his former love, and devises a plot to publicly execute her. Through it all, Batman tangles with Catwoman and allies with Robin to stop Gotham from descending into total chaos.  

‘Batman in Gotham’ Feel

Whether Batman’s motivation is love or heroism, Rocksteady’s primary motivation for relocating the Caped Crusader to Arkham City was to deliver, according to Hill, that “Batman in Gotham feeling.”

“That sensation of gliding through the streets of Gotham City as the Dark Knight was one of the key objectives we set for ourselves,” says art director David Hego. “Moving the action out of the asylum and onto the streets was a huge creative and technical undertaking; Batman’s navigation abilities needed to step up, providing an entirely new set of gameplay opportunities for the player. From an artistic perspective, the priority was to create a world suffused with a lot of realistic elements so it would feel believable but still uniquely Gotham.”

This uniquely Gotham feel borrows and expands on the architectural styles and atmosphere set in Arkham Asylum. Like before, crumbling Gothic and Victorian buildings abound, where old-fashioned turrets, spires, and gargoyles clash with glaring splashes of neon signage. For Arkham City, however, art directors added flourishes from other architectural and art movements of the 20th century.

“Of course, the architecture is reminiscent of Arkham Asylum for the simple fact that we wanted the world we created to remain consistent and logical,” says Hego. “However, we expanded on the world architecturally, buttressing it with new conceptual pillars. Gothic and Victorian-style structures are still present as the foundation and DNA of Gotham City and its dark feel. On top of these two strong styles, we’ve added Art Nouveau elements in the architecture and design. It’s fascinating to explore real-world history and borrow elements to re-create piece by piece for our world.”

In addition to borrowing from real-world history, the team extracted visual threads from the early history of cinema and wove them into its dark, visual tapestry. “Another inspiration for the atmosphere of Arkham City came from German expressionism (think 1920s The Cabinet of Dr. Caligari). We took cues not so much from structures and perspective, but more in the way we lit the world, with crude light and shadows, which is appropriate for Gotham City,” adds Hego.

The multiplicity of villains greatly informed the set design, too. Over the course of the game, Batman squares off against a who’s who of villains, including Mark Hamill’s Joker, a Cockney Penguin, Two-Face, and Mr. Freeze. Each villain has staked out his or her own little enclave in Arkham City, where the architecture, graffiti, lighting, and art direction personify the unique psychology of the character. It’s a city diversified and variegated by villainy. Hence, as players make their way from one enclave to another—say, from the courthouse of former DA Harvey Dent (aka Two-Face) to the Penguin’s Iceberg Lounge—they had to feel as if they were making a physical transition to another “emotional space” through the art.

“A great example of this is the Solomon Wayne Courthouse, where Two-Face is holed up,” says Hego. “Not only is the location specifically relevant to him as the ex-district attorney of Gotham, but he’s also remodeled the building to reflect his own duality. The right side of the courthouse—both inside and out—is defaced, like his own mutilated right half, smashed up and burnt, symbolizing his lust for chaos and carnage, while the other side is perfectly rendered in accordance with his belief in order and justice. So, in this way, moving from one district to another is signposted by subtle changes in the features and landmarks of the street.”

By the same coin, the Joker has established his territory in the Sionis steel mill nestled in the industrial part of the North Gotham docks. Here, the Joker’s gang has redesigned the area into a massive, morbid funfair. “The mix of funfair elements with the industrial setting creates an explosive environment, rich in color, just like the Joker’s personality. It was a great experience trying to imagine how each villain’s faction would mark its territory,” adds Hego.

This ghettoized city stretches out before Batman in great vistas when he enters his Power Dive, arcing over streets and the skyline. “The city is such a rich, dense place, filled with these little iconic elements and details,” Hego points out, “that we had to be clever with what we display on screen. To that end, we employed a complex LOD system to hide superficial details at a distance, while keeping texture density and geometry impressively high at street level or while grappling between buildings.”

Unreal to the Max

Using Epic Games’ Unreal Engine 3 and Auto­desk’s 3ds Max, artists forged all Arkham City’s texture maps, geometry, and lighting. The German Expressionist cinematography—crude, angular, brooding—came mainly from the way the moon lights the world. “It’s not just about the lighting by itself, but about how the light interacts with the materials, the normal map, and the specular levels of the snow, the water, the buildings; and the way the water towers and chimneys cut through the moonlight with dynamic light shafts,” says Hego. “That’s the key to capturing the striking Gothic atmosphere.”

Another crucial light source in the game, of course, is the Bat-Signal, not just because of its connotation within the Batman universe, but for its narrative function, too, pointing the player to the next objective as it refracts and reflects off smoke and clouds. “The Signal can be placed arbitrarily anywhere on the map by the player (which means it could end up too distant and dim), so we had to find a way to make its integration coherent without feeling fake. Using the stock lighting of the Unreal engine, somewhat re-engineered by our coders, we decided to use an arrow-style representation of the cone of light through the smoke, instead of a solid light cone. With the arrow-style lighting, the Signal achieves its functional purpose and is visually impactful without looking out of place.”

Rogues Gallery

The city’s massive cast, composed not merely of homogeneous non-player characters (NPCs), but of highly unique super-villains and their equally unique minions, put Rocksteady’s character modelers to the test. After sculpting a rough base mesh in 3ds Max, modelers refined the geometry in Pixologic’s ZBrush to produce a high-resolution version of each character. From this, they created the in-game model and extracted the normal map. “The poly counts of the in-game models aren’t low, ranging around 15k per character, but the normal map is still vital, to keep all the details of the high-res version,” says lead artist Pablos Hoyos.

While most of the intricate details—wrinkles, scars, caking face paint, and so forth—were baked into the normal maps, artists used ZBrush and Adobe’s Photoshop to paint diffuse maps, specular, and specular power maps, as well as transmission maps, to simulate subsurface scattering of light through skin, flesh, and veins. “We always try to add as much detail as we can, especially in the faces. We have skin imperfections, like moles and skin marks, different types of pores, stubble, skin lines, wrinkles, skin tones, and so on. All these details are present in each map of the shader system and, when layered together, produce an astonishing sense of realism,” says Hoyos.

Indeed, unlike the square-jawed, nondescript neckless grunts who inhabit most games, Arkham City’s faces reflect the subtleties of strong, nuanced personalities. “Some of the faces presented quite unique challenges, such as Two-Face’s burnt flesh, Solomon Grundy’s “zombified” look, and Penguin’s old skin, which he cakes in makeup because of his vanity,” says Hoyos.

Separating the villains with a colorful individuality was a challenge, contends Hoyos. Catwoman was all about playing with her proportions until we got the right mix of beauty and sex appeal. Penguin was all about making his face look pure evil, and the broken glass monocle was an unusual, character-defining touch. Mr. Freeze’s armor is a complex assemblage of many individuals, so modelers had to carefully plan out their work with the rigging and cinematics teams to ensure the pieces behaved correctly (without intersections) and looked as good as possible.

Mocap Method Acting
    
To drive the characters’ animations, riggers built an IK skeleton using a basic 3ds Max biped rig that was augmented with additional facial bones. “Because we also have to deal with a lot of motion capture, we built a version of this IK setup that runs in MotionBuilder— the primary work space for motion-capture-based animation,” says lead animator Zafer Coban. Unlike Arkham Asylum, in which animators relied heavily on normal-map blending for delicate deformations in subtle facial expressions, this time Rocksteady wanted to push the range in the performances by relying more on mocap.

“The primary difference between the two games relate to Batman’s face and face setup,” says Coban. “Primarily, we wanted to enhance his performances and those of all the characters by developing a facial motion-capture pipeline whereby face actors would repeat the lines of the original voice actors [like Mark Hamill or Kevin Conroy, who plays Batman], providing facial acting in the process.” This necessitated a more highly articulated facial rig, granting a much larger and more accurate range of possible motion.

In more than 200 motion-capture sessions, Rocksteady shot a total of 14 hours of facial data and 17 hours of body motion for the in-game animation (excluding the hours shot for cinematic sequences). In fact, every single character’s animation set includes some motion capture.

The motion-capture setup at Rocksteady is an optical, marker-based Vicon system comprising 32 cameras. The team uses Vicon’s Blade software to capture and process the initial data. “We have a solution that ensures good, sturdy data is baked onto the skeleton within Blade before moving things on to the next stage in MotionBuilder,” says animation programmer Tim Rennie. Here, animators can take the original, actor-scale performance and drive the final in-game character setup. Tweaks and embellishments to the performance happen in MotionBuilder, but much of the final animation is keyframed in 3ds Max, which is the final destination before export to the engine.

“We produced about 45 minutes of final facial capture for most characters, during which time our facial actors match their voices to the original actors’ performances,” says Rennie. This involved a complex process in which the facial actor would repeat 10-second clips until we had a perfect, fully synced performance. These clips would be captured with their correct time code so they could be reassembled in MotionBuilder. In MotionBuilder, animators would use video reference from the capture to combine everything—marker data, hand-tweakable controls, and automatic correction scripts—to drive the final performance. The resulting facial animation could then be merged separately onto the body capture within the Unreal Engine without the animators having to worry about breaking the sync to the final audio.  
 
Along with facial capture, the facial animation system also employs OC3 Entertainment’s FaceFX, depending on the importance and complexity of the scene. Using blendshape targets in a fully articulated FaceFX rig, animators could keyframe expressive eye animations or subtle facial tweaks to punctuate a particular body movement, polishing very quickly hundreds of lines of dialog. “In addition, we also constructed a hassle-free pipeline to quickly embed FaceFX animations onto body animations for all in-game movements,” says Rennie. Small mercy in a game with 100-plus unique faces to animate, excluding weight variants, like the fat or thin.

Feline Finesse

While Batman remains the story’s main hero, for about 10 percent of the game, the player can slip into the sleek, skin-tight leather of Catwoman. Armed with a whip and bolo, Cat­woman exploits the chaos in Arkham City to go on a kleptomaniacal rampage, thieving jewels and valuables like there’s no tomorrow. In her first mission, to steal an orchid for Poison Ivy, she glides lithely across rooftops and alights upon some unsuspecting Tyger security guards standing over a manhole cover—her access point to a maze of sewers leading to a vault. Inside, she performs her signature “ceiling climb,” dropping down on guards to pickpocket their keys. When the alarm blares, she unloads with a flurry of fluid roundhouse leg kicks that would daze Batman with their blinding speed and grace.

In designing Catwoman’s gameplay, Rocksteady’s first objective was to make sure players did not feel like they were guiding a curvier version of Batman. This entailed a wholesale reworking of the rigging and weighting of the standard IK chain, enabling greater speed and flexibility in her animations.

“Catwoman’s rig is, of course, unique to her. She’s a lot slimmer, shorter, and has a bunch of bespoke controls for her whip,” says Coban. “As soon as we decided to include Catwoman as a playable character, we wanted the gamer to have a totally new experience of the Arkham City world, not a re-skin of any sort. With combat, we’ve taken our influences from acrobats, gymnasts, and ballerinas to bring a unique flavor to her fighting style. We’ve concentrated on legwork more, and left the hard-hitting, brutal punches to Batman himself.  

For example, Coban says, when Batman hits, the impact registers with sheer, blunt-force trauma, whereas Catwoman’s attacks, while less impactful, are faster, more agile, athletic. “We played with those elements, and it really shows during combat. Players will particularly appreciate this unique legwork in her Stealth Predator gameplay, where she’ll flip up onto a thug’s shoulders, wrap her legs around his arms and head, and choke him out with those long, hardened legs.”

Everything from Catwoman’s gadgets to her navigational skills is custom-made for the feline fatale. “She doesn’t have the Grapnel Gun, so we’ve given her the Whip Swing and the Claw Climb; altogether, it looks and feels very different playing her,” says Coban. Catwoman is also a prime example of Rocksteady not only pushing facial and body mocap to enhance performances, but also run-time physics to enhance the dynamic motion of hair, coats, straps, and, specifically, Catwoman’s whip, which snaps and coils with astonishing realism. “We always had the ability to add additional movement to a character’s animation using run-time physics simulation, but we really pushed it hard on Arkham City.”

Cape Animation

Newly enhanced run-time physics simulation also underlies much of Batman’s improved cape animation, which billows in the wind, unfurls during the character’s Power Dive, and pleats and settles as he slows into combat mode. The in-game cape combines a mixture of elements: real-time cloth simulation driving a skeletal mesh rig; hand-keyed skeletal animation; and off-line cloth simulation, authored in 3ds Max cloth and baked onto skeletal animation. To produce the ultra-realistic clothing animations in the pre-rendered cinematics, artists baked this off-line cloth sim onto vertex animation.

From the outset of production, the team wanted to have extremely fine control of the cloth but also ensure that it was reacting in a natural and dynamic way to the environment, the weather, and Batman’s movements. “The biggest change during the development of Arkham City was the redeveloping of the cape rig midway through production. We had to make it easier for animators to pose the cape more intuitively for any particular action,” says Coban. This pose would then form the driving shape for the final physics simulation at run time.

During most of the gameplay, the in-game cape is pure real-time simulation, but when a particular iconic move or a stylized result was needed, the keyframed animation kicks in. In some situations, an animation is used for the overall shape, while the sim adds physics detail at the edges. For example, the wind rippling through the cape is achieved by level artists placing volumes in the world nearby. “We spent a lot of time trying to retain a high level of animator control while still running the cape under live physics simulation,” says Coban. Balancing animator control with live physics simulation was also crucial to animating Robin’s staff. “It’s a complicated piece of kit that can bend, flex, extend, and turn into a shield, all while he’s swapping it from hand to hand,” he adds.

Combat Choreography

To handle the crush of assailants and the sprawling, open setting, Batman’s range of movement for maneuvering through the environments and for hand-to-hand combat has undergone an aggressive expansion. In fact, his animation set has doubled. According to Hego, the expansion of the game world drove a redesign of every aspect of Batman’s navigation and combat, as well as a huge overhaul in the way that the team conveyed story and narrative elements to the player. “For example, the enhanced Power Dive—through which Batman gets around the city and the player experiences the freedom and exhilaration of flying through alleys and over the skyline—was a completely new development challenge for us, resulting in the full-momentum gliding system,” he says.

Furthermore, doubling the number of combat moves was essential to convey a sense of variety in Batman’s combat skills, so critical to the feeling of power and dominance offered by the FreeFlow Combat system.

Waylaid by Oswald Cobblepot’s goons and the Frankenstein-like Solomon Grundy in the Iceberg Lounge, Batman chains his punches and kicks to clear the room, following up a roundhouse kick to one thug with a swift leg sweep to another in a seamless series of multiple, simultaneous counters, all the while reacting to thrown objects and without the slightest hitch in the blending system.

“In Arkham Asylum, thugs would generally attack one at a time, but in Arkham City, we’ve blown that out of the water, letting thugs rain punches and kicks in simultaneous assaults that really make the player feel pressure—as they would in a real fight,” says Hego. “Consequently, Batman can now perform double and triple counters, whereby he deflects and dodges all these blows and sends counter­attacks [on multiple characters] in one swift move.”

The daunting task of programming AI for these complex ensemble fights and endless counterattacking fell to AI programmer Tim Hanagan and his team of coders. “There were so many challenges involved in increasing the crowd combat from 10 or 12 to about 30. First, we had to optimize the performance of all the various systems so that these large-scale fights could run at a consistent 30 frames per second,” he says. “The second was managing the positioning of so many enemies, to prevent the fights from feeling too cramped.” From a visual standpoint, the group had to allow the player to see clearly and assess each situation. Most of these challenges were addressed through the studio’s custom character collision system.

Implemented within the Unreal engine, this character collision system replaces the existing stock Unreal system with one that’s much faster, more efficient, and streamlined.
Hanagan explains: “It uses the navigation mesh data to allow faster collision queries against an approximation of the real-level geometry. This was a major contributing factor in allowing us to support so many active characters at once.”

The team additionally implemented a real-time path-smoothing system to improve the look and realism of the paths the AI take when traversing levels. In addition, they developed a character scripting system within the Unreal Kismet scripting editor to allow animators to implement complicated scripted events without any code support—all this while still allowing for a high level of player interaction. While the character collision system has the capacity to support more than 30 combatants in some areas, Hanagan cautions that pushing the crowd beyond that number only made the gameplay confusing.

Batman is also armed with a new “context-sensitive mechanic,” which allows him to integrate his immediate environment into his fighting—improvising with a nearby railing, brick wall, pillar, or street lamp—to subdue assailants. These context-sensitive moves require code that could rapidly sample the local area to identify whether the surrounding geometry could be used within the current combat move, notes Hanagan. The problem with all systems like this, however, is trying to make sure that the developers balance the accuracy of the checks with the need for fast run-time performance. “You always want to minimize the number of line checks performed, but at the same time, you don’t want to end up performing a wall animation on a 10 cm-wide lamppost, or end up slamming a thug’s head into what should be a railing but, in reality, is the space between two railing-high benches,” he says.

In building this robust combat system, Rennie contends that the most important thing wasn’t any particular piece of technology, but the animators, gameplay coders, and tech artists all sharing the same studio space and collaborating closely. Beyond this close collaboration, Rocksteady’s artists relied on the studio’s own animation blending system, which harnesses all the standard tools and techniques. This includes cross-blending, time warping, additive animation, motion extraction, and mirroring. A particular focus was placed on automatically aligning animations. For example, if two characters are interacting with each other, the system will automatically blend them into the correct position based on their relative positions in the animation. If a character wants to interact with an item in the world, then a positional marker embedded in the animation tells the system how the character should be aligned.

Of course, much of this complex interaction with Gotham’s urban jungle (as Batman or Catwoman) involves scaling walls and ledges, climbing through sewers and ventilation ducts, trying to gain a precarious foothold or handhold on a cornice, gargoyle, crack, or crevice. For aligning hands to walls and ledges, Rocksteady used some of the Unreal Engine’s built-in arm IK. However, the team tackled most of the challenge by building the environments to standard grid sizes and then animating to those sizes. So for a wall climb, there are separate animations for 128-, 256-, and 384-unit height walls, and if the animations don’t quite match up to the real wall, then the artists would use a blend to shift the entire character.
 
“For aligning the character to the floor, we dynamically calculate a virtual floor plane that approximates the actual floor geometry underneath the character. A standard two-bone leg IK then skews the animation to fit the plane. In some situations, where leg IK is insufficient (like Catwoman crawling on the ceiling), we rotated the entire character to fit the virtual plane,” says Rennie.

Dark Knight Rising

Indeed, Rocksteady’s commitment to storytelling, high production values, and acting—both animated and mocapped—has delivered the studio to the forefront of the industry’s leaders, and Arkham City to the forefront of contenders for Game of the Year. At this year’s Comic-Con, Hamill, Conroy, and Dini held court on a panel that was one of the convention’s biggest draws, no small feat considering the presence of Peter Jackson and Steven Spielberg pushing Tintin.

And it’s all by design, too, for the seeds of Arkham City were laid, like Chekov’s gun, in secret plans hidden in a backroom of warden Sharp’s office two years ago in Arkham Asylum. Who knows what little bread crumbs have been dropped for future sequels in Gotham’s mean streets. Only Rocksteady knows.

What’s certain is that the developer’s achievements left fans waiting with bated breath for this sequel, and if Rocksteady holds fast to its resolve of pushing the bar higher and higher, The Guinness Book of Records may find itself passing the mantle again…and again.  

Martin McEachern is an award-winning writer and contributing editor for Computer Graphics World. He can be reached at martinmceachern@hotmail.com.