 |
Virtual humans play an important role in keeping
their human counterparts safe. |
Several years ago, professor Karim Malek had a small lab in the University of Iowa, staffed with approximately 10 students working on a research project. They wanted to see if they could predict and simulate human movements by applying robotic principles to human anatomy. As their project gained momentum, the US Army came knocking on their door, with a check for $2.7 million.
Malek explained why the military took an interest in his work: “You can do a lot of testing [on a tank design] in the virtual world. Thermodynamics, aerodynamics, stress tests—all these can be done on a computer. The only time you must build the tank is when you want to put a soldier in it, so you can ask him or her, ‘How does it feel inside there? Can you assemble this system? Can you engage the target or see through the visor from where you’re sitting?’ But what if you can put a virtual soldier into the virtual tank?”
With a robust bank account to attract top talents, Malek assembled a team comprising nearly 30 experts from all over the world to take on the Virtual Soldier Research project for the Army. Then contracts began rolling in, from Caterpillar, Rockwell Collins, the US Marines, Ford, GM, and many more entities. Nearly a decade later, having invested close to $30 million to refine the technology, the professor’s humble lab gave birth to Santos and Sophia, a digital duo for the commercial market. Malek is now part of a team that oversees SantosHuman, Inc., a company spun off from the lab.
The young firm (formed in 2010) represents and licenses the intellectual property of the University of Iowa. The university conducts research and development; the company markets and distributes software suites.
Santos and Sophia are part of a new outsourcing trend. This time, the jobs are not shifting from high-salaried, skilled workers in the First World to low-cost laborers overseas. Instead, they’re shifting from human workers to digital workers. But you may take comfort in the fact that the types of operations ceded to digital humans are the works many of us would consider too dangerous, stressful, or painful to perform to begin with. (Sometimes, they’re literally backbreaking jobs.) When the Army needs to figure out how a 10-pound vest would affect a sniper’s agility and fatigue level, when an automaker needs someone to crawl into a tight space previously untested, when a plant manager needs someone to perform a series of ergonomically risky maneuvers, they may now turn to digital human models (DHMs).
 |
| Some manufacturers use digital human models to identify and correct ergonomic issues in tight-fitting vehicles, like the cockpit of a plane. The simulation shows Siemens PLM Software’s Jack and Jill. |
Working with CAD Geometry
DHM-incorporated simulation exercises often involve 3D CAD files because the human-machine interaction usually takes place in a virtual environment, detailed and populated with standard CAD objects (3D models of armored vehicles, mechanical assemblies, and plant layouts, for example). Studying product designs (how a driver might navigate inside a smaller-than-average electric car) and manufacturing operations (how a repair technician might install an exhaust pipe) invariably require a mix of files created in different CAD programs. Therefore, for DHM-incorporated simulations, a solution that can accommodate— in other words, import—not just the software maker’s proprietary format, but also many different CAD formats, is preferable.
Many design, engineering, and simulation software makers have been developing and perfecting their own technologies to cater to this relatively new field. Siemens PLM Software’s digital humans, called Jack and Jill, are part of the company’s Tecnomatix digital manufacturing software suite and integrated with the company’s NX CAD package. A basic version of PTC’s digital manikin is included in the company’s flagship product, Creo Parametric (formerly Pro/Engineer). Another version with greater functionalities, dubbed PTC Creo Manikin Extension, can be purchased from Dassault Systemes’ Virtual Ergonomics Solutions suite, which includes male and female digital manikins called Teo and Sia, and are integrated with the company’s CATIA (CAD), DELMIA (computer-aided manufacturing), and ENOVIA (product lifecycle management) software packages. SantosHuman partners with Okino Computer Graphics, a CAD translation technology developer, to make its DHM software compatible with industry-standard 3D file formats.
Human Constraints
Unlike the type of semi-autonomous extras and player-controlled characters commonly found in video and computer games, digital humans used in simulation are designed with significant motion restrictions. In virtual environments like Second Life and World of Warcraft, an avatar could be a 9-foot-tall, blue-skinned elf archer with superhuman strength. After a few quests, he may even be permitted to wield a club twice the size of his body mass or fly through the clouds. After all, doing what’s not physically possible in real life is one of the main attractions of the virtual world.
In simulation applications, it’s important to prevent digital humans from performing moves and maneuvers that are outside of what is possible within the human population. The restriction is deliberate. It’s meant to give users—army engineers, facility managers, plant designers, and ergonomists, among others—confidence that the digital humans’ reaches, crouches, and bends accurately represent an average soldier’s or employee’s range of motion, along with his or her limitations.
“The breadth of things our digital humans [Jack and Jill] can do cover the breadth of things real humans can do,” explains Tom Hoffman, director of Tecnomatix Global Marketing for Siemens PLM Software. John Buchowski, vice president of product management at PTC, notes, “The degrees of freedom available to our manikin are the same ones available to real humans.”
“In the movie
Avatar, the Na’vi characters are 10 feet tall,” notes Julie Charland, product manager of Virtual Ergonomics at Dassault Systemes. “In our applications, the manikin has to be no bigger than [typical] humans, with accurate anatomy and kinematics controlling their movements. If you pull on our manikin’s arm, for example, it will only stretch as far as a human arm will—it can’t go farther. Game avatars don’t need to know where their center of gravity is. Our manikins need to know that.”
“The difference between 3D characters in games and DHMs in engineering software is the difference between making something look good and making something right,” notes PTC’s Buchowski. “When you’re dealing with human behavior simulation in product design, there’s actually a library of typical body types you need to use as reference: for instance, 50 percentile North American male or 30 percentile Asian female. I challenge you to find a woman who matches Lara Croft’s proportions in real life.”
 |
| The CG Santos digital human is versatile; here “astro” Santos provides vital information to real-life researchers and technologists. |
The Science of Agony
DHM simulations produce more than visual references for movements, reaches, and lines of sight. They’re designed to collect other data, such as the amount of force applied to joints (often called joint torque), joint strength capabilities, internal muscle forces, and intervertebral disc compressive force. This allows army engineers, product designers, and ergonomists to address questions such as, How long can a pilot remain in a small cockpit without feeling stress on his or her back? How long can an average person continuously lift and drop a 50-pound part in an assembly line? How far would a repair technician need to bend to remove a cap? Is it ergonomically safe for the technician to execute such a maneuver? On the Virtual Soldier Research project, Malek once received directives from a colonel in the US Army. “He said, ‘I want to be able to find out how long and what distance I can have [a virtual squadron] walk before I allow them to sit down, chew on ca ffeinated gum, and have a drink of water.’ These are the type of questions they want answered,” Malek says.
That meant Malek and his programmers had to embed in their digital human, Santos, certain biomechanical intelligence, including his energy expenditure, the rate at which his fatigue increases, and the impact of the armor’s weight on his mobility over time. For validation, programmers review motion-captured data of real humans performing similar tasks to make sure Santos’ simulated behaviors and biomechanical feedback re flect the same outcome.
If you ask Santos to perform something that’s not humanly possible, Malek explains, “Santos will come back and say, ‘You’re asking me to carry something that’s too much for my elbow joints and muscles—the load is much higher than what I’ll ever be able to carry.’ ”
 |
| Sophia is the female counterpart of Santos. Both are virtual models created by Karim Malek at the University of Iowa. |
Issuing Directives
In games and movies, all the behaviors and movements of a character, from its facial expression to its distinct walk, are governed by a skilled animator’s script. In simulation, however, DHMs must come with a software interface accessible to those with little or no animation skills. It’s safe to assume that most DHM users will not know how to set up character rigs and define paths. For these users, commanding the DHMs has to be as straightforward as selecting a manikin and choosing a standard action (Walk, Go Here, Reach, and so forth) from a menu. To perform these tasks, digital humans must rely on the built-in kinematics, with little or no intervention from software users.
"One of the advancements made in the DHM technology is to move away from the keyframe-based animation used by [movie and video game] animators,” says Siemens PLM Software’s Ho man. “Here, what we’re doing is instructing the digital human to perform a task. If we say, Reach for this object,’ it can gure out on its own how to move there.”
Dassault’s Charland points out that 10 years ago, that was much more di cult. “Now, it’s much easier. ings that took about 10 clicks now take about three clicks, because when you tell the manikin to grab something, it knows exactly how to reach for it and grab it,” she says.
Most of the experts hired by Malek came with biomechanical and simulation expertise. But two years ago, Malek hired two senior programmers with experience in video game interface design. Their task was to revamp Santos’ interface so it would be more accessible to non-technical users. “Our mandate is that after three days of training, people should be able to use the [Santos] software,” he says.
Use-Case Scenarios
According to Dassault’s Charland, some progressive airplane manufacturers are beginning to consider end-of-life disassembly procedures: How should a plane be dismantled when it has reached its retirement? “In these cases, they use [DHMs] to simulate the process because they want to know how people might get to di erent parts of the plane and remove them,” she explains.
DHMs prove to be particularly useful in simulation exercises where a certain stressful or dangerous action must be performed repetitively in order to understand its impact on human anatomy. “If you use real people to simulate [an assembly operation], you cannot possibly run tests on an entire cross section of the population,” says Dassault’s Charland. “That is something you can do with DHMs. The technology is so powerful you can answer a lot of what-ifs: What if we move this part from here to there? What if we move the opening another foot?”
Fifteen years ago, plant managers and ergonomists had no easy way to identify certain repeated motions that, in the long run, prove hazardous. Today, using DHMs, they can reasonably predict the long-term impact of certain operations on workers. The aim is to prevent and reduce injuries by designing a safe, risk-free environment for manufacturing.
 |
 |
| (Top left) A basic version of PTC’s manikin is included free of charge with its flagship CAD software Creo Parametric. An advanced version, Creo Manikin Extension, is available for purchase. (Top right) As shown in the Sequence Editor dialog window in Siemens PLM Software’s Tecnomatix, digital human software makes it easy to construct a series of actions the model needs to perform. |
All Shapes and Sizes
Looking back at earlier incarnations of Siemens PLM Software’s Jack and Jill, Hoffman admits that, originally, figures were much chunkier, more rigid, and didn’t deform well. “Now, a mesh network covers the figure,” he says. “That lets us represent the shapes of different people more accurately than before.”
Accurate representation of body types is important not just for aesthetics, but also for accuracy of the analysis results. In many cases, designers employ DHMs to understand how people in the far ends of the spectrum—those who are extremely tall, heavy, thin, or short— will react to their products. For example: How much legroom should be engineered into a cockpit to accommodate the tallest pilot? Is the lawn mower seat big enough to fit the heaviest potential user? Being able to adjust and customize the DHMs gives users a better understanding of the hazards posed by the design, the risk of injury involved in a certain factory layout, or the discomfort a consumer may suffer when using a product.
“Our [DELMIA] manikin is resizable to match different segments of the human population,” notes Dassault’s Charland. “This is important because, today, a product designed in one place of the world might be used somewhere else in the world. So if you are manufacturing your design in Malaysia, for example, you’d want to resize your manikin to match the typical Asian population there. By default, we keep our manikins within average sizes, with normal range of motions, but if someone knows something about the target user, they can tweak the manikin further to get the right range of motion with DELMIA software.”
Although currently it is not the primary focus of DHM technology developers, academics and researchers have now begun studying and compiling data on under-represented segments of population, such as disabled people. Ron Hamameh, who authored the paper titled “Digital Human Models of People with Disabilities” (Digital Commons, Wayne State University, January 2010), observes, “With the injured veterans from the Iraq and Afghanistan wars returning home and the baby-boomer generation exceeding retirement age, there is an increase in the disabled population, the elderly population, and the need by both those populations for assistive technologies. With DHM software programs being utilized by more and more industries, including the medical device and assistive technology industries, it only reinforces the notion that DHM software use will increase dramatically over the next few years.”
Demand for Realistic Visuals
“Five or six years ago,” recalls Dassault’s Charland, “the manikin looked more like a robot.” The complaint the company often got from users was that kids’ $50 video games had better- looking humans than the high-end professional engineering software.
“The DHM figures are going to be much more lifelike,” says Siemens PLM Software’s Hoffman. “Things are definitely going that direction. [Researchers] may focus on ergonomics, on answering specific questions, but the people they present their findings to—upper management—are usually influenced by the visuals they’ve seen in the games their kids are playing. So they find it difficult to trust analysis results where figures look inferior to what they’re used to seeing in video games. That’s a challenge we recognize.”
Some DHM software allows users to incorporate scanned 3D data (usually saved as point-cloud data) and high-res images. They can be used much in the same way as texturemapping on 3D volume to create video game characters with realistic skin and clothing.
 |
| Professor Karim Malek’s research project led to the development of SantosHuman, Inc., specializing in using digital soldiers to simulate military operations. |
The Next Frontier
Malek currently serves as president of the Inter national Human Simulation Society, a relatively new industry group. Design and simulation software developers have quickly jumped on board; founding members include Siemens PLM Software, Autodesk, and Dassault Systemes. At the Society’s first International Summit on Human Simulation early last summer, Ulrich Raschke, Siemens PLM Software’s director of human simulation products, was named vice president.
“I’m working with Ford, GM, and Chrysler,” Malek says. “Human modeling is such an important issue for them, particularly to reduce injuries on the assembly line.” One of the society’s objectives is to establish industry standards for digital human modeling.
Although better graphics and increased power in computing now enables Santos DHMs to show results in real-time animation for straightforward scenarios (for example, predicting posture when reaching for a certain lever), the more complex jobs—like asking a digital marine to perform a series of tasks with a certain load, then generate a detailed report of the force, torque, and fatigue—can take some time, from three hours to two days. Santos software can take advantage of parallel processing for most jobs, so users with a highperformance computing server running many cores simultaneously will see greater performance. But optimization formulation remains an area that cannot be parallelized due to the algorithm involved.
At the present time, DHM technology employs a mix of robotic, simulation, and biomechanical principles. But the demand for more accurate results may push developers to incorporate a dose of psychology, as well. Malek has a daunting list of requests from his clients in the military; they want him to add cognitive parameters into the software code.
“ They want to load the digital marine with [inputs such as] being scared, his mood this morning, and whether he’s battle-hardened,” explains Malek. “ These soft [inputs] are important to the client, but they are difficult to model.”
The challenge for Malek as well as other digital human model developers is to figure out a way to represent fear, anxiety, state of mind, and combat experience through a series of equations and algorithms.
In case you’re open-minded enough to befriend a digital human model, Santos has a Facebook page (you can add him as a friend at www.facebook.com/people/Santos-Version One/1777013781). Don’t be o ended if it takes him a while to respond. He may be in the middle of performing a very crucial task for a client.
Kenneth Wong is a freelance writer who has covered the digital video, computer gaming, and CAD industries. He can be reached at Kennethwongsf@earthlink.net.