Digital humans with starring roles in feature films and video games might capture the most attention, but behind the scenes, software programs shaped like virtual people are hard at work in research institutions, industry, and, especially, in the armed forces. One such digital human, Santos, the first-born avatar in Iowa University’s Virtual Soldier Research (VSR) program, recently reported for duty in a new type of battleground: the US automobile industry. Santos has taken a factory job at Ford Motor Company.
Private Santos
Previously, the US Army Natick Soldier Research, Development, and Engineering Center (NSRDEC) outside Boston enlisted Santos to simulate the effectiveness of body armor, one of several ways the virtual soldier has served the Department of Defense. Recently, for example, Santos began evaluating equipment loads for the Office of Naval Research.
“There are lots of software programs that can do biomechanics, but Santos is the only one that can predict motion,” says Karim Abdel-Malek, cofounder and CTO for SantosHuman, Inc., the company that is moving Santos out of the university’s lab and into the market. “The difference is wide and huge. Santos reacts to things using predictive dynamics. Everyone else records motion, brings it to software, and analyzes that motion. We tell Santos, ‘You go do it,’ and he may or may not. We can load him with gear, put a helmet on him, tell him a joint is injured, and then tell him to walk, run, dive, and take a prone position. He tells us if it would be possible to do that and how he would execute it.”
Groups of virtual muscles inside Santos actuate joints using calculations based in physics and provide data on joint-based strength and fatigue.
In 2004, Abdel-Malek, then a robotics professor specializing in motion prediction and human simulations at the University of Iowa’s Center for Computer-Aided Design, began a research program for the Defense Department. The goal was to study human dynamics in the virtual world. The result was the VSR program, which begat Santos.
In 2007, a collaboration between General Motors, Chrysler, and Ford, known as USCar, supplied more funding, with each manufacturer’s $500,000 contribution over the past three years adding to the approximately $10 million supplied by the federal government. “We knew that the area of dynamic evaluation would benefit us all,” says Allison Stephens, technical specialist in assembly ergonomics at Ford. “This has been an exciting research project that’s finally coming into implementation.”
The following year, in July 2008, a group of faculty members and researchers at VSR founded SantosHuman: Malek; Timothy Marler, chief research officer; Jasbir Arora, chief science officer; and Steven Beck, chief operations officer. They hired Jay A. Johnson as CEO in December.
“My first assignment was to complete the licensing that allowed the University of Iowa research foundation to transfer the software and science to a for-profit company,” Johnson says. By April 2009, the new company was ready for business and a continuing collaboration with Iowa University researchers.
The researchers modeled Santos in Autodesk’s Maya; they have a close relationship with the Maya developers in Montreal. “One of [the Maya team’s] main projects is to create detailed, high-fidelity human models, so we’ve been helping each other get there,” Malek says. “We make Santos pretty in Maya and then bring him into the real-time simulation environment.”
Inside the Maya skin, which a user can stretch to create a tall, small, fat, or skinny Santos, are groups of virtual muscles that actuate joints using calculations based in physics. The muscles have an activation value, and the joints have a torque value. “We can evaluate joint-based strength and fatigue to calculate the strength and fatigue of the whole body in real time,” says Marler. “Conceptually, we start at the joints, and everything grows from there. Then, it’s about what Santos can do. He can predict a posture, a motion, a muscle wrapping, and direction, and provide output about what he can or can’t do. Sometimes, just a pass/fail.”
Ford’s specialists in assembly ergonomics will feed dynamic strength curves based on data collected at the University of Iowa into Santos so that he can analyze stress and fatigue caused by dynamic work.
Blue-collar Santos
So, how will all that help Ford build cars? For the past three years, as part of the USCar project, Stephens has worked with Laura Frey on the VSR team to collect human strength data. “She puts students in a chair and has them move an arm quickly to change weight and get maximum forces at different velocities,” Stephens explains. “She does this for legs, hips, and back, isolating joints. Sixty subjects per joint. She gets high velocity and low velocity to collect the maximum.” Soon, Stephens will feed the resulting dynamic strength curves to Santos.
Currently, the manufacturing and engineering department uses Jack and Jill, digital humans from Siemens PLM Software and developed at the University of Pennsylvania, to crawl under cars, under car hoods, and sit in passenger seats.
Santos determines how much load this digital soldier can bear and still assume this position. Digital bones and joints move his skin. He does not have soft tissue at his joints, nor does the software simulate fat and tissue for his whole body yet, though finite-element analysis helps researchers evaluate potential injury from blasts to limbs and internal organs.
“Jack still has a home with us, and so does Jill, but the area they can’t analyze is dynamic work,” says Stephens. “A lot of people are familiar with ergonomics in terms of setting up driver interfaces and layout of controls in a car. The focus for assembly ergonomics is setting up the assembly plant, and currently our digital models are limited to static evaluation. They give us a snapshot in time. If the question is, ‘Can I reach something?,’ Jack and Jill can move and answer ‘Yep, I can,’ or ‘No, I can’t.’ But, if I want someone to lift a large piece of carpet, manipulate it, and move it into a car, I’ll call on Santos. The exciting thing about Santos is that he will have data on movement.”
Stephens hopes that by the end of the year, Santos, now in beta, will be doing the dynamic evaluations she has in mind at Ford. In addition to giving the VSR team information on what data they needed, Stephens has helped the team understand exactly what they want Santos to do.
“We were able to say, ‘This is what we want to see, this is where we want instant feedback, and this is how we want to manipulate something,” Stephens says. “Santos will do lifting and push-pull evaluations because those are things I use regularly. Part of our job is to make sure we set up the factory line correctly and minimize the risk of injury. We also know if we can guarantee that someone can do a task car after car without inducing fatigue; it has a huge impact on quality.”
For example, the ergonomics specialists at Ford use Jack and Jill now to calculate fatigue factors—to know, for example, how many hoses someone could install and how many electrical connectors the person could snap together during a workday. “I have some data about that,” Stephens says, “but I don’t have the data across the board in every posture and every action. I’m using a broad brush. Santos will give me specific data for fatigue on the muscles in that simulation.”
Stephens also expects Santos to help them understand how a worker can most efficiently and safely install a carpet, a heater, or a seat into a vehicle. “I have the carpet in CAD, and all the parts of the car,” Stephens says. “So we can decide where to put Santos’s hands. Which is better for the spine—to cradle the carpet or put one hand above and below? Should Santos roll the carpet into a smaller size? We can evaluate all those impacts of moving the carpet into the vehicle.”
Similarly, Stephens plans to have Santos help her minimize the stress workers face when unloading tires from a transport truck. “We don’t know how to evaluate that now,” she says. “The tires come 700 to 750 in the back of the truck, laced in there. It’s incredibly hot in the summer, and the tires sweat and get condensation on them. We have portable air conditioners, but it’s a nasty job. I can tell Santos it’s 90 degrees in the truck. Santos will calculate the energy it takes to lift the tires out of the truck and, given the temperature, recommend how long the person can work. We’ll be able to do a work/rest schedule for this high-demanding job.”
Santos’s first day on the job at Ford also signals the first of what SantosHuman hopes will be many jobs in many other industries. “We have multiple licensing scenarios,” Johnson says. “Right now, we have maybe 100 people in some large corporations actively engaged in creating simulations and testing and deploying the software internally. Currently, our tool is using sophisticated science to solve difficult problems by expert users. But, if you tweak that science, millions of people could use Santos. That’s the next part.”
Someday, we might spot Santos testing running shoes, evaluating swimming suits for an Olympic team, assisting physical therapists with rehabilitation plans, helping stunt doubles prepare for shots in a feature film, and perhaps even, taking a starring role in a feature film. n
Barbara Robertson is an award-winning writer and a contributing editor for Computer Graphics World. She can be reached at BarbaraRR@comcast.net