Recently, Lauren Jarmusz - a Doctor of Physical Therapy Student at Northeastern University graduating in May 2016 - and I interviewed Dr. Karim Abdel-Malek, who is nationally and internationally recognized in the areas of robotics and human simulation. Dr. Abdel-Malek is a professor of biomedical engineering at the University of Iowa, where he is the director of the Center for Computer Aided Design, a world-renowned research center with seven departments, including a national lab. Dr. Abdel-Malek has founded or co-founded three private companies in the simulation, robotics, and educational fields. Along with his team, he is the creator of Santos®, a human modeling and simulation environment that is used by military and commercial clients to assess human factors issues in the design of equipment, armament, vehicles, and other large equipment. We were very interested in learning more about how he created a virtual human avatar and what the use cases were. See our interview below:
Marquis Cabrera: What is VSR?
Dr. Karim Abdel-Malek: VSR is the acronym for the Virtual Soldier Research (VSR) program at the University of Iowa. Our research is aimed at creating interactive, intelligent, and predictive human models that operate in virtual, physics-based environments. The male model is called Santos® and the female model is Sophia.
The program software has been contracted with the US Marines, which was titled ETOWL (Enhanced Technologies for Optimization of Warfighter Load). ETOWL measures the stress placed on each avatar's joints as well as its balance, flexibility and center of gravity. The 3-D simulation program allows the user to individualize the body types using a "drag and drop feature". We basically created a real life human in a virtual environment. Using real human data from biomechanics and physiology processes, we are able to predict how a human would respond to certain stressors in an environment, such as high elevation or heavy weight.
Lauren Jarmusz, sDPT: Where did you get the idea for it from? Who has helped you to develop it?
Dr. Karim Abdel-Malek: The program was founded in 2003. My lab originally conducted traditional research developing mathematical formulations to predict robotic motion. My "Ah ha" moment occurred when I realized that my mathematical calculations could be directly connected to human movement using "Newton's laws and the equations of motion". I fleshed out my realization with continued research and eventually had the opportunity to present these formulations at a conference for digital human modeling. The concept was met with great success. Everyone at the conference, even non-like minded researchers, were interested in the concept. Six months after presenting at the conference, we received a significant contract from the US Army, at which time I established the VSR program, and co-direct it with Professor Jasbir Arora.
The company that commercializes the product out of this research is Santos Human, Inc. (licensed technology from the University of Iowa).
Marquis Cabrera: What pain point is VSR is solving? What is the primary goal?
Dr. Karim Abdel-Malek: The ultimate objective is to develop a complete human being inside the computer, one that walks, talks, and behaves like we do. The idea is to have Santos and Sophia test things for us that we are unable to test by ourselves, and to reduce cost and time. Consider for example the design of an army tank. It takes many years to accomplish the first prototype and it costs hundreds of millions (sometimes billions) to produce the first full prototype. Today, in engineering, we test every aspect of aerodynamics, stress, noise, vibration and much more inside a computer, in a virtual environment. This virtual prototyping has reduced cost and time. However, the need for human testing is still one of the only reasons to create physical prototypes. Therefore, if we are able to create a human that lives inside the computer, and can test these virtual prototypes, then these costs and long development times will significantly be reduced.
The US Military is using Santos to study lightening the load on its personnel, by optimizing load carriage versus agility and mobility. It is used by the Navy to design and test new armor configurations, and it is now poised to be used by the Military to mitigate injuries. The software is in use by the US Marines to test new equipment/gear. Every time new technology comes out, it must be tested, using hundreds of personnel, in varying locations, and under different environmental conditions. Rather than spending the time and cost, Santos® is able to run simulations that predict how a US Marine would handle this new piece of equipment. Ultimately this is hoped to prevent future injuries and reduce the time and financial cost of unnecessary field trials, saving billions of dollars.
Marquis Cabrera: Is the application of VSR primarily for military use? What do you see as the future of Santos?
Dr. Karim Abdel-Malek: No, the Santos software is being used by many Fortune 500 companies such as Ford and Milsco (Milsco designs and produces seats for the motorcycle, marine, construction equipment, and agricultural equipment). Using the software, designers now have the ability to test how humans will interact with their newly developed cars or motorcycles. The software has the ability to be used in almost any industry: fashion design, automobile manufacturing, athletics, ergonomics, etc.
In the future, my dream would be to use Santos in the professional athletics field using Predictive Dynamics to determine how to specifically train elite athletes to reach their maximal physical potential. Ideally, the software would be used to calculate the musculature, power, and technique required to carry out an athletic task, such as the long jump. With this knowledge, coaches would have the ability to better inform athletes on how/what to train to reach their maximal athletic potentials.
I can also see the Santos software used to develop child human simulation. There is currently no software that is able to produce a child virtual simulation, whether to test products that are still in the design process, or to assess situations for safety.
Marquis Cabrera: How has the government helped you move VSR forward? I saw that you received some funding from the DOD.
Dr. Karim Abdel-Malek: Funding for the Santos effort has come from many sources over the years. In some cases, we respond to a BAA or RFP requesting the technology, in other cases, companies have to come to us seeking help in solving a problem. Overall, 60% of our funding has been military based. (How much money has the military provided in funding? ---$28M)
Lauren Jarmusz, sDPT: I know you have so many varying applications of VSR - Whole Body Vibration, Immersive Virtual Reality, Musculoskeletal Model - but can you give me a use case on how VSR works? And, maybe explain what you think the coolest applications of VSR are?
Dr. Karim Abdel-Malek: For example, Ford Motor Company uses Santos® to conduct studies and experiment with virtual protypes of their new vehicles in the digital world. Santos tells Ford engineers whether performing tasks over months and years will cause back strain, for example, and they can make adjustments until they find the optimal way to get the job done.
The US Marines Expeditionary Rifle Squad (MERS) uses Santos to evaluate new gear that has been designed but not fielded. Santos virtually tests the new equipment, evaluates its new range of motion, stability, and overall task performance, thus enabling better and faster decision making, yielding reduction in cost and time. Funded by the Office of Naval Research, this tool has transitioned to the US Marines.
Politecnico de Milano in Milan, a premiere university in Italy has integrated Santos® into their design curriculum.
Lauren Jarmusz, sDPT: Although pithy, what is the future of VSR? I noticed on your website: "Santos includes a muscular system with the ability to predict muscle activation and muscle forces in real time." Does VSR have implications beyond just the military? Can it be applied to help improve functioning of an ordinary citizen? Could an osteopathic doctor - i.e. a Physical Therapist - use digital human modeling and simulations to improve the physical functioning of adults?
Dr. Karim Abdel-Malek: The answer is that Santos is getting closer to having a full biomechanical and physiological model underneath. For example, we have been involved with the Army's Center for the Intrepid in San Antonio to use Santos in the evaluation of prosthetic limbs as they are fitting injured warriors. Similarly, we are now getting more involved in athletics, with an effort to model and simulate strengthening and conditioning. At the moment, the software is too expensive for the typical consumer or Physical Therapy clinic to utilize. Consumer options such as the Microsoft Connect is much more affordable to the consumer and is able to provide simple biomechanical evaluations.
Lauren Jarmusz, sDPT: What advice would you give to someone looking to use digital human modeling to improve musculoskeletal health? Beyond just come work for you!
Dr. Karim Abdel-Malek: There is no application at the moment that uses a digital model to improve musculoskeletal health. However, we are on a path to creating one as a result of this new funding from the Office of Naval Research. The technology to use motion capture to track someone's gait for example, then feed this motion profile into Santos for analysis, then obtaining a set of measurements and recommendations (muscle weaknesses, asymmetry, etc.) is all possible.