Description:
TECHNOLOGY AREA(S): Air Platform, Battlespace, Human Systems
OBJECTIVE: Develop a methodology and tool that results in a capability to investigate the training effectiveness, comparable utility, and return on investment of an augmented reality solution for applied aviation training tasks.
DESCRIPTION: Augmented reality (AR) has been on the cusp of ushering in a training paradigm shift for over a decade by allowing overlays of a digital world on real platforms. Although the Navy and industry counterparts have been exploring the value of using AR technology in training, its perceived utility remains modest and has yet to make a substantial and sustained impact in the training domain. Additionally, few rigorous measurements of effectiveness have been conducted of AR itself, as well as comparing it to other related training technologies (e.g., tablets or game-based virtual training); this is likely due to the lack of methods and tools that support a quantitative comparison of these AR technology solutions with other training mediums. Yet, as technology improves, AR remains a promising training capability as it enables embedded train as you fight training by providing a means to fuse natural cues from physical surroundings in an organic setting with virtual or synthetic components. The Navy is seeking a quantitative analysis tool grounded in a methodology that supports comparison of AR and alternative solutions for a representative training environment. The resulting tool should include development of generalizable, best-of-breed methodology that will allow researchers to quantify the effectiveness of modern AR training and how AR training performance compares to related technologies. This effort focuses on delivering a rigorous measurement of effectiveness of AR and ability to calculate return on investment or design solution tradeoffs of comparative technologies. Additionally, considerations for how to conduct these analyses using early proposed training system designs or early prototypes through modeling or other means is desirable. Comparative training technologies include using a virtual reality solution (interacting with a simulated plane in a virtual environment), and a tablet-based training application. This technology comparison supports future investigations of AR usefulness as a training tool in other domains.
PHASE I: Demonstrate feasibility for the development of AR and alternative training solutions (e.g., handheld tablet training, game-engine based virtual environment training) for a representative training task (e.g., aviation preflight checklist training). Consider experimental design planning, including identification of applicable methods of assessing effectiveness, utility performance comparisons, and return on investment analyses for research in Phase II.
PHASE II: Develop and demonstrate functional prototypes of at least three alternative training solutions (e.g., AR, handheld tablet training, game-engine based virtual environment training) for a defined representative training task (e.g., training to support aviation preflight checklist training). Execute an experimental plan based on the designed methodology to compare the AR solution with the VR and tablet application alternative training technology choices. Based on the method, outcome and lessons learned of this analysis, develop a decision support tool that provides outputs including training effectiveness, task performance results, training utility (e.g., benefits and limitations of each solution), and return on investment calculations. Where feasible, the resulting tool should support automated capture and analysis of pertinent parameters.
PHASE III: Based on the Phase II results, refine, as needed, the methodology and tool(s) developed to meet training requirements for a wider variety of aircraft preflight checklists and/or similar scenarios to support transition and commercialization of the product. Investigate the potential of expanding the decision-support tool to more complex training environments and developing the paper-based decision-support tool into an automated, online support tool for researchers that will guide the design, development and test of future AR/VR and mixed reality immersive training solutions. Private Sector Commercial Potential: The advancement of AR technologies in recent years continues to strengthen interest in applying the technology in a variety of domains. The results of this effort can be far reaching, and would provide guidance and best practices for determining when and how to use AR training solutions. Specifically, the commercial aviation, military, medical, and educational domains stand to benefit the most. Many of these already use AR or VR for training and could use the results to modify or optimize their current or future training needs.
REFERENCES:
1. Azuma, R. T. (1997). A Survey of Augmented Reality. Presence: Teleoperators and Virtual Environments, 6(4), 355-385. Available at http://dl.acm.org/citation.cfm?id=J628&picked=prox&CFID=831999154&CFTOKEN=46426398.
2. Billinghurst, M., Clark, A., & Lee, G. (2015). A Survey of Augmented Reality. Foundations and Trends in Human-Computer Interaction, 8(2-3), 73-272. Available at http://www.slideshare.net/marknb00/a-survey-of-augmented-reality.
3. Hoff, W. A., Nguyen, K., & Lyon, T. (1996, October). Computer-vision-based Registration Techniques for Augmented Reality. Photonics East '96 (pp. 538-548). International Society for Optics and Photonics. Abstract available at http://inside.mines.edu/~whoff/publications/1996/spie1996.pdf.
4. Livingston, M. A., Rosenblum, L. J., Brown, D. G., Schmidt, G. S., Julier, S. J., Baillot, Y., Swan, J. Edward, Ai, Zhuming, & Maassel, P. (2011). Military Applications of Augmented Reality. Handbook of Augmented Reality (pp. 671-706). Springer New York. Available for purchase at http://link.springer.com/chapter/10.1007%2F978-1-4614-0064-6_31.
5. Livingston, M. A., & Ai, Z. (2008, September). The Effect of Registration Error on Tracking Distant Augmented Objects. Proceedings of the 7th IEEE/ACM International Symposium on Mixed and Augmented Reality (pp. 77-86). IEEE Computer Society. Available for purchase at https://www.computer.org/csdl/proceedings/ismar/2008/2840/00/index.html.
6. MacIntyre, B., Coelho, E. M., & Julier, S. J. (2002). Estimating and Adapting to Registration Errors in Augmented Reality Systems. Virtual Reality, 2002. Proceedings. IEEE (pp. 73-80). Available at http://www.dtic.mil/get-tr-doc/pdf?AD=ADA500367.
7. Van Krevelen, D. W. F., & Poelman, R. (2010). A Survey of Augmented Reality Technologies, Applications and Limitations. International Journal of Virtual Reality, 9(2), 1-20. Available at http://kjcomps.6te.net/upload/paper1%20.pdf.
8. Connor, Monroe (editor). Fused Reality: Making the Imagined Seem Real. 29 September 2015. Available at http://www.nasa.gov/centers/armstrong/features/fused_reality.html.
9. Graham, Luke. Bored with Virtual Reality? Microsofts betting on ˜Mixed Reality Now. 1 June 2016. Available at http://www.cnbc.com/2016/06/01/bored-of-virtual-reality-microsofts-betting-on-mixed-reality-now.html.
10. Example Preflight Checklist: http://www.pilotfriend.com/training/flight_training/fxd_wing/preflight.htm.-
KEYWORDS: Augmented Reality; Virtual Reality; Mixed Reality; Mobile Device; Training; Measuring Effectiveness
