You are here

Streaming Model for Field-of-Light Displays (SMFoLD)

Description:

TECHNOLOGY AREA(S): Human Systems

OBJECTIVE: Develop a model for the multimedia data stream required for next-generation field-of-light display (FoLD) systems to project full-parallax video-rate 3D images without eyewear. Demonstrate the model on a FoLD system in a command center environment.

DESCRIPTION: Collection, storage, transmission, and viewing of 3D data by a variety of DoD sensor systems has increased dramatically over the past 15 years and even more rapid growth is anticipated. The Air Force has identified a requirement for true 3D visualization systems to increase productivity of operators dealing with the 3D data deluge. The Air Force further requires the data be viewed without special eyewear on a new class of display, a so-called Field-of-Light Display (FoLD) visualization system.

A variety of prototype FoLD systems have been developed that each uses a unique, proprietary approach to transmit and visualize the same 3D data. The government (DARPA, IARPA, Air Force) has sponsored several efforts to foster the development of FoLD systems. Each effort has recreated the underlining software to ingest 3D content for delivery to their device. Lack of a common streaming media model has emerged as a barrier creating FoLD systems acceptable within a command center environment.

Government leadership is required. The focus of commercial standards bodies has been exclusively on the Stereo 3D (S3D) class of 3D display. The S3D class requires special eyewear and is, for a variety of reasons, not acceptable in a command center environment. S3D has caused eye fatigue and nausea in certain viewers due to a conflict in the accommodation and vergence cues it provides to the human visual system. The nausea can be reduced, but not eliminated, if the viewer is stationary and the content is tailored pixel by pixel (which is possible in movies over several months of post production but wholly impractical in a command center). Furthermore, S3D has limited value for parallax correct viewing since the perspectives are simulated from imagery that was captured from only one or two points of view (POV). These human interface limitations of S3D have prevented its adoption to address the 3D data deluge in Air Force command centers.

The emerging new FoLD class of 3D visualization system offers non-eyewear full parallax viewing and perspectively correct visualization for multiple persons. The FoLD class comprises several types including lenticular, volumetric, and holographic. Furthermore, many existing 3D capture methodologies based on Light Detection and Ranging (LiDAR) sensors, Synthetic Aperture Radar (SAR) sensors, or plenoptic cameras, capture a 3D environment that can be viewed correctly from many perspectives only on a FoLD visualization system.

Today the burden of integrating a FoLD system into an application space or environment is placed, over and over, on each software application developer. The emerging hardware technologies have yet to unite behind a common model for streaming a 3D scene description. Proprietary 3D display hardware and software formats limit the adoption and interchange of 3D visualization devices.

The next step in the evolution of 3D visualization is the creation a common streaming model for 3D data--including a scene description protocol and transmission format--that is display technology agnostic. The standard should define a streaming 3D scene that can be viewed on any 2D, S3D or FoLD visualization system and allow such flow and POV control as is required by the host application or content. Current and future display prototypes in any class (FoLD, S3D, and 2D) could then create an optimal visualization from the same streaming scene description.

PHASE I: Define display-technology agnostic, 3D streaming model for FoLD systems that is similar to existing 2D protocols. Establish definitions for streaming 3D content, audio content, compression, metadata, encryption, key frames, and error recovery. Integrate protocol and definitions into the model. Organize and conduct workshop open to all government and industry to publicize results.

PHASE II: Revise streaming model to address industry comments at the workshop and publish as a technical report to be entitled "Draft Data Streaming Model for Field-of-Light Display (FoLD) Visualization Systems." Brief the report at multiple scientific and engineering meetings, including SMPTE, IEEE, and SID. Conduct a second workshop and revise the technical report. Document performance tradeoff analysis of choices made in a final report. Develop a software tool to implement the model.

PHASE III DUAL USE APPLICATIONS: True 3D displays have multiple military and civilian applications including modeling, geospatial representations, design and exploration. Data interface standards will be required to expedite the optimization and commercialization of these display technologies.

REFERENCES:

    • (a) Klug M, Burnett T, Fancello A, Heath A, Gardner K,O’Connell S, Newswanger C. A Scalable, Collaborative, Interactive Light-field Display System. Society of Information Display Symposium Digest of Technical Papers 2013 pp. 412-415; (b) Zscape holographic motion displays, http://www.zebraimaging.com/products/motion-displays.

 

    • (a) V. Michael Bove, "Engineering for Live Holographic TV," SMPTE Motion Imaging Journal, pp. 56-60 (Nov/Dec 2011); (b) S. Jolly et al, Computational Architecture for Full-Color Holographic Displays Based on Anisotropic Leaky-Mode Modulators, MIT Media Lab, SPIE (2015).

 

    • Stephan Reichelt, Ralf Haussler, Gerald Futterer, and Norbert Leister, "Depth cues in human visual perception and their realization in 3D displays," Proc. SPIE 7690, 76900B (2010); http://dx.doi.org/10.1117/12.850094.

 

    • Levent Onural, Fahri Yaras, and Hoonjong Kang, Bilkent Univ. (Turkey), "Digital Holographic Three-Dimensional Video Displays", Proc. IEEE 99(4), pp 576-589 (2011). Http://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=5709964&isnumber=5733920.

 

  • (a) Pierre-Alexandre Blanche, "Toward the Ultimate 3D Display", SID Information Display 28 (2&3), pp. 32-36 (Feb/Mar 2012); (b) Sechrist, Steve, “TVs, 3-D, and Holograms at Display Week 2014,” Information Display 30 (5), 14-18 (2014).

KEYWORDS: streaming model, field of light display, FoLD, full parallax, accommodation-vergence congruence, air operations center, holography, hogel, voxel, depth planes

  • TPOC-1: Darrel Hopper
  • Phone: 937-255-8822
  • Email: darrel.hopper@us.af.mil
US Flag An Official Website of the United States Government