John Baker

Augmented Reality (AR) is rapidly maturing, and as this it develops, it will present the training community with unprecedented capabilities. However, there remain several key technical challenges that must be addressed in order to deliver credible and immersive experiences. At the Training & Simulation Industry Symposium 2019 and in subsequent public forums, the Simulation & Training Technology Center identified Dynamic Occlusion as a key technological challenge critical to the successful… Show more

Augmented Reality (AR) is rapidly maturing, and as this it develops, it will present the training community with unprecedented capabilities. However, there remain several key technical challenges that must be addressed in order to deliver credible and immersive experiences. At the Training & Simulation Industry Symposium 2019 and in subsequent public forums, the Simulation & Training Technology Center identified Dynamic Occlusion as a key technological challenge critical to the successful implementation of AR applications and specifically critical to AR training. Dynamic Occlusion is the ability of an AR system to realistically spatially integrate synthetic and real-world content. Dynamic occlusion allows moving real-world objects, such as people, to occlude virtual content in a credible and natural manner. For example, if a person walks in front of a synthetic box, the person should occlude the box and the box should not be visible “through” the person. This concept is intuitive to observers, but there currently does not exist a commonly understood metric for measuring dynamic occlusion performance of various AR systems. In this paper, we propose a metric for assessing Dynamic Occlusion and break Dynamic Occlusion down into constituent factors of false positive and false negative occlusion. We will discuss the relative merits and challenges of these differing types of occlusion and their ultimate effect upon user acceptance and suspension of disbelief in AR experiences. We will also discuss recommended methodologies to gather Dynamic Occlusion metrics over a variety of conditions. The intent of this work is to provide a common conceptual framework that can facilitate the establishment of requirements as well as objective comparisons of performance across various systems. This framework will allow solutions to be accurately compared as we collectively tackle this key technical challenge to the implementation of AR training systems. Show less

Within training, the Department of Defense (DoD) has a strong interest in augmented reality (AR) for its ability to combine live and virtual assets to reduce cost, increase safety, and to mitigate unavailability of needed live assets. Answering this is the commercial sector, which is rapidly advancing a host of capabilities to support AR, such as head mounted displays (HMDs). An important capability of AR is the realistic occlusion between live and virtual objects based… Show more

Within training, the Department of Defense (DoD) has a strong interest in augmented reality (AR) for its ability to combine live and virtual assets to reduce cost, increase safety, and to mitigate unavailability of needed live assets. Answering this is the commercial sector, which is rapidly advancing a host of capabilities to support AR, such as head mounted displays (HMDs). An important capability of AR is the realistic occlusion between live and virtual objects based on their respective depth in the augmented reality scene. Existing solutions use pre-scanned environmental depth information to provide this capability; however, this is only useful for objects that will never move or static objects. This does not address live objects that move or dynamic objects (e.g., live Soldiers). Dynamic objects must be constantly scanned using a depth camera(s) to provide the occlusion information for an AR-enabled system. Of the few commercial sector vendors that provide depth cameras on their HMD, most are lacking the sufficient depth range to adequately support occlusion from the HMD – anything beyond roughly two meters has greatly diminished fidelity. This paper describes a solution that will add a network of fixed infrastructure cameras with a centralized occlusion server to merge the depth images from various sources. This then creates depth images suitable for occlusion on the HMDs at any range with realistic fidelity. The paper will report the use of commercial off the shelf (COTS) computers and cameras that instrument an area such that it can be used for occlusion in a training system. The paper will speak to the performance of fusing data, given resolution and volume. Finally, this paper will show scenes where virtual objects are added to a real scene and how the performance of the occlusion system affects the quality of the visuals for the participants. Show less

Within training, the Department of Defense (DoD) has a strong interest in augmented reality (AR) for its ability to combine live and virtual assets to reduce cost, increase safety, and to mitigate unavailability of needed live assets. During the past two years, a rapid increase of interest in AR for consumer use has spawned a multitude of innovations for head mounted displays (HMD). Increased fields of view (FOV), tetherless… Show more

Within training, the Department of Defense (DoD) has a strong interest in augmented reality (AR) for its ability to combine live and virtual assets to reduce cost, increase safety, and to mitigate unavailability of needed live assets. During the past two years, a rapid increase of interest in AR for consumer use has spawned a multitude of innovations for head mounted displays (HMD). Increased fields of view (FOV), tetherless computing, integrated depth-sensing, external spatial audio, and simultaneous location and mapping (SLAM) are just a few features that have become a boon for military use, such as dismounted Soldier training. However, the usefulness of these features varies for the dismounted Soldier training use case. This paper examines features from nearly a dozen of today’s consumer AR HMDs and contrasts the tradeoffs required to reap their benefits. We evaluated these HMDs primarily against key tactics and skills required in ATP 3-21.8 ‘The Infantry Rifle Platoon and Squad’ doctrinal framework – specifically employing fires, offensive operations, defensive operations, and patrolling. Finally, this paper explores what features are missing or are suboptimal on these HMDs for dismounted Soldier training use. Show less

Author of two speculative fiction novels (think Groundhog Day meets serial killer Ted Bundy), which enjoyed status in the Amazon Top 25 Best Sellers for its genre for six weeks in 2014. I am beholden to a pseudonym, but for those of you who know me, it's pretty much an open secret. The third book will be published in 2024.