Here at Ford we’ve been exploring Augmented Reality (AR) solutions for the future of engineering. A range of available technologies are being tested, with selected individual solutions being utilized for multiple task and use cases. AR offers an extensive array of possibilities and may well prove to increase both productivity and profitability in the future.
But, with AR still being a relatively new technology, challenges have arisen when carrying out certain tasks. For example, some AR standalone devices, such as the Microsoft HoloLens, contain the necessary components to provide an AR experience to the user, without requiring larger PCs to render the environment. These devices, however, may have limited computing power, which notably may result in minor hindrances. Nevertheless, there are skilled teams actively working to resolve these hardware limitations using cloud-based computing. We dig deeper into some of these issues later on.
Ford has carried out extensive research into various AR technologies and solutions. Let’s take a closer look at some of these as demonstrated through a variety of tasks and use cases.
We’ve utilized Microsoft 365 Dynamics guides to deliver augmented work instructions. The app has proven to be effective for displaying and processing full size models, whilst the available objects have also proven to be adequate in showing the required motions for a task. However, there has been some difficulty regarding the implementation of animations for tasks that require a component to move, which are being investigated.
Microsoft Remote Assist has been considered as a solution for remote technical assistance. HoloLens users can share a live point of view with technicians, showing both their holograms and the real world around them with the help of the device’s front camera. And, technicians are able to add virtual call-outs to the HoloLens which are mapped in a 3D space for the user. This allows the user to have instant visual feedback on their task to easily follow through with the technician’s instructions.
Here at Ford, we’ve explored the app, Spatial, for conducting virtual meetings. It has proven that it can effectively replicate the personal presence of traditional, in-person meetings. It also has the ability to present 3D models of objects, allowing for easy design reviews which enable an enhanced level of immersion when compared to traditional video conferences. In its current state, however, it must undergo significant refinement before we can expect mass adoption of presentation methods for PowerPoint, spreadsheets, etc.
AR Sketching for Rapid Prototyping
Some employees have explored the rapid sketching of simple models with AR. The app, Graffiti3D, allows users to quickly produce sketches and 3D models, sometimes faster than CAD and sketching by hand. This technique, though not ideal for dimensional accuracy, provides benefits by easily being able to produce straight lines, adjust colors, and instantly transition to a 3D space without needing to remodel in design software. There is also no limitation on the size of drawings in AR, as opposed to sketching on paper.
AR Cleaning Instructions
Ford has also generated work instructions for the cleaning of all extended reality devices, including both manual and automated processes. Automated methods include utilizing technologies such as UV light and contained ozone sanitization. Additionally, separation devices for extended reality headsets have been included in the cleaning instruction documentation to reduce the transmission of bacteria from the head directly to the extended reality headset.
However, these technologies and solutions have not come without their challenges.
Whilst Microsoft Dynamics 365 Guides appears to be one of most practical apps currently available for the HoloLens, it’s very demanding on the device. This can be noticed as soon as a work instruction is being opened, where loading times can be quite lengthy. As nothing is displayed during the loading process, users can be led to believe that the app has crashed. In addition, the device can heat up after long periods of time, causing discomfort, visual lag, and may potentially result in unintentional user inputs. Another common issue is placing virtual markers such as hands, arrows, or circles next to other virtual models. As of now, it is difficult to re-select placed markers and typically will result in selecting the model instead.
Hand tracking has also been found to cause inconsistencies with some devices. Controls that require the pointer finger and thumb to interact (air tap/grabbing) are sometimes not detected at all or are falsely activated. This can cause frustration as it can be time consuming to figure out how to correctly orient and moves one’s finger, so that the detection is accurate. There appears to be a steep learning curve in order to reliably control the HoloLens using hand movements.
As demonstrated, even though AR still has a long way to go, the benefits are already being identified and explored. As the technology continues to advance, becomes more widely adopted over time and, importantly, more affordable, it will play more of a key role across the automotive industry and beyond in other industry sectors. Here at Ford, we’re committed to going further into this new reality and beyond.
This editorial has been developed as part of the AREA Thought Leaders Network content, in collaboration with selected AREA members.