A comprehensive Guide to Augmented Reality
22 minutes | 05 Feb 2019
Augmented Reality doesn’t just have the potential to change how we work in manufacturing, but how we think about the manufacturing process too.
To give you a thorough knowledge of what Augmented Reality is, how it works and the potential impact that it can have on your business and the industry as a whole, we’ve put together this comprehensive guide.
- Introduction to Augmented Reality
- Augmented reality in manufacturing
- Augmented reality in the supply chain
- AR: How to train your workforce
- Design Engineers and AR
- AR: The Future
- Augmented reality case studies
Introduction to Augmented Reality
From the high-tech Augmented Reality (AR) interactivity of movies such as Iron Man and Minority Report to the mobile games revolution of Pokémon Go and 2018’s Harry Potter: Hogwarts Mystery gaming sensation, it’s safe to say that AR has touched most people’s lives.
AR vs VR
AR differs from Virtual Reality (VR) in one crucial way and the clue’s in the title: VR is a fully computer-generated environment, within which the user is immersed. AR, on the other hand, overlays virtual information on an existing, real-life environment, providing a composite, part digital, part real view. Often represented as futuristic, forms of the technology have been around as far back as the 1990s. Think of the heads-up displays in many fighter aircraft showing data about speed, direction and attitude, all within a pilot’s field of view.
Today’s AR devices use complex sensors, optics and other electronic elements mounted in a variety of materials. These include glass, plastics, ceramics and textiles, in the form of glasses and visors, headsets, projectors and consoles. Each adds sensory information to a user’s environment. Text, graphics, images and audio can all be superimposed, allowing users to see and interact with actual or virtual objects in unconventional ways.
Over the past 10 years or so, a variety of companies have developed devices like these, such as MIT Media Lab’s Sixth-sense, combining the use of a camera, small projector, smartphone and mirror to project images that can be manipulated by the user. Then there’s Google Glass with its projection of overlaid images, videos and sounds on to a user’s lens screen.
"This technology looks set to transform the way we work as well as play"
As the potential for AR expands into business and industrial applications in areas such as design and development maintenance and repair, monitoring and logistics the technology looks set to transform the way we work as well as play.
Pick-list order fulfilment
Medical equipment manufacturer GE Healthcare is using Up-skill’s industrial Augmented Reality application platform Skylight on smart glasses to empower warehouse workers in their MRI factory to complete pick-list order fulfilment. Skylight replaces the use of printed order sheets to locate items with smart glasses that display the items to be picked, including a description and quantity needed. Initial results show a 46% improvement on the pick-and-pack activity.
Future Prediction for AR
According to the latest forecast from the International Data Corporation (IDC) Worldwide Quarterly Augmented and Virtual Reality Headset Tracker, AR head-mounted displays will see increasing growth over the next five years, as
standalone and tethered devices grow to account for more than 97% of the market by 2022.
Augmented reality in manufacturing
Manufacturing is the latest sector to tap into the practical business uses offered by Augmented Reality (AR). Companies of all sizes have much to gain from AR technologies that have the functionality to enhance operational efficiency in various processes such as:
- Maintenance and repair
- Quality control
- Product design and development
- Rapid visualisation
Such applications have the potential to fast-track time to market and create new ways to collaborate and facilitate the supply chain.
Emerging technologies such as AR can be used as a tool to transform traditional manufacturing methods. Product development teams can use the rapid visualisation capabilities of AR to refine and optimise designs at an earlier stage in the process. Object concepts and alternatives can be overlaid, reviewed, adjusted and modified quickly, incorporating elements such as ergonomics, access, and look and feel. Digital models can be tested, analysed and simulated, delivering a rapid iterative design process, significantly improving productivity.
"Augmented Reality also represents new ways of collaborating across geographic boundaries in real time"
No geographic boundaries
Augmented Reality also represents new ways of collaborating across geographic boundaries in real time via heads-up, hands-free video calling, pulling together vital cross-functional team input, as well as the participation of potential clients. Being able to connect and communicate, and fine-tune at speed translates into an efficient and streamlined product development process.
AR in the warehouse
Back in the warehouse, AR looks set to transform the software used to manage the supply chain with the ability to help lorry drivers, warehouse workers and management keep track of products from the second they leave the factory until the moment they arrive at the customer’s door. Factory managers can perform advanced analysis using sensors on packaging, increasing or decreasing production or redistributing items based on their findings.
Industry sectors investing in AR
Industry sectors taking the lead so far in their uptake of Augmented Reality include aerospace, automotive, energy, defence and medical. Global aerospace manufacturing company Polamer Precision has used AR to map out its plant layout with a 3D model to position workstations and tooling, ensuring forklifts and other equipment have room to operate safely and efficiently. Polamer plans to test overlaying work instructions specific to a role or work centre on HoloLens to simplify worker tasks and ensure more efficient on-the-job training.
Likewise, design teams at early adopter Ford Motor Company have been leveraging AR’s capabilities using Microsoft’s HoloLens AR headsets to view potential car designs overlaid on a physical prototype in real scale, giving them room to be more creative, improve decision making and collaborate better.
According to PricewaterhouseCoopers, AR, similar to VR, is being used across the US manufacturing sector as an advanced manufacturing tool, in a similar way to robotics, 3D printing and the Internet of Things. But the most popular application of AR is in product design and development. Across 98 respondents, the use cases were revealed as follows:
- 38.8% product design and development
- 17.3% virtual assembly/improved process design
- 27.6% safety and manufacturing skills training
- 19.4% maintenance, repair or operation of equipment
- 19.4% data and information access
- 19.4% remote collaboration
- 13.3% customer engagement and communications
- 7.1% supply chain collaboration/communications
- 18.4% other
Augmented reality in supply chain
As the Industry 4.0 revolution gathers pace, Augmented Reality (AR) is helping turn manufacturing enterprises into smart factories. From product design and development to maintenance and repair, AR is enhancing operational efficiency and improving productivity across organisations. As its impact takes effect, the technology looks set to transform the software used to manage supply chains.
Augmented Reality overlays virtual information on to a real-life environment, providing a composite, part digital, part real view. In logistics, the technology can support lorry drivers and warehouse workers, and helps management keep track of products from the second they leave the factory until the moment they arrive with customers.
Logistics and AR success
According to a report by ABI Research, logistics was one of the first markets to adopt and deploy Augmented Reality, accounting for 24% of global smart glasses shipments in 2017. What’s more, shipments of smart glasses for logistics usage are expected to generate revenue from $52.9 million in 2017, growing to $4.4 billion by 2022.
Sensors on packaging enable managers to carry out advanced analysis of the flow of a company’s products. This means that, in real time, a factory can increase or decrease production based on that day’s sales across multiple retail locations. Digital technology allows data scientists to generate new insights from the products in the digital supply chain, regardless of where the shipment is located: in the back of a delivery lorry or on a warehouse shelf. AR can be used to identify, monitor and track products throughout the supply chain, making it effective in many applications, including the following.
- Pick-and-pack services: Augmented Reality is being used in warehouses to locate products more efficiently and pack them into delivery boxes. AR glasses can paint an imaginary line on the warehouse floor to simplify searching and training. They can also provide graphic overlays of packages on shelves, minimising the time needed to find and identify items. At GE Healthcare – a global provider of medical imaging, monitoring, bio-manufacturing, and cell and gene therapy technologies – a warehouse worker receiving a new pick-list order through AR completed the task 46% faster than when using the standard paper-based method.
- Collaborative robotics: designed to physically interact and collaborate, safely, with humans in a shared workspace, the cobot is lightweight, compact and simple to operate. With the implementation of Industry 4.0 technologies, the development and usage of cobots is evolving fast to deploy across a broad range of facilities in manufacturing, warehousing and logistics. Workers sitting at their desks can wear AR glasses, allowing them to see what a cobot in the warehouse sees, charting their journey through the warehouse and using their strength to lift and move heavy products.
- Maintenance: fixing a problem before it happens is the most cost-effective way to maintain factory-floor machines. AR glasses with enhanced image recognition techniques enable fault detection and correction in the heavy machines used for sorting and repackaging goods. Users can also inspect machines and identify any mechanical or structural faults in them. This enables timely maintenance of systems and avoids major malfunctions or delays in the supply chain.
- Last-mile delivery: the last mile of delivery to customers is the most expensive. AR can save costs by cutting the time spent on last-mile delivery by nearly half. According to logistics company DHL, drivers spend 40% to 60% of their day searching inside their own truck for the correct boxes to deliver. Augmented Reality can be used to identify, tag, sequence and locate every parcel.
While still in the early stages, AR applications have shown there is great potential to transform and improve productivity in the supply chain.
AR: How to train your workforce
What will the next manufacturing workforce look like when Augmented Reality (AR) intersects with traditional production processes; and are they willing and able to take up the AR challenge?
AR and the skills gap
There’s a growing workforce gap developing as large numbers of baby boomers retire and emerging technologies in automation combine to disrupt the manufacturing sector. New workers coming into manufacturing companies must adapt quickly to new working environments where novel visualisation techniques, collaborative robots and machine-learning applications are helping to realise the smart factory.
Advanced manufacturers want operators and technicians with both traditional and new skill sets. The challenge is how these prospective workers can bridge the gap and how manufacturing companies can train them, and future-proof their workforce and their organisations.
Augmented Reality allows you to overlay information in the real world. It can provide real-time, on-site, step-by-step visual guidance on tasks such as product assembly, machine operation and warehouse picking. For example, if an employee is learning how to carry out repairs, they will need to point a device at the machine they’re working on and overlay it with video or other virtual information. The AR device provides them with additional information in the form of a composite view in order to interact with the machine in the real world.
Using AR for employee training
One of the advantages of learning and using AR in this setting is that workers get to experience just-in-time training on the job, right where they are ready to perform the task. There is no need to take employees out of their work environment and put them in the classroom; and it’s particularly suited to training that’s specific to the nature of their job.
Some well-known companies have successfully implemented AR learning programmes to train new manufacturing personnel. Cisco has built an AR experience allowing technicians to see how to install components in an AR overlay on top of a physical device, increasing installation efficiency by 30% and first-time accuracy by 90% in the process. Elevator company thyssenkrupp has implemented Microsoft HoloLens to train 24,000 service engineers. When off-site, service technicians learn how to visualise and identify problems with elevators, and have remote, hands-free access to technical and expert information when on-site.
So how should manufacturers approach their AR training programme to boost engagement and minimise time off the shop-floor?
- Find a solid business use case for AR.
- Consider launching with a pilot programme.
- Define your learning goals.
- Engage your IT department.
- Identify the AR tools to get started with.
- Measure your results and track your outcomes carefully.
- Prepare your workforce for the future; consider how you’ll support them as new technologies emerge.
So how is AR being received by manufacturing workforces? According to Capgemini Research Institute’s 2018 Augmented and Virtual Reality Survey, 57% of early adopters have experienced more than 10% efficiency increase with AR/VR compared to only 23% of the rest of the companies surveyed. When looking at the different strategies employed for expanding their AR/VR initiatives, the study found that early adopters were investing heavily in agile, in-house teams of experts to train their workforce, with 92% choosing to up skill employees through specialised in-house training, as opposed to 75% of the rest. Likewise, 91% of early adopters were hiring people with AR/VR expertise, as opposed to 75% of the rest.
Design Engineers and AR
Augmented Reality (AR) creates value. It improves performance across the value chain, in assembly, maintenance and repair, quality control, training, monitoring, logistics and more, using virtual information as a tool to support real-life activities. But in its take-up in smart manufacturing, as part of the evolution of Industry 4.0, there’s one area in particular that holds great promise for the application of AR: product design.
AR devices use complex optics, sensors and other electronic elements mounted in materials including glass, textiles, plastics and ceramics within headsets, glasses and visors, consoles and projectors in order to add virtual information to a user’s environment. AR can superimpose text, graphics, images and audio on to a view of the real world, allowing users to see and interact with actual or virtual objects in front of their eyes. For example, an engineer or architect can view
a design with a 3D CAD projector and have the ability to modify features with hand gestures.
Visualisation is a core activity at the very heart of the design process and in the early stages of development, product concepts have been conventionally formed by sketching and CAD modelling. However, the limits of its functionality mean designers’ interaction with those models has been through 2D windows on computer screens, making it harder to conceptualise designs fully, leaving them disconnected from real-life contexts of space and scale.
Add AR and you can bring the model to life, pulling it out of the screen and dropping it into the real world at the right scale, where you can walk around and through it, and gain a more authentic feel for that object’s true form, proportion and relationship to the environment.
Bosch Rexroth, a global provider of power units and controls used in manufacturing, uses AR-enhanced visualisation to demonstrate the design and functionality of its smart, connected CytroPac hydraulic power unit. The AR application lets customers see 3D representations of the unit’s internal pump and cooling options in multiple configurations.
With emerging technologies, such as additive manufacturing, a component can go through any number of changes in the time it would take for one iteration using conventional development methods. AR lets design engineers superimpose CAD models on to physical prototypes to compare them. Volkswagen uses this technique to check alignment in digital design reviews, improving the accuracy and speed of the quality assurance process. Previously, design engineers would have painstakingly compared 2D drawings with prototypes.
"AR enables product development teams to refine and optimise designs at an earlier stage in the process."
AR enables product development teams to refine and optimise designs at an earlier stage in the process. Concepts and alternatives can be overlaid, reviewed, adjusted and modified, quickly incorporating elements such as access and ergonomics. Digital models can also be tested, analysed and simulated, resulting in a rapid iterative design process.
AR also represents new ways of collaborating across geographic boundaries in real time by means of heads-up, hands-free video calling, linking teams together with the participation of potential clients. Microsoft’s HoloLens allows teams of designers, engineers and maintenance experts to work together while geographically separated, communicating and fixing problems while sharing graphic material. Being able to modify and iterate at speed translates into a streamlined product development process.
AR: The Future
Once the stuff of science fiction, Augmented Reality (AR) has moved out of the realms of conjecture and now firmly inhabits the real world, with examples of its implementation across manufacturing, from assembly, maintenance and repair to product design, monitoring and logistics.
However, there’s still some way to go before widespread adoption, with plenty of companies on the periphery of AR, waiting for the right time to dive in. Over the coming years, key indicators that will prompt widespread AR take-up include:
- Computational power: the development of dedicated processing units that can increase computing power and speed, while reducing power consumption, will allow AR hardware to do more.
- Battery life: many guidance and collaboration applications remain impractical until batteries in AR headsets can last an entire work shift
- Field of view: increasing the field of view of devices toward that of a human’s at 180 degrees will expand a worker’s observable environment, enabling more immersive experiences
- Eye tracking: advances in new eye-tracking techniques may reduce power consumption, increasing responsiveness
- Price point: price influences consumer adoption and a decrease may drive enterprise adoption
- Comfort/weight: as lighter and sleeker devices become available, adoption will grow.
So, in industry, where are we likely to see the greatest advancements in the next decade?
- Holograms: physicists at the Australian National University (ANU) have invented a tiny device that uses a new nanomaterial to create 3D projections. Millions of tiny silicon pillars, each up to 500 times thinner than a human hair, act as pixel projectors to create high-quality, light-based 3D images with future applications in AR devices, displays and ultrathin, lightweight optical devices for cameras and satellites.
- Digital twins will help companies see problems and opportunities using detailed virtual 3D models and AR. It will revolutionise the way companies understand customer needs and fix problems remotely, improving products and identifying new business models.
- Contextual information: by leveraging the vast network of Internet-connected mobile devices in the world today, mobile AR systems are set to transform user experience in real-time. In any situation or environment, such systems will be able to incorporate the user’s contextual, sensor-generated cues, without the need for a manual search. For example, on-site maintenance and repair teams will be able to overlay machine data and instructions onto their work space, to the virtual training of a new workforce, the take-up of AR and its software ecosystems hold great promise for the manufacturing sector.
- Augmented Reality technologies will give us access to just-in-time information, anytime, anywhere, by using a headset or other immersive device, enhancing education, training and travel experiences.
- AR will transform management in every industry; think fighter pilot-style displays for supervisors on the shop floor, able to read performance data and pinpoint trouble-spots in real time.
- Real-time instructions via AR apps to recognise objects and overlay text and/or video instructions.
- Immersive 3D for communication: AR-enabled 3D avatars that mimic expressions to create a personal and immersive experience.
- Future jobs for the AR-assisted workforce: new learning opportunities for Augmented Reality programming and interface development.
- Revolutionising product design with AR-enabled 3D modelling software to extend the design surface from the PC to the real world. Designers will fine-tune their designs in real time, while also engaging stakeholders for instant feedback.
- Just-in-time training: AR-enabled devices will bring step-by-step instructions to users, right in front of their eyes.
- Making dangerous jobs safer: exposing workers to potentially hazardous environments from the safety of the classroom demonstrates the value of AR-enhanced training; providing real-time virtual assistance in real-world scenarios.
Augmented reality case studies
Emerging technologies, such as Augmented Reality, are being used to transform traditional manufacturing methods. Companies of all sizes are beginning to tap into the practical business uses offered by the technology in order to enhance operational efficiency in processes such as assembly, maintenance and repair, product design, training and logistics.
An anonymous 2018 survey by the High Value Manufacturing Catapult Visualisation and VR Forum with contributions from the Institution of Engineering and Technology, explored how immersive technologies are being used across many different industrial sectors. These included aerospace, oil and gas, and construction.
The majority of respondents (53%) had been using immersive technologies for between one and three years with only 5% having been utilising the technology for more than 10 years. The second largest group were organisations who had been applying it for less than one year.
The survey also asked where in the product life cycle they were applying the technologies. Design was the most enthusiastic at 23%, with assembly and manufacturing each taking a 13% share.
AR and 3D modelling
Global aerospace manufacturing company Polamer Precision has used Augmented Reality to map out its plant layout with a 3D model to position workstations and tooling, ensuring forklifts and other equipment have room to operate safely and efficiently. Specialising in the manufacture of complex engine components, the company also plans to use HoloLens to test overlaying work instructions specific to a role or work centre. This will simplify worker tasks and ensure more efficient on-the-job training. With Microsoft HoloLens expected to decrease in price once it hits mass availability, Polamer sees the opportunity to use manufacturing data in a new way to manage production.
Aerospace manufacturer Boeing has deployed Google Glass to the wire assembly process of its 787–8 freighters, leveraging significant gains from Augmented Reality. Before it harnessed the technology, Boeing’s workers needed to refer, continuously, to their laptops to ensure that numerous wires were correctly concentrated. The process caused fatigue in employees and took up a significant amount of time. With AR headsets, the Boeing workers now have identical information right before their eyes, making the company’s wire construction faster and more comfortable. They can also deliver voice commands and ask a co-worker to join the headset’s video stream to support multiple businesses. The result: Augmented Reality manufacturing has decreased Boeing’s wire assembly method time by 25% and lowered errors to nearly 0%. There’s also an added bonus in the fact that the solution has significantly increased employee performance and, consequently, worker retention.
Design teams at early AR adopter Ford Motor Company have been harnessing AR’s capabilities using Microsoft’s HoloLens AR headsets to overhaul their design process. Using the device, they can view potential car designs overlaid on a physical prototype in real scale, giving them room to be more creative, improve decision making and collaborate better. The technology allows workers to leave comments and feedback in AR for other team members to access. Given that AR allows teams to work remotely, the solution provides an extremely robust and skilled combination. It has provided significant benefits to Ford, allowing the company’s designers to be more productive, develop decision making, cooperate better and work faster.
Reading instructions can hinder the efficiency of machine installation and repairs. Technicians need to be able to understand the steps and tools required in order to fine-tune and fix machinery across the factory floor. AR can overlay virtual tutorials on to equipment to provide technicians with quick and visual demonstrations. American technology company Cisco wanted to reduce customer support cases and increase the efficiency of installing its devices. Using AR creator Blippbuilder, it built an AR experience to let technicians launch virtual demonstrations straight from their devices. Workers could then see how to install different machine parts in an AR overlay on top of the physical device. The experience removed the need to read manuals, which increased installation efficiency by 30% and first-time accuracy by 90%.
Medical equipment manufacturer GE Healthcare is using Upskill’s industrial Augmented Reality application platform Skylight on smart glasses to empower warehouse workers in their MRI factory to complete pick-list order fulfilment. The initial result revealed an immediate 46% improvement upon first-time use of smart glasses. In the past, warehouse workers flipped through printed orders to locate parts and walked, sometimes across the entire warehouse, to a computer workstation as they found depleted stock locations to search for alternatives. Now workers use the Skylight platform, where they can see the items to be picked displayed on their smart glasses, showing details including the item description, quantity needed, and primary and alternate locations.