Providence Delivers Immersive Learning with AR

Providence uses video technologies that enable engaging augmented reality training for caregivers, leaders, and patients.

At a glance:

  • Since 1856, Providence has aimed to improve healthcare access and transform the experience by putting compassion at the forefront of their services.

  • To enable engaging and immersive training, Providence worked with Evercoast to create an in-house studio called the UPmmersiv Studio that uses Intel® RealSense™ cameras and the Evercoast volumetric video capture platform to record actors from every angle and output a 3D model ready for use in AR training experiences.

author-image

By

Since 1856, Providence has aimed to improve healthcare access and transform the experience by putting compassion at the forefront of their services. With the goal of enriching the healthcare experience, the US West Coast healthcare provider wanted to transform their staff training capabilities. Led by Darci Hall, chief talent officer, and Johnny Hamilton, senior design and innovation consultant, the Providence learning and development team envisioned a working environment where caregivers and leadership could learn, practice, and apply new skills from anywhere, at any time.

Challenge: Building a Better Way to Upskill Their Workforce

As they embarked on their learning transformation journey, Providence saw three main learning and development (L&D) challenges in the healthcare industry. First, while previous training programs provided effective learning opportunities, they didn’t leave participants feeling confident enough to put their new skills into practice. Second, as demonstrated by the COVID-19 pandemic, it can be difficult to virtually train multiple employees at once when in-person training is not possible. Third, employees don’t always retain or apply the information that has been relayed to them because training experiences can be unengaging. The need for upskilling has been noted by 87 percent of CEOs1 and as a top priority by 59 percent of L&D professionals.2

As leaders in learning innovation, Hall and Hamilton believed that immersive learning powered by augmented reality and virtual reality (AR and VR) had the potential to solve their challenges in upskilling their workforce. They also believed that it could be the key to unlocking the engagement needed to increase comprehension, proficiency, and retention at Providence. However, they would need to explore new strategies, learning designs, and technologies to architect a solution.

To guide their development strategy and maximize results, the team leveraged the Push-Anchor-Pull approach, for which they had recently won a Gold Brandon Hall learning innovation award for its use in compliance education. In the Push-Anchor-Pull approach, employees are “pushed” to learn a set of new skills or knowledge, they “anchor” those skills through practice, and then they “pull” resources when they need to actively apply those skills in the flow of work. This learning design allows employees to stay focused and engaged while maximizing comprehension, retention, and performance.

Additionally, the Providence AR training solution needed to meet certain requirements. The solution had to be deployable, scalable, and virtually accessible so that training could happen anywhere. It also needed to be as immersive and engaging as doing exercises with a facilitator or coach—if not more so—and it had to enable employees to effectively retain their knowledge and put their skills into practice in the workplace.

To deliver AR-powered training experiences, Providence had to find a way to easily capture virtual recordings of people (virtual holograms) that could be used in AR. Providence would also have to identify or create an application to distribute these AR experiences to their workforce so the training could be experienced anywhere.

Another requirement was to align the solution to an existing business need. With Providence as a mission-driven organization, their imperative is to foster a culture of unity and reconciliation. With diversity, equity, and inclusion (DEI) as a strategic driver, the team decided to explore implicit bias for their first immersive learning experience and chose the topic of responding to microaggressions in the workplace.

When the COVID-19 pandemic hit, Providence knew that immersive, engaging, and remote training would be needed more than ever. They soon found an opportunity to obtain funding and financial support for an AR-based training solution from Intel’s initiative to combat the effects of COVID-19. As part of the initiative, Intel introduced Providence to Evercoast, a 3D volumetric video platform that could provide the capturing, processing, and streaming technology Providence needed to turn their vision into a reality.

To kick-start their transformation, the team decided to use an existing 90-minute virtual instructor-led training (VILT) on implicit bias as the foundation for their first AR experience. They sought to transform a five-minute video from the VILT into an engaging, immersive learning tool.

Intel’s Pandemic Response Initiative

Since April 2020, Intel has donated a total of USD 100 million to combat the myriad effects of the COVID-19 pandemic. Part of Intel’s goal was to fund projects that would accelerate access to technology for both patients and caregivers.

“Rather than needing to prove our idea would have its expected impacts, Intel removed that barrier and gave us the funding we needed. It reduced our risk and greatly accelerated our existing strategy and speed-to-business impact.”—Johnny Hamilton, senior design and innovation consultant, Providence

The Solution: A State-of-the-Art Studio for Volumetric Video Capture

To enable the engaging and immersive training they envisioned, the Providence team worked with Evercoast to create an in-house studio that could record high-resolution volumetric video content. Volumetric video is a video-capture technique that captures three-dimensional footage.

The studio, which Providence calls the UPmmersiv Studio, uses depth-sensing cameras and the Evercoast volumetric video capture platform to record actors from every angle and output a 3D model ready for use in AR training experiences. UPmmersiv is a term coined by Hamilton that means to “upskill using immersive learning.”

In the first phase of production, the volumetric video was captured. This process starts with an actor standing in the middle of the studio and performing their lines. Video of the actor is captured on an Intel-powered workstation in the studio, which is connected to more than 20 Intel® RealSense™ cameras that record the actor in 360 degrees. The captured footage is consolidated by the Evercoast platform running on the workstation.

Leveraging machine learning and advanced computer graphics technologies, Evercoast uses a completely
automated software platform to combine every angle to form a complete 3D model of the performer in full motion. Users see a live preview of the volumetric video as they record the training content. Once the content has been recorded, it is uploaded to Evercoast’s cloud platform. This platform runs on an Intel® Xeon® processor-based Amazon Web Services (AWS) cloud instance with built-in AI acceleration for final rendering, which takes about 30 to 40 minutes.

It took less than three days and three team members to build and configure Providence’s UPmmersiv Studio on one of Providence’s Portland, Oregon campuses. However, once it was built, Providence was unable to use the studio because of COVID-19 restrictions. As a result, the Providence team decided to use Evercoast’s volumetric video studio in Brooklyn, New York, to record their content. Once the studio was completed, the next step was to deliver the virtual holograms via an easy-to-use application that would enable caregivers to interact with their training in an engaging, immersive way.

Putting Virtual Holograms (Volumetric Videos) to Use

The second phase of production was to deploy the volumetric videos captured in the studio via an easy-to-use training application that would enable caregivers as well as leaders to interact with the training in an engaging, immersive way. The Providence team worked with the Gronstedt Group, an award-winning leader in the AR/VR development space, to create their first UPmmersiv application for smartphones. This technology was groundbreaking, and the Gronstedt Group worked with engineers at Evercoast to build and test the app’s infrastructure in the Unity engine.

In this app, called Responding to Microaggressions in the Workplace, learners use their smartphone camera to place a virtual hologram of a person in their room. Next, they choose a scenario and are confronted by a virtual hologram of a person who demonstrates a microaggression. Learners choose how to respond, and then the scene continues. When the scene finishes, a video of a facilitator provides feedback on their response. Learners can repeat the scene and choose a different response or select another scene.

In Providence’s UPmmersiv Studio, an Intel-powered Evercoast workstation equipped with Intel® Xeon® processors is connected to 20 RGB-D cameras. The depth-sensing cameras provide the ability to only accept video from within the capture ring, which is about seven and a half feet in diameter. Each camera captures a different angle of the actor.

Results: Unlocking the Power of AR-Enabled Training

Providence incorporated the microaggressions pilot into their DEI Resources learning pathway. They also recently launched a new course as a part of their broader learning transformation journey. Providence received over a hundred responses from a series of nine one-hour VILT microaggressions conversations classes.

In this data, caregivers reported that the immersive learning experience the Providence team developed gave them great insights about situational awareness and a deeper understanding of microaggressions. They also believed that the AR-enabled training had the potential to educate caregivers and leadership in any learning environment. In general, caregivers rated the UPmmersiv AR app better than other modalities, including videos, role playing, infographics, and articles.

Most of all, the caregivers reported that the app helped foster curiosity and excitement, boost their understanding, and enhance their ability to remember and apply the content in their work. In addition, the facilitators noted that the virtual holograms created an immersive, shared experience, which served as a catalyst to more engaging and thoughtful conversations. This demonstrates that immersive learning is an effective modality to rapidly upskill any workforce.

The initial success of this pilot has created substantial buzz within and beyond Providence regarding the possibilities and impacts of immersive learning. They shared their progress at Training Industry Conference & Expo events in fall 2021 and spring and fall 2022 and at the Intel Vision conference in spring 2022, and were awarded a 2022 Gold Brandon Hall HCM Excellence Award in the “Best Use of Games or Simulation for Learning” category.

With help from Intel, Evercoast, and the Gronstedt Group, Providence successfully turned their vision into reality by creating a deployable, scalable, and virtually accessible way to train employees in a more engaging way. Evercoast and Intel’s involvement with Providence’s learning transformation journey enabled Providence to build their state-of-the-art UPmmersiv Studio where they can easily create volumetric videos for immersive learning experiences powered by AR and VR.

Paired with the current smartphone app and with new apps in development, this studio positions Providence to create a wide variety of engaging learning and performance support experiences in the future. Providence sees their thought leadership in immersive learning as a way to attract and retain talent in the healthcare industry, which is a top concern of most leaders. By creating a space where innovation meets compassion, they are truly enabling health for a better world.

The Next Step for Providence

In 2022, Providence received a second grant from Intel, and they partnered with Microsoft to expand their immersive training from their workforce to their patients. They will be using their UPmmersiv Studio to reimagine rehabilitation training for orthopedic patients. The objective is to provide patients who experienced total joint replacement (knee, shoulder, or hip) with an UPmmersiv app on a Microsoft Surface tablet that will help them learn how to perform recovery exercises. Patients will also be able to access the app on their own mobile devices at home during their outpatient recovery. By automating patient learning with immersive experiences, Providence anticipates lowering costs, improving caregiver experiences, and boosting patient health outcomes and experiences in the future.

About Evercoast

Evercoast enables the creation of real 3D digital humans in a world that’s transforming from 2D to 3D. The company provides a cost-effective, scalable, and deployable solution for volumetric video capture and streaming in a cloud-based software platform that can easily create 3D humans in VR, AR, and 3D video. Evercoast firmly believes in the growth of the virtual training market for midsize businesses, which is exactly what Providence was looking for.

Download the PDF ›