OpenAI. (2024). A futuristic scene showcasing the power of Extended Reality (XR) in digital marketing. [AI-generated image]. DALL-E.

Blog written by Samantha Yueh, Keynote Speaker at the CDM Summit

In recent years, we are in a world where you can virtually try on clothes, explore real estate properties from your living room, or interact with digital products as if they were physically in front of you. This is the power of Extended Reality (XR), a technology that’s rapidly transforming how brands connect with their audiences in digital marketing. But it goes beyond just visuals—XR is about what we see, what we feel, what we hear, and even what we smell and taste. It’s a fully immersive sensory experience.

The field of XR—which includes Virtual Reality (VR), Augmented Reality (AR), and Mixed Reality (MR)—is advancing quickly. These technologies are not just trends; they are expected to bring significant efficiency and value to our daily lives. One of the most significant shifts we are witnessing is the move from 2D to 3D interfaces. This transition requires new design approaches and skills, pushing us to see, think, and imagine in three dimensions. By integrating virtual objects into real-world environments, XR creates immersive experiences that engage all our senses across industries like gaming, education, training, and luxury retail.

In this evolving landscape, User Experience (UX) design is more critical than ever. The challenge for designers is to create compelling, intuitive experiences within these immersive environments. How can we enhance these sensory experiences? What are the key elements to consider? The traditional 2D UX design mindset is no longer sufficient; we now need to think from multiple angles, engaging all the senses, to craft truly immersive experiences.

To better understand the different facets of XR, it is like an umbrella, under the umbrella covers VR, AR, and MR — can be visualized as different ways of interacting with this umbrella. 

XR as the Umbrella

OpenAI. (2024). An umbrella representing Extended Reality (XR) with Virtual Reality (VR), Augmented Reality (AR), and Mixed Reality (MR). [AI-generated image]. DALL-E.

XR serves as the overarching term that covers all immersive experiences. Imagine the umbrella itself representing XR, a protective canopy under which different realities coexist, offering varying levels of interaction with the digital and physical worlds.

VR Under the Umbrella

Virtual Reality (VR) is like stepping fully under the umbrella. In VR, you are entirely immersed in a new environment, cut off from the physical world around you. You can’t see what’s outside the umbrella—only the completely virtual world that surrounds you. This full immersion allows you to experience a new reality, but it also means that your sensory input is entirely controlled by the virtual environment.

AR Under the Umbrella

Augmented Reality (AR) is like standing under the umbrella but still being able to see the world around you. The umbrella adds a layer of digital information to your view of the real world. For example, imagine looking through the umbrella and seeing digital content, such as text, images, or animations, superimposed on the environment around you. You can still see the real world clearly, but with added digital elements that enhance your experience without blocking your view.

MR Under the Umbrella

Mixed Reality (MR) goes further, allowing for interaction between the digital elements and the real world. It’s like standing under the umbrella, seeing both the world around you and digital objects that appear to coexist with the physical environment. These digital objects are not just overlaid; they interact with the real world in meaningful ways. For instance, you might see a digital character that appears to stand on your real coffee table, and you can interact with it as though it were truly there. MR blends the virtual and physical worlds more seamlessly, allowing them to influence each other.

This umbrella analogy helps distinguish the different technologies within XR, clarifying how VR, AR, and MR each offer unique ways to experience and interact with both the digital and physical worlds. Understanding these differences can create effective, sensory-rich experiences in design and marketing.

The Sensory Dimensions of XR

XR is redefining how we interact with content by engaging all our senses far beyond what traditional and digital marketing has achieved. Where traditional marketing focused on visuals and text, and digital marketing added dynamic audio, XR immerses users in experiences that engage sight, touch, hearing, and more.

• Sight: In Virtual Reality (VR), users are transported into a fully immersive environment where detailed, 360-degree visuals surround them. Imagine exploring a virtual showroom where you can view a car from every angle, inside and out as if you were physically there.
• Touch:  Haptic feedback in XR, controller or smart gloves allows users to feel virtual objects, adding a tactile dimension to the experience. When you reach out to touch a virtual object, such as a piece of furniture, in a virtual home tour, haptic devices can simulate the texture and resistance, making it feel as if you’re truly interacting with the object.
• Hearing: Spatial audio in XR allows sounds to come from specific directions, enhancing the sense of presence. For instance, in a virtual concert, you can hear the music as if you were standing in front of the stage, with crowd noises coming from all around you, making the experience feel real.
• Smell: While still in the early stages, XR is beginning to incorporate olfactory elements. Imagine a virtual cooking class where, as you watch the chef prepare a dish, you can actually smell the aroma of the spices being used, adding a new layer of immersion to the experience.
• Taste: Although taste is the most challenging sense to simulate, there are experimental XR applications in development. Consider a virtual wine tasting experience where, as you interact with different wines in the virtual environment, you receive taste cues that mimic the flavors of the wines, enriching the sensory experience.

From a UX perspective, engaging multiple senses in this way doesn’t just make the experience more immersive—it makes it unforgettable. When users can see, touch, hear, smell, or even taste within a digital environment, the experience becomes more than just an interaction; it becomes a memory. This multi-sensory approach is important because it deepens user engagement, making the experience more emotional and impactful. As a result, these immersive experiences can significantly influence user behaviour and perception, leading to stronger brand connections and higher customer loyalty. 

Transitioning from 2D to 3D Interfaces

As we move further into the world of XR, one of the most significant changes is the transition from 2D to 3D interfaces. This shift is more than just a visual upgrade; it challenges the very foundations of traditional design practices and requires a complete rethinking of how we approach user experience (UX) and digital marketing.

For decades, digital marketing has largely been confined to 2D screens, where information is presented through text, images, and videos. However, XR technologies push us beyond these flat surfaces into fully three-dimensional spaces. In these environments, users are not just passive viewers; they are active participants who can move around, interact with objects, and experience depth in ways that 2D interfaces simply cannot offer. This transition to 3D is particularly impactful in the context of user experience, where the goal is to create immersive and inclusive environments that fully engage the user.

Consider a traditional museum, which might offer simple interactive experiences like collecting stamps at different exhibits. In contrast, with XR technologies like AR or MR, the same museum can offer a far more engaging and sensory-rich experience—one that doesn’t even require physical presence. Visitors can simply download an app and interact with one of the museum’s themed exhibits from anywhere in the world. Through this app, they can explore a virtual exhibit, interact with artifacts through AR, experience spatial audio, or even receive haptic feedback that simulates touching the objects. Imagine exploring the museum’s ancient Egypt exhibit from your home, where the app releases subtle scents associated with the artifacts or plays background sounds that immerse you in the ancient world.

This remote interaction not only sparks curiosity but also acts as a teaser, enticing users to visit the museum in person for a more extensive exploration. By offering a taste of what the museum has to offer, the app can drive real-world visits, encouraging users to experience the full range of sensory interactions that can only be truly appreciated on-site. Additionally, the museum can gather valuable data on how users interact with the virtual exhibit, allowing them to refine both their digital and physical offerings to better meet visitor interests and needs.

To navigate these challenges, designers in the digital marketing space must develop new skills that enable them to think, see, and imagine in three dimensions. This requires a shift in mindset—from designing flat screens to crafting environments that feel immersive and inclusive. Designers must consider how objects and interfaces appear from multiple angles, how users might interact with elements at different heights or depths, and how to create experiences that are both engaging and accessible to all users, regardless of their physical or cognitive abilities.

In the museum example, this approach could involve creating an AR experience that adapts to different user needs, such as offering audio descriptions for visually impaired users or providing interactive games that appeal to different age groups. The sensory engagement could be further enhanced by incorporating tactile elements, such as virtual buttons that simulate different textures or feedback when pressed or olfactory cues corresponding to specific exhibits. The ability to visualize and prototype in 3D is becoming increasingly important, as is an understanding of spatial relationships and how they affect user experience. Traditional design tools are evolving to meet these needs, with 3D modelling software and VR/AR prototyping tools becoming essential in the designer’s toolkit.

This focus on inclusive, sensory-rich design not only enhances the user experience but also broadens the appeal of the museum, attracting a more diverse audience and increasing visitor engagement. For digital marketers, this represents a valuable opportunity to create more personalized and effective campaigns that resonate with a wider range of users.

Interaction design in a 3D environment is fundamentally different from its 2D counterpart. In 2D, interactions are typically limited to clicks, taps, and swipes on a flat surface. In 3D, however, users can reach out and grab objects, move around within a space, and interact with elements that exist all around them. This requires designers to think about spatial awareness—how users will navigate through the environment and how to guide them intuitively from one interaction to the next. User orientation becomes crucial; without the clear boundaries provided by a screen, users need cues to help them understand where they are within a space and how to move through it.

In the museum example, these principles of interaction design are key to creating a truly immersive and sensory experience. By using AR or MR, the museum can offer interactive elements that guide visitors through the space in an intuitive way, perhaps through gamification elements that encourage exploration and discovery. For instance, rather than simply collecting stamps, visitors might use an AR app to solve puzzles or find hidden clues throughout the museum, with each interaction accompanied by sensory cues—such as vibrations when a clue is found or specific sounds that signal a discovery. This not only makes the experience more engaging but also encourages repeat visits, as users are drawn back by the promise of discovering something new and sensory-rich each time.

From a digital marketing perspective, these interactions provide valuable insights into user behaviour. By tracking how visitors interact with different exhibits and sensory elements, the museum can learn which features are most popular and use this information to tailor future marketing efforts. This data-driven approach allows for more targeted and effective marketing campaigns, driving higher engagement and, ultimately, greater business value.

The shift from 2D to 3D interfaces represents a major evolution in design, one that challenges us to rethink how we create and interact with digital environments. XR offers an unprecedented level of sensory engagement, transforming user experiences into something far more immersive, inclusive, and memorable. As digital marketers and UX designers, it’s essential to embrace these changes and develop new skills and approaches that can fully leverage the potential of XR. By doing so, we can create lasting, impactful experiences that resonate deeply with users and drive business success in ways that traditional methods never could.

Learn More from Samantha Yueh at the CDM Summit in Surrey

Samantha Yueh wrote this article. Samantha Yueh is a digital transformation expert specializing in XR (Extended Reality) technologies. Throughout her career, she has successfully managed large-scale projects and diverse teams, showcasing innovative product and project features through the strategic application of XR technologies. Samantha excels in tailoring solutions to meet clients’ unique needs across various industries, demonstrating her adaptability and deep understanding of emerging technologies.

Currently, Samantha is a Digital Transformation Consultant and a faculty member at University Canada West, where she teaches the Digital Transformation course in the MBA program. Her extensive expertise is further validated by several patents related to AR/MR technologies, underscoring her significant contributions to the field.

If you enjoyed reading this article, we invite you to join us at the CDM Summit, a marketing conference happening in Surrey on September 17-18. Samantha will be speaking on the Fundamentals of XR. Buy your tickets now and join the community of digital marketers passionate about taking their skills to the next level.

References

Marr, B. (2019, August 12). What is extended reality technology? A simple explanation for anyone. Forbes. https://www.forbes.com/sites/bernardmarr/2019/08/12/what-is-extended-reality-technology-a-simple-explanation-for-anyone/

OpenAI. (2024). An umbrella representing Extended Reality (XR) with Virtual Reality (VR), Augmented Reality (AR), and Mixed Reality (MR). [AI-generated image]. DALL-E.

OpenAI. (2024). A futuristic scene showcasing the power of Extended Reality (XR) in digital marketing. [AI-generated image]. DALL-E.

Shekaraubi, S. (2022, September 13). Navigating the transition from 2D to 3D UI/UX design in VR. Medium. https://medium.com/@ShahroozShekaraubi/navigating-the-transition-from-2d-to-3d-ui-ux-design-in-vr-771dc756ba3a

Visual Capitalist. (2022). Extended reality (XR): The complete guide. Visual Capitalist. https://www.visualcapitalist.com/extended-reality-xr/

Xu, X. (2022). Consumer consciousness in multisensory extended reality. ResearchGate. https://www.researchgate.net/publication/359648356_Consumer_Consciousness_in_Multisensory_Extended_Reality