Meeting the Expectations of Generation AR

Transcript

(This transcript has been edited for brevity.)

DAVID CARTY: What better way to find your voice than on the stage? OK, well, maybe a big musical number isn’t for everyone. But in his most formative years, Tom Emrich found himself in musical theater.

TOM EMRICH: I love my musical theater background. I was very interested in musical theater coming out of high school. I found that I had a voice joining the choir and was part of some of the school plays, and I think what I really loved about musical theater, first and foremost, was the community. I found a lot of community in musical theater and also a community and outside of the community that really enjoyed my talent. And so the combination gave me the confidence to go to Sheridan College, which was, or is, a college in Canada, which is where I’m from. And I studied musical theater. I had a very short career of one or two roles, including playing Dickon in The Secret Garden, and this was back in the late ’90s. So not trying to date myself, but– yeah, I’ve really enjoyed musical theater also because you got to play characters. I mean, it was very formative. You put yourself in a very vulnerable spot in theater. That’s the other thing that I think I really liked. It really pushed me to be visible– that’s the first thing– and not be somebody that’s just in the corner trying to be quiet and not make a lot of noise and attention. It’s the quite opposite. You need to make a lot of noise and need to have a lot of attention on you. It really also helps you be OK with rejection, especially during the audition process. 9 times out of 10, you’re going to be told you’re not good for the part or you just nearly made it to get to the part. And so there’s a lot of lessons that I take from that time in musical theater.

CARTY: His favorite musicals include a soft spot for some ’90s nostalgia, and it continues to be a passion that not only fulfills him in his personal life, but also informs his professional work.

EMRICH: I’m a big fan. I was super excited to the fact that Wicked, the movie, became such a popular phenomenon. I knew it would. It’s still one of those musicals that really takes my breath away. But I have my favorites. There’s a lot of new musicals that have come out that, speaking of the ’90s, are really reimagined ’90s storylines, like Back to the Future, and Beetlejuice, which has been kind of interesting. Yeah, I love watching what’s happening in the musical theater space. And also, going to the theater, I’m always on the lookout for how theater productions are using technology to really engage the audience. And I’ve seen a number of productions where they’ve used projection mapping that have been really compelling ways to introduce augmented reality to bring the stage production to a whole new level. Yeah, I’ve really enjoyed musical theater also because you got to play characters and put yourself into the role of characters. And fast forward, I’m still kind of working in that space from a different angle within augmented reality and virtual reality, which is all about world building, all about enabling you to step inside other characters’ shoes, in virtual reality in particular. And so there’s a lot of parallels between what you do in theater in terms of setting up the stage, setting up the space, the costumes, the characterization, and these immersive technologies, which is what I’ve kind of dedicated my life to for the past 15 years.

CARTY: That musical theater background brings a perspective to product development that you really can’t teach. And in a roundabout way, it allows Tomto get even closer to his products’ users.

EMRICH: You as the actor, you as the performer, you’re just one piece of the overall puzzle when it comes to creating a play or a musical, and I think the appreciation to the parts is something that is really important when it comes to creating immersive experiences. Within immersive experiences, you need to think about the same thing. Where is this taking place? What is the mood? What is the lighting that needs to be put into place? What are the spatial logic that I need to ensure that this works the way that it’s meant to work? Who are the characters? And what role does the user play? That’s the other thing. And what expectations should you have of the user? And then what are the variables that you need to consider along the way? A lot of my time working in immersive technologies has been in product management. Product management– like, key to success within product management is to be obsessed with your customer, your user. And this is something that I think humanities has really come into play. And thinking about the character that I am going to be on stage is, in very similar ways, what I need to do as a product manager, as I put together my user persona and my user stories. And so there’s definitely a lot of parallels between what you do in the humanities and what really bodes well for success within the business side of technology or industry at large.

CARTY: This is the Ready, Test, Go podcast, brought to you by Applause. I’m David Carty. Today’s guest is thespian and AR pioneer Tom Emrich. With over 15 years of experience as a journalist, investor, and product leader, Tom is widely recognized as an expert in the fields of AR, virtual reality, and wearable technology. Tom is also the author of the book The Next Dimension, a definitive guide to leveraging augmented reality for business growth in the era of spatial computing, which is available now. AR and VR experiences are hardly a new notion or technology, but the demand for these experiences, combined with emerging technology, is opening up new avenues for innovation-forward brands. But how can they learn from the past mistakes of other brands to deliver immersive experiences that really move the needle? The wizard and I got into that.

Tom, first and foremost, congratulations on the book. Let’s start by explaining the term “generation AR,” which you use in the book. A Snap Survey found that 92% of Gen Z consumers want to use AR. Another survey found that 60% of Gen Z consumers said the use of AR made the marketing experience more personal. And over half said that they would pay more attention to marketing with AR. And you mentioned all of this in the book. Now, this is not a marketing podcast, but that certainly points to a market segment that is intrigued by the opportunities presented by AR experiences. So how should and how will we see brands adapt to their digital strategies to serve that market?

EMRICH: Yeah. I mean, generation AR definitely speaks to a generation that craves interactions, engagement, and experiences. And this really speaks to this new era that we’re in, the experience age , where both Gen Z and millennials are looking to spend money more on experiences and things that create memories than they are tangible apparel and items for them to store. And what’s great is that immersive technologies, augmented reality, or AR, virtual reality, or VR, these immersive technologies are experience mediums. They ask consumers and users to be very active in the content experience. And this is a step change from the content that brands have been creating for marketing up until now. For the most part, when you create a marketing campaign, you engage your consumers with text, with images, GIFs, video, audio. And for the most part, this is a relatively passive experience where your consumer and user is asked to consume that content. There may be some cases where you’re engaging them with quizzes or other more interactive experiences. But when we think about augmented reality in particular, it really asks for the consumer to be put into the driver’s seat of the experience. They’re not meant to just sit on the couch for many cases, but instead asked to get up and walk around and engage with digital content as if it was within the physical space. And so that’s the first way in which I think we’re really going to see digital strategies change, is that really thinking about experience and making engagement the hero metric within digital strategies, knowing that augmented reality and virtual reality asks for users to be engaged and active and not passive. The second thing that I think we’re going to see with digital strategies is this new P in marketing. So we have the five P’s in marketing, and presence is going to be a new P. And what I mean by that is that the user or consumer is now going to be present within the digital experience. And the digital experience either lives in the environment in which they’re in– their living room, outside– or they are in the experience as the character or trying on the product, on their face, on their feet, for example, on their hands. And so this really makes the experience much more personal and contextual and meaningful and in this way really puts the consumer right in the center of the experience. In fact, without them and the environment, there is no experience because they are essential ingredients within that digital AR marketing campaign. Both presence and being active within the experience really help to collapse the traditional linear marketing funnel. And so instead of being pulled through steps and stages of a marketing funnel, for the most part, we either find that the consumer is pushing themselves through and charting their own course through the marketing funnel, so changing the course of that funnel, or, even better, collapsing the marketing funnel together and experiencing a lot of the stages all at once, again, because of how engaged they are and how personal and meaningful augmented reality is.

CARTY: Right. And if there’s a low-hanging fruit for AR and spatial computing, customer journeys are one of those big areas. If you’re envisioning the retail customer journey of tomorrow, what does that look like, and how can innovation-forward brands lead the way there?

EMRICH: Yeah. I mean, retail is one of those sectors where augmented reality is quite active. And I think we can think about three things when we think about retail. We can first talk about how augmented reality can bring a physicality to e-commerce. The second is how augmented reality can bring a digital experience to the physical retail space and digitize brick and mortar. And then the third is a brand-new opportunity of just allowing for consumers to buy digital-only goods. And so when we think about e-commerce, one of the great things about e-commerce is that it’s really convenient. You can do it from your home. But it lacks the tangibility that you get when you walk into a store. You’re not able to try out the product. You’re not able to see the makeup on your face. You’re not able to see the shoes on your feet. You’re not able to see the furniture in your space. And this is something that you can do with augmented reality, virtual try-on and virtual try-out. And what’s really great about this is that it allows for the consumer to really have increased confidence in their buying decision, because they’re able to see their furniture in the space. Therefore, they can walk around the furniture from different angles. They understand the size of it. But even better, something that you can’t do even with brick and mortar is see that couch alongside the other furniture that you have in your space. So this not only increases the chance of conversion, but it also reduces returns. And there’s a really great use case that I have in the book from Avon that allowed for 400 of their makeup products to be used for virtual try-on using a technology from Perfect Corp, and they cited on a case study on the Perfect Corp website that they increase conversion by over 300% in using augmented reality. So the numbers are really showing great promise there. On the brick and mortar side, the one thing that you lack on brick and mortar is not the tangibility. It’s a physical space. But you lack the digital insights that you get from e-commerce. And this is something that augmented reality can really bring to the table. In addition, augmented reality can bring a digital component to the physical store and can really extend inventory, such as the case of augmented reality mirrors. So augmented reality mirrors are like big screens with a camera that have computer vision capabilities. Your consumer can walk up, and then they’re able to try on clothing that isn’t on the shelf. Or maybe it’s on the shelf, but they want to try on different colors or styles. And so this is a really great way to allow for an e-commerce-like experience to be within the store. Or you can use these same AR mirrors in the window to attract foot traffic. And we are seeing a lot of cases of that. And then the last thing I want to say on this point is that augmented reality is about bringing digital objects, digital experience into the physical world. And we’re starting to see the early signs of virtual-only sales, meaning you go and you buy a digital shoe that only exists in the digital world. And Gucci’s done this with their neon shoes that they sold for $13. And these shoes, you could see an augmented reality on your feet using your smartphone, for example. But you can also use the digital versions of these shoes within virtual reality worlds like, VRChat or virtual worlds like Roblox. And so I say this to the audience because I think this is a brand-new opportunity that is bubbling up, and so it’s something to keep your eye on and definitely quite exciting and only something that we’re going to be able to do within this era of spatial computing.

CARTY: And probably, as the hardware continues to evolve, you’ll be able to have more opportunities there with the digital-only types of products. So we’ve established there’s interest, there’s opportunity, and there are clearly some brands that are having some great results with AR and spatial computing, but it’s also an area where some brands have struggled to connect with their customers over the years, despite some really intriguing possibilities. So what are some of the biggest digital quality challenges that companies face when implementing AR, and how can they overcome them?

EMRICH: Yeah, that’s a really good question. I think one of the big things is device fragmentation. And so augmented reality can be experienced on a smartphone. And smartphones– there are 8 billion people in the world, and a large number of them have smartphones in their hands or in their pockets. And so they represent a large opportunity for reach and scale for any business. And so definitely this is the area or the device to focus on if you’re looking to tap into that reach. But even within the smartphone space, there are a number of operating systems, different generations of phones, as well as different connectivity options, wireless or otherwise. And so there’s a lot of things to consider within the device space just with the smartphones. And then, if you start to layer in the fact that augmented reality can also be utilized on headsets, for example, like the Meta Quest 3 or the Apple Vision Pro, here typically called mixed reality, or MR, that also causes some complexity when you’re thinking about your overall rollout. And I think, with devices, the most important thing to think about is who your user is and really narrow in, first, who is your user and then get an understanding of what devices do they have available to them and where they are, what connectivity they have access to, and really make sure you incorporate that as part of your planning. The second thing to consider is the environment. So this is one of those step change considerations for immersive content. When you create a video and you put a video out there or a podcast out there, you don’t really need to think about essentially where the user is. Maybe you need to think about the fact that they have access to watch the video or it’s quiet enough to hear the podcast. But here, it’s much different. The environment, including the people, the things, the place, there are essential ingredients and variables that change within the augmented reality and the virtual realities content. And so you need to think about things like– and account for things like, what is the lighting? How much room does the individual have to move around? You need to account for varying skin tones and different hairstyles, if it’s a face effect, for example. So there’s a number of things– again, going back to that same thing with device fragmentation. Who is your user, and where are they ideally going to be, or where you’re going to optimize for them to be able to experience this? And then the last thing is just the fact that consumers have high expectations for any digital experience, including emerging technology experiences. And they have competing content that is trying to get their attention, not just within the AR/VR space, but within legacy media. And so you have to really ensure that your performance is there, and that the experience can be easily accessed quickly, and that the onboarding is not confusing, because those first few seconds, if you run into any of these issues, you run the risk of losing that user. And the chance of getting them back is going to be much harder. So these are just a few things to consider, I think, when you’re putting together your planning for augmented reality.

CARTY: Right. And I want to follow up on that because there’s really no overstating the sprawl of devices, of experiences, of network connectivity, right? OSs, all of this, especially at a global scale. What works for one launch might not work for another. Is that an area where brands have failed in the past? Are there horror stories there in terms of their ability to launch effectively to their customer bases?

EMRICH: I would say that it’s something that is an important consideration in your rollout. And in fact, I think it’s OK to ensure that you make decisions that allow for the users that you want to access and make it clear that if users that are, for example, not on Wi-Fi or on 3G and not 5G can either not access it, unfortunately, or that they’re getting a downgraded experience or that there runs the risk that this might be a slower experience and just setting that expectation. And so I’ve seen that happened in the past with brand rollouts. And I think setting the expectation is one way to mitigate these risks. And I think users appreciate that you’ve thought through what an ideal experience is. And putting that messaging up front when someone goes to access it could be a really great way to ensure that you’re setting your augmented reality experience up for success.

CARTY: And for you, what are the hallmarks of successful AR or MR experiences, and how can brands do a better job of meeting those benchmarks?

EMRICH: Yeah. Good question. So the first is performance, making sure that your experience is stable across various devices, and also that they can be accessed easily. So thinking, again, about your user and where that user is going to be and what device do they have is going to be key here, as well as testing. And so testing is going to be really important from a performance standpoint. Testing on the various devices that you’re optimizing for but also testing on the devices that you’re not optimizing for, if you have that time, could really help you understand what tweaks you need to make. And often you can rely on platforms to help with simulation, which is really key here, to allow, for example, for you to test in certain environments or on certain devices or within certain browsers, for example, if you’re using web-based augmented reality. So I think performance is definitely key here when it comes to a hallmark experience. But then moving on from the technology, I think the next is really value. And so augmented reality, that experience needs to deliver value for the user. That’s the exchange that you’re giving in terms of the time that they’re donating. And so you need to think about, what is going to benefit your user? Why should they be engaging with this experience? And making that clear up front but also ensuring that that sings through the content experience itself. You can help refine that by phased releases, for example, doing alpha or beta releases, followed up by surveys, to help you ensure that you’re really nailing that when you go out for a full rollout. And then the last is goals. And this is more for the company itself. But ensuring that your augmented reality experience has goals and that you’re able to measure those goals is really key. There’s a wealth of analytic platforms that can help gather that data for you. And so, here, companies need to make sure that you implement the analytics and set up the right experiments and then perform experimentation. So pilots really help here. And then listening and learning from those pilots to refine the experience in order to set yourself up for better success, again, for that full rollout.

CARTY: This touches a bit on my next question, and you just mentioned the word “access.” There’s also the challenge of making an experience accessible and relevant across different customer segments. From a localization and an accessibility standpoint, what should brands be keeping in mind to succeed with AR experiences? Because those are nuanced and ethically important areas.

EMRICH: 100%, 100%. So I think the first thing is really designing for inclusivity. And so this really thinks about all people. And so thinking about the person as a person is really key. This includes ensuring that the content can be accessed by people with disabilities, by making it available with page readers, for example, and also making sure that the tech is accessible. And so thinking about the different devices and trying to widen, as much as possible, the different devices and the different connectivity options that are available within your demographic are going to be key here. And so this could be, for example, perhaps you’re building something for a headset and wanting to make use of mixed reality headsets, which really are going to allow for you to show your content with a much wider field of view, more robust sensors, really deliver on the overall promise of augmented reality, but knowing that maybe not all of your demographic yet has access to a mixed reality headset. And so thinking about a companion experience on a smartphone that more of your demographic will be able to access is also within this umbrella of inclusivity. The second you mentioned is localization. So, of course, localization here, thinking about language, detecting where folks are, and allowing for different language to be an option within the experience. But it’s also thinking about, again, the device capabilities, the devices that are used within different localized areas, and also the culture of that area, as well, especially if you’re thinking about storytelling and really being privy to how that story is going to resonate with different people within different cultures is really important. And then the last thing, I think, to really think about– and maybe this lends itself well to the first point, which is designing for inclusivity– is that headsets and phones have different inputs and interactions. And so thinking about ensuring that your content can be engaged with using a variety of inputs and a variety of interactions, whether it be voice, whether it be eye tracking mixed with hand tracking, or relegating to tapping on the screen and swiping, whatever it might be, you can ensure that those variations can be accessed. Using controllers versus hands, for example, is a good example in headsets. And so just running through all those scenarios and just taking that extra step to ensure that those options are available can really ensure that more people can enjoy your content.

CARTY: In the book, you share lessons from some pioneers in AR, including a few brands that you’ve worked with. What are some lessons our audience can take away from some of the pioneers in AR? Any examples come to mind there?

EMRICH: Yes. I have a big one. I did not work with them, but I can’t stop talking about what Gucci did in the Apple Vision Pro. If you haven’t had a chance to check out their documentary, Who is Sabato De Sarno?, I highly recommend it. And I’ll tell you why, as also helping brands understand some of the principles that might help them get their feet wet, especially within mixed reality headsets, which is a large opportunity. The reason why I love what Gucci did is because they leverage legacy media and then added a couple of really powerful 3D moments in combination with that legacy media within the spatial computer, Apple Vision Pro, to really justify why it should exist within spatial computing. And it really took the experience to the next level. So what I mean by legacy media is that it was a video. So they brought a video in. It was a really great documentary, awesome music, as you can expect from this brand. And what they did is they added three different types of 3D experiences. The first was they flew in 3D assets at particular points of the video timed with the video. So as an example, when the video showed De Sarno at the airport, then a 3D large plane flew over the video that was floating in my space as I was wearing the Apple Vision Pro. Something just really simple, but it took the experience to a brand-new height and something that you wouldn’t be able to do just watching the video on YouTube or on your monitor. The second thing that they did is they used immersive environments. And so when, for example, you were on the runway in Italy, your world disappeared, and you felt like you were watching the video on the runway. So that was a really compelling way to bring you into the video versus the first example, which is bringing things from the video out. And then the last is that, at certain points, since this was from Gucci, they did pause the video and let you interact with 3D digital twins of the product so that you got really up close with watches and other apparel from Gucci. And because this is the digital space, you were able to make the products really large to get up to the detail in ways that you wouldn’t otherwise and then spin them about and see them from different perspectives. Again, going back to really bringing a physical angle to an e-commerce-like experience. And so I found this really compelling, something that I could see brands bottling up and using. And what I loved about the experience, as well, is that it leveraged the video that is also being used in other social media and other existing digital channels that their users are going to engage with. So there’s a good economies of scale there.

CARTY: Right, that’s fascinating. Whimsical, in a way, too, that you can’t really achieve in other forms of media. So that’s really interesting. In the book, you encourage experimentation for spatial computing champions, specifically mentioning beta testing, as you mentioned earlier, as a helpful way to gather some feedback and resolve some issues. How would you recommend soliciting these sort of beta testers in the real world for their feedback, especially on a global sort of scale?

EMRICH: Yeah. I think soliciting beta testers is the same for any digital experience or software solution. Some of the avenues that I would look at utilizing as a company is social media, so using your own social media or, even better, going out and finding the right influencers to help them raise up the opportunity to get involved early. I think the influencer avenue might be most opportunistic if you’re thinking about a mixed reality experience on a headset, because you’re going to want to find a community of headset users and users that already have the device that you’re looking to target. There are some beta testing platforms out there, of course. And so thinking about especially smartphone augmented reality, I think those beta testing platforms, they work quite well for you to be able to tap into existing audiences that are there, specifically to give feedback on digital experiences. And then don’t forget to tap into your own network, so what you’ve done in terms of building your newsletter or your email database, your clients. I think it doesn’t have to be such a wide net of consumers, especially if your target audience is B2B. Selecting some of your top clients or getting a wide range of your clients and then inviting them to an alpha program or a beta program can be quite helpful for you to get the feedback that you need early. And then the last is a lot of the development platforms themselves can be also quite helpful. Again, many of them providing simulation capabilities, so not, essentially, beta testing with users, but also others that have tools that can help you at least set up your beta testing in the right way. And so I would definitely look to engage with those features that are becoming available within the immersive space.

CARTY: I’ve joked before on this podcast that we are a tech podcast, so we are fundamentally obligated to discuss AI, but at least we were able to save it for the last question. You did create a custom GPT for this book that enables readers to ask questions about the book, and that’s a super cool use case coming from somebody that also builds GPTs on occasion. Let’s talk about AI on a grander scale. How do you see AI factoring into AR experiences and how those two will sort of dovetail in the near and the long-term future?

EMRICH: Yeah, AI is becoming like a peanut butter to the immersive jelly, as it were. And so, first off, AI is really the underlying technology of augmented reality. Augmented reality uses computer vision in order to help make sense of the space and your face and your feet in order to make the magic happen. So I just want to point that out. They’re definitely not at odds, although at some times in the media it makes it seem like we have to choose between one emerging technology to the other. Specifically, I think, with generative AI, I think there’s a couple of really interesting opportunities that are bubbling up. The first is in helping to create assets. And so we’re seeing this, of course, with text and video and GIFs and photos. But there have been great advancements in the use of generative AI to create 3D models, 3D animations, 3D meshes, and also to use the video and the photos that are being generated as textures to apply on these 3D meshes. And so it’s still somewhat early days for 3D media creation, but it really is in a good state to get your feet wet with prototyping, in particular, maybe some background trees, for example. And so I think it’s just a matter of time. As we look to the long-term future, I think we can definitely see that the writing’s on the wall that generative AI will play a big role in 3D asset creation. And this is really important because, at the end of the day, when we’re talking about augmented reality and virtual reality, we’re really talking about this shift of content from 2D to 3D, and you’ll need to have 3D content– models, animations, spatial logic, spatial audio– in order to create a compelling, immersive experience. And so having this generative AI assist in this way can really make this experience creation more efficient and perhaps more within your budget. The second way is as a co-pilot. And we’re seeing this, I think, across the board in software, but allowing for generative AI to be paired with developers to help with debugging and to help with the coding experience itself could really, again, help make this development process much more efficient, a lot faster, and allow it to really ramp up your teams. And this is another key thing with spatial computing is that this move from the wave of computing that we’re in today to where we’re going is going to require a brand-new skill set within your teams, and generative AI can really help with that. The last is this new WYSIWYG. WYSIWYG, this new one, is what you say is what you get. And so the dream is that you can allow your end users to become co-creators within immersive experiences. So using GenAI, eventually, your user can go into a virtual reality space and point to the ground and say “tree,” and a tree appears, or change the sky, or create different characters they can engage with that are in turn powered by GenAI, so they’re more intelligent NPCs. And so this generative world building is something that we’ve seen some concepts for and some just early signals on, and it’s a really exciting way to, again, really engage your consumer as an active participant, but this time not just experiencing but co-creating with you the brand, you the company, and your experience.

CARTY: I’m sure it’ll be here before you know it. And for the record, we also would have accepted the chocolate and peanut butter analogy, like peanut butter cup. That also would have been acceptable.

EMRICH: That would have been a better one. Now I’m hungry. [LAUGHTER]

CARTY: All right, Tom, lightning round for you here. First, what is your definition of digital quality?

EMRICH: I think my definition is a digital experience that is frictionless, that keeps users satisfied and engaged and coming back.

CARTY: What is one digital quality trend that you find promising?

EMRICH: I’ve been really interested in AI-driven testing in automation. I think that this is an exciting digital quality trend that allows for you to automate test cases and even help with speeding up development cycles. And so that is one thing I’d love to put on people’s radar.

CARTY: What’s your favorite app to use in your downtime?

EMRICH: OK, I want to say MasterClass because I want to sound smart, but it’s probably TikTok. I spend a lot of time on TikTok. I still learn on TikTok just as much as I would a MasterClass.

CARTY: Hey, it works for a reason, right? Yeah. And finally, Tom, what is something that you are hopeful for?

EMRICH: I am hopeful for a world in which people are wearing wearable technology, and that wearable technology helps people change their perception of the world and really adds to who we are as humans and gives us superpowers.

CARTY: Amen to that. Thank you, Tom. We appreciate you joining us. It’s been fun.

EMRICH: Thank you so much for having me. I appreciate it.