Intro

Bertrand Schmitt

Hello. Welcome to episode 45 of Tech Deciphered. Today, we are going to talk about VR, AR, MR; mixed reality, in the context of the announcement of the Apple Vision Pro, their first spatial computer. Obviously, already talking about VR, AR, MR is old school, we should be talking about spatial computer. Apple has been very careful about not employing the world VR or AR. Maybe let’s start about what is all of this and how do they work?

Section 1 – What is it and how does it work?

Nuno G. Pedro

Let’s start with an easy one, which is extended reality. Extended reality is everything. Extended reality encompasses augmented reality, mixed reality, and virtual reality. When you see XR, that means extended reality. It’s the encompassing word for all these forms that we’re going to talk about today.

Nuno G. Pedro

Augmented reality is probably the simplest form. It’s a view of the real world with some pieces of digital. But normally the definition for augmented reality is it’s a non interactive domain. We’ll come back to that when we talked about mixed reality. It’s you basically seeing things that are on a plane, for example, using your mobile phone, your iPhone, an android device, and looking around you and seeing things that appear that are digital, but they appear in the real world. That would be augmented reality. There’s almost no interaction involved, etc, etc.

Nuno G. Pedro

Mixed reality brings in the element of interaction. You still see the real world, but you also see these digital objects and these virtual objects and you can interact and make them interact and you can interact with that world. To be very honest with you guys, I’m still an old-school guy. So for me, augmented reality and mixed reality are effectively the same.

Nuno G. Pedro

I think at some point, someone decided to do this distinction, maybe because of the devices that are linked to it and to really separate in particular what was happening in mobile augmented reality, what was happening with mobile phones. But ultimately, for me, they’re very similar. Obviously, for me they’re almost indistinguishable anyway. The people out there, that like distinguishing them, that’s the difference between AR and MR; mixed reality.

Nuno G. Pedro

Finally, there’s virtual reality, which is a fully immersive virtual world experience, the one that normally uses things like in the past look like helmets, things that basically take over most of your head because you want to be fully immersed. It can’t be just full, a simple goggle experience. It needs to be a more immersive experience. That’s basically what defines virtual reality or VR.

Nuno G. Pedro

Again, XR; extended reality, AR and MR, very similar. The distinguishable pieces, they’re both using real-world and virtual-world elements. What normally makes the distinction between AR and MR is that MR is more interactive, AR is more static type of experiences. Finally, you have virtual reality VR, which is the fully immersive piece.

Bertrand Schmitt

Yes, Nuno. I think I’m a bit with you, I’m a bit old school in term of AR versus MR versus XR. I certainly make a big difference with Zeus and virtual reality in the context of for me virtuality is you don’t see the rest of the world, you don’t interact with it, you don’t care, you are fully immersed. That for me is clear and I will say everything else is probably packed together in my mind.

Bertrand Schmitt

I think it might have come originally from, as you say, some devices that were able to do just very basic AR, adding some stuff on your regular glasses and that was it. But I guess pretty early on people realized there was not a lot of use for that. On top of this, a few distinctions we can make between a lot of these devices. Some of them require a PC to work or sometimes a smartphone and some others are fully independent, wireless typically as well and basically don’t require anything else to work.

Bertrand Schmitt

Obviously, that create typically some issues because a PC can be made very powerful if we want. But if you have to wear this device on your head and it includes CPU, GPU and the like, it won’t have the same ability in term of graphics. I guess you can argue that’s where Apple’s magic to reuse the CPU and the Vision Pro might make a difference.

Bertrand Schmitt

But typically, there is a big difference in terms of computing power in all this independent headset and that has been well exemplified. We’ll talk more about it with the Quest from Facebook that has cute graphics but definitely not too advanced graphics.

Nuno G. Pedro

Before we move further into this realm of AR, VR, MR, XR, etc. Maybe just a quick reminder, why does it matter? It matters because this is part of the domain of input and output. The domain of input and output, some people might have heard of it as I/O. Many will know because of Windows errors, right?

Nuno G. Pedro

I remember there were I/O device errors all the time on Windows in the good old days that we would see in those beautiful green screens and before that on DOS and all that stuff. I/O is, as the name indicates, input and output. Output is what you see. So it’s like your screen, it’s a manifestation, it could be your goggles. It’s something that you see basically that you are observing. It’s an output of what’s happening.

Nuno G. Pedro

A printer is also output because it prints, there’s something coming out of it that then you can actually observe or take a look at or do whatever you want with it. That’s the output side and then the input side are things like your keyboard, like your mouse, like a trackpad, etc. The reason why AR, VR mixed reality matter dramatically is that they are leading or they will eventually lead when they have a lot of adoption. They will lead to a fundamental shift on how input and output is done.

Nuno G. Pedro

Input and output today is done on the move. In particular, it’s done through your smartphone. The input is done on the screen of your smartphone, it’s done through voice, it’s done through other mechanisms and then the output is what shows on your screen and a variety of other things that you can link to that screen.

Nuno G. Pedro

The limitation today is you have these devices that sometimes look maybe way too big, but the limitation is still on the device. Actually, if you look, for example, at battery consumption on smartphones today, most of that battery consumption is on the screen, on the screen usage. So wouldn’t it be wonderful if we didn’t have that limitation defined like that?

Nuno G. Pedro

Wouldn’t it be wonderful if people like myself and others that basically use glasses so Bertom myself, for example, would fit into that, could do something with our glasses that is not just see through them so that we see better the world, but could we add other things to it? Could the output be more interesting?

Nuno G. Pedro

The reason why this fundamentally is important is if AR, VR, mixed reality takes off and before we talk about it taking off in terms of content, it needs to take off in terms of devices. If those devices take off, the world changes. The way we interact changes as it did when smartphones took off. That’s why this is interesting, that’s why this is super exciting.

Bertrand Schmitt

Yeah, the world changes or we create a new world, a separate world. Some will call it the metaverse. Going back in a way to input output, in term of devices, we talk about the difference where you have independent device headset or you need a PC. There is also the big question of outside-in tracking or not. Outside-in tracking means like you track it from the outside world.

Bertrand Schmitt

Typically early on, as it was used, there were base stations that were required in order to find where you are in the virtual world in your room, trying to position you in the virtual world and step by step, starting with the Oculus Quest, I guess, to have the first inside out tracking. So tracking done from your headset itself, so your headset having its own cameras and from these cameras trying to create a picture of the world and a position in that world.

Bertrand Schmitt

So that has been the approach used by basically most independent from the PC headsets and the Apple Vision Pro will be also an example of that. So the benefit of this approach, obviously, is that you don’t need to set up your room. Having done that, I must say it’s a hassle. When you have to start installing stuff either on your desk or in your room, you start to run wires, you have to have additional stuff you probably already kill how it’s going to spread as a device, pretty big time. The more cables and stuff you need.

Nuno G. Pedro

Yes. At the end of the day all of this is only purposeful as any device in the world, if there’s content. We’ll talk about it later today, but it’s only cool if there’s something fantastic that you can do with it. But irrespective of it’s, a new world that’s being created, a new reality that’s being created or not, this could fundamentally shift everything that we do.

Nuno G. Pedro

We could at some point, maybe a decade out, maybe two decades out, I’m not really sure all be using devices that look a lot more like what we see today, that we used to see in science fiction. Unless these sort of mobile devices that we have today with this limitation on screen and this limitation on how we do inputs into it, etc.

Bertrand Schmitt

To finish trying to explain how it works. The other piece of the puzzle, obviously, is how do you interact with this virtual world with this augmented reality? So some started with dedicated controllers, and that’s typically what you get with most headsets today. But Apple is launching probably the first headset without dedicated controllers and basically just requiring your hands, voice and eye tracking to interact with the world.

Bertrand Schmitt

So obviously, a way that is way more natural than requiring dedicated controllers. These controllers, they look like a big console joypads, joysticks, one for each hand. So pretty cumbersome typically to use and not prone to really immerse you in the way that just using your hands and eyes and voice could immerse you.

Bertrand Schmitt

Definitely they were not perfect for that. But I guess there were also a way to limit cost because the more you add stuff to the headset, the more you increase cost, the more you increase heat, the more you decrease battery life, and the more expensive it is to build.

Section 2 – The last (lost?) decade

Nuno G. Pedro

But we’ve been going at this whole AR, VR, mixed reality thing for some time now, easily over a decade and a half, much more than that, probably. So what’s been happening? One, there’s been obviously, a lot of investment, particularly in devices.

Nuno G. Pedro

We’ll come back to some miserable failures, but probably people will remember one of the biggest successes, which was Oculus and its acquisition by Facebook, which was for around $2 billion in 2014. There were also a lot of failures around the way. A lot of people probably remember the fanfare of Google Glass launch.

Bertrand Schmitt

Yes.

Nuno G. Pedro

I still have a pair. I fought so hard to get that pair, and it launched in 2013 and it seemed like a great idea and it was super limited. The battery is still limited and it didn’t quite work out very well.

Bertrand Schmitt

I think Google Glass launched 10 years ago, were a great example, actually, of a pure AR device in the sense of this virtual world that you display on your glasses doesn’t interact with the physical world at all. It’s just displaying on top of it some information. It’s not connected to what you are seeing it’s not adjusting itself to what you are seeing, just adding stuff on top.

Bertrand Schmitt

It was very limited in term of what you can view. Personally, myself, I always questioned the value ultimately. Obviously, there was this huge painful effect of how crazy do you look when you wear them and you are outside in the real world. That will be a question that all these manufacturers are trying to answer. I will say that it’s probably one of the very few ones that try to be advertising itself as something you could use in the outside world.

Bertrand Schmitt

Not at home, not in a plane, but really just out there walking in the street, going to a restaurant. If we remember, that created actually some significant backlash in public place where people realise that if you wear these Google Glasses, maybe you are constantly recording people, you have even no idea that you are being recorded.

Bertrand Schmitt

So that created a lot of questions about how do we want to live as people. You could argue today with smartphones in every pocket, everyone can be recording all the time, but typically at least you see that someone is specially using their phone to record you, to record around you, and you can take action or not. It’s not an assumption you are always on recording and you don’t even know if people are recording. Infamously, some people will say Google Glass were killed by Robert Scoble taking a shower with them.

Nuno G. Pedro

No, poor Robert. He’s a friend. It wasn’t his fault.

Bertrand Schmitt

No, of course. No. But maybe people started to realise that if this device you are supposed to wear anywhere, you bring it to the shower, maybe it’s a dead end.

Nuno G. Pedro

Yeah, there’s been some companies that have failed miserably in their own right. We’ll get to the big ones in just a bit, but I remember ODG. Many people probably don’t remember ODG.

Bertrand Schmitt

What is ODG?

Nuno G. Pedro

It’s Osterhout Design Group. It’s a company that did their own devices. They got a bunch of money from Microsoft, I think, 2014 for their IP. Many say some of it was used on HoloLens, others say maybe not. Unclear how much that IP was used, but I think they paid 150 million for that IP. So the company wasn’t bought, just the IP was bought. They came back to the table and tried to become a product company again.

Nuno G. Pedro

I believe they raised quite a bit of money, maybe 58, $60 million, led by 21st Century Fox or 21st Century whatever it was called back then in 2016. Then the company effectively ended up, from what I understand, imploding. They tried to do a patent sale again because they developed obviously, new IP. After the IP that they sold to Microsoft back then, I believe they just actually fell apart finally.

Nuno G. Pedro

The company exists for 20 years. That’s pretty impressive stuff. They had a great product. I remember testing it, etc. They had a great product, but it was a bit clunky. It was very difficult to attract developers for it, content developers, app developers, etc.

Nuno G. Pedro

So that’s one that I remember that I was personally in some ways. I wasn’t involved in. We didn’t invest in that company, but I certainly saw what the company was doing. But there’s other miserable failures and maybe the biggest of all is what, Magic Leap?

Bertrand Schmitt

I must have always been suspicious about this business since I’ve heard about them. It looks sketchy from the get-go, at least from my perspective. I didn’t look at the deal itself, but from far it just looks sketchy in term of science, in term of promise, in term of potential breakthrough, all from people who at least some of them didn’t look so inspiring and too focused on the marketing side.

Bertrand Schmitt

Some people try to copy Apple for the marketing for instance, but don’t realise that they need to also do the same level of R&D in software, in hardware. So Magic Leap actually raised 3.5 billion for a consumer device. They might still exist today, but in a much more limited scope, more B2B, focused. I think, and we will talk later about it, there is always this question of do you do a glass first type of device?

Bertrand Schmitt

So real glass that lets you see the world through it, but then something is added on top of it by external mean? Or do you do a camera virtual world first? I think it’s a very big difference in philosophy. Both HoloLens and Magic Leap and Google Glass went through the glass first type of approach trying to project additional stuff.

Bertrand Schmitt

Rumors says that Facebook also believe in that approach, even if they have not launched such devices. Ultimately this approach have been failing because technology is not ready. We’ll talk more about the Apple Vision Pro. But for me that’s really what’s interesting with Apple approach. Is that they acknowledge that if you want to go anything beyond VR, AR, MR, XR, you cannot use simply glasses, you need to go with cameras and you need to project everything on your virtual screen.

Bertrand Schmitt

Ultimately, Magic Leap failed. I’m not sure how real was their tech, but for sure is that they didn’t manage to make it a consumer device and there is probably to be frankly, little hope to do a brand new consumer device where the tech is totally unproven and where you need new type of apps and where you need new SDKs and new stuff from an upstart.

Bertrand Schmitt

I think there are ways and we’ll talk more about some players like Big Screen Beyond or Varjo that have a much smarter approach. Not trying to trailblaze in term of technology but do what’s the best possible with your current tech.

Nuno G. Pedro

There’s still dramatic investment in the space. Google’s been a little bit off the news on this, but obviously, Facebook has injected a tremendous amount of capital.

Bertrand Schmitt

I mean Facebook, we’re talking about 10 billion a year for their reality labs division. These are insane numbers. When you see the output, I would say it’s great that Facebook in a way has kept the space alive long enough to pass the torch to Apple.

Nuno G. Pedro

What are they spending the 10 billion on really?

Bertrand Schmitt

But it sounds like a lot of money. It sounds like too much money actually. Something not reasonable in term of what has been the output. That’s always a bad sign because of course scientists, engineers can go crazy for years with a lot of money. But at some point you need not just to try your product market fit, but you have to have some real success. We’ll talk more about the numbers.

Bertrand Schmitt

Should we talk about maybe one of the last one that was pretty visible? It’s the HoloLens. A device from Microsoft. Some sort of Magic Leap maybe we can say. Definitely a glass first type of approach. Very focused on mixed reality. It’s quite recently like a lot of big tech Microsoft reality and apparently the HoloLens team was one of the big casualty where most of the team was fired. Not all the team, but most of them. To be clear, Microsoft has other efforts in place in term of virtual reality, mixed reality. But that team, that device is mostly gone.

Section 3 – The inflection point for devices?

Nuno G. Pedro

Yeah. It’s maybe Microsoft saying we’ll wait and see and what happens. It’s maybe one of these spaces where there’s been so much attempt, so much capital put into it that people are just waiting for some big breakthrough to come through. But that leads us to moment zero. Is this moment zero. This the same moment we had with Apple iPhone.

Nuno G. Pedro

When Apple announced the Vision Pro this year, is this the moment where we are getting the revolution in this space? You’ve mentioned already some characteristics of the product and we’ll go a little bit more in detail on it. I think it’s interesting that the reviews are positive. Although the reviews are very limited. They were done in a very sort of constrained environment, limited to 30 minutes, all of that.

Nuno G. Pedro

But at least very good to hear that at least the reviewers were very impressed and some of them were skeptics. So it’s great to hear that. It’s very important to note that this might be the first big product launch of Apple that Jobs wasn’t really involved in. Everyone’s like, “Oh but there’s all these things they’ve done,” well a lot of the things they’ve done over the last many years are accessories.

Nuno G. Pedro

There’s some debate whether the watch was already something that Steve had been involved in or not. This is a big SKU, right? This is a big product line, not just an SKU. This is a big bunch of SKUs that Apple over time will hope to launch into the market. I don’t know. We’ll go into more details on it. The price, 3.5k, all of that stuff. It seems like a too big to fail. I think I’ll get one. Although now there are some rumors that it’s by appointment only and all that stuff.

Nuno G. Pedro

So we’ll see how that works. Very limited production. I’ll probably try and get one. I suspect you Bertrand will get one anyway as well. Just because it is Apple and it’s a big thing. But is this the one?

Bertrand Schmitt

Like you, I’ve been collecting gadgets for a long time and smartphones, of course, but also headsets. I didn’t get the Google Glass, but I got the first Rift, I got the Quest, I got a G2 from HP. I got a Varjo. I ordered another one, Bigscreen Beyond. We can talk more about some of them. My point is that I have some perspective on all of this, firsthand perspective.

Bertrand Schmitt

I must say I’m getting quite excited by the Vision Pro. For sure, on the tech side, it’s impressive. In a way, rightfully so, given for 3,500, it’s more expensive than a lot of headsets out there. Not all of them, but definitely more expensive than any consumer-scale headset. In a way they were able to put all the tech that they believe they need. It’s really, in a way, an approach of build it and they come, and you hope they will come. But as you, I’ve been pretty impressed by the reviews. I’m not sure I saw a negative review on the Apple Vision Pro.

Bertrand Schmitt

For me, the most important part is that it’s definitely an XR device. Can it do pure VR in a way? Yes, it seems so, even if Apple is absolutely not talking about VR, or not presenting any VR example so far. For a device that could potentially do it, it’s pretty impressive how the marketing went all the way to not talk about VR and not propose any use case of VR and actually not include any VR peripheral with the device. You could argue that in VR where you don’t see the outside world, you absolutely need VR peripheral.

Bertrand Schmitt

It’s very surprising. It means that Apple is focusing on an XR environment, mixing the real world with the virtual world, or proposing some sort of old school let me show you a movie in 3D, for instance. But technically it’s not VR because you can still see the background of your home, even if it’s really dimmed. They’re going for something else in terms of interaction.

Bertrand Schmitt

As I said, no peripheral, just hand, eyesight, and voice. For sure, when I say just, I want to be clear. It seems impressive. It seems like this time really working. Hand tracking perfectly working, eyesight. It’s not the first headset where they track your eyesight. I have a Varjo Aero that track eyesight, but it’s the first one where it’s definitely your UX. You are using that to control where you want to point to click.

Nuno G. Pedro

But the battery is connected to it. It’s standalone with a wire to a battery. No?

Bertrand Schmitt

Yeah, but the battery can be in your pocket.

Nuno G. Pedro

Yeah, but that’s not… I mean, I could see poor Steve Jobs turning around in his tomb like, “Mate.” The device does look beautiful, right. But it’s like, really.

Bertrand Schmitt

I have trouble to criticise because some headset manufacturer put a big battery in the back of the band in the back of your head. Is it really so much better to increase the weight, to increase the heat on your head versus, “Hey, let’s just move it back to a pocket?” I don’t know. For me, it’s fine until tested. Maybe I will say, “No, it’s impossible to use,” but so far I will say, “Why not?”

Bertrand Schmitt

For me, it’s independent. I was surprised. I was preparing myself for something that requires an iPhone, for instance. But no, it’s totally separate device. Actually, the big stuff is that Apple is really positioning it as something else, like a spatial computer. For me, that was very interesting.

Bertrand Schmitt

Obviously, it’s marketing, but at the end of the day, it might go deeper than that. It might be that this is a different type of computer, it’s different from everything else, and as a result, it requires its own compute. We cannot do it by relying on something else. I guess hopefully at some point you don’t need the battery in your pocket. I can imagine people providing additional headband to put the battery on your head if you want. The battery is detachable, so I can imagine smaller batteries, bigger batteries coming up, and batteries you can put at different locations.

Nuno G. Pedro

As you guys can listen to our comments, I feel maybe Bertrand is a little bit more optimistic on this. I’m a little bit more on the negative side. I think we’re still in fuzzy logic, so we’re still not fanboys, we love this, whatever, or we hate this, it’s going to be awful. But I’m a little bit more on the negative side.

Nuno G. Pedro

The device does look beautiful. Apple’s amazing at marketing. The whole spatial computing thing is classic repositioning of a product category saying, “No, whatever you guys are talking about, this is not that.” It has other advantages. It has the advantage of you saying, well, it’s not really VR in that sense. It’s not really that or this or whatever. Gets potentially a get-out-of-jail free card, but this is a bit too big to fail.

Nuno G. Pedro

The interesting thing is it’s too big to fail but they haven’t gone all in either. Just the limitation on the production is clear to me that Apple is edging on the product launch. Because if Apple was like, “This is going to be huge,” they would be going huge on it. There’s no way in hell they would come out saying it’s going to be 500,000 in production for the first year.

Bertrand Schmitt

I’m not sure they are saying that officially. It’s more based on rumors that initially production run of a million.

Nuno G. Pedro

But still, Bertrand, they’re not saying anything, but if it is true that it’s sub one million devices, it’s not a huge bet by Apple at this stage. They’re waiting to see what’s going to happen with the product. Some question marks on whether they put all their resources into this. This is a huge new product introduction. My feeling, and it’s a little bit more than a feeling, but my feeling is that they are seeing if this is going to work or not. We’ll see. Again, too big to fail, because it is a big deal and they have to find new product lines. At the same time, it seems like they’re edging a little bit on the launch, which is a little bit unApple, I think.

Bertrand Schmitt

I don’t know if they’re already hedging. Yes, it’s not a lot of device for a first year. At the same time, it’s a new kind of device. What I’ve heard is that they definitely have production issues. When you look at this device, they have gone crazy in the sense of it’s so many new stuff and done in a way that is very difficult to build. You have this simulated view of your eyes so that others can see your eyes. A lot of stuff that personally, I’m not sure this is really necessary for everyone.

Bertrand Schmitt

Maybe they went a bit too far has really created production issues. At the same time, I would say historically, Apple has really killed products. They try a V1, they try a V2, and typically by V3 it starts to get great. You have seen that with the iPhone, we have seen that with the smartwatches, even the iPad. There’s a big difference between the V1 and the V2. I will say compared to a lot of companies out there, maybe we can say Google, they are trying harder. When it fails or when it’s limited success, they try harder. They launch a V2, they take their time. It’s part of a strategic push, so we’ll see.

Bertrand Schmitt

But I must say that, yes, I’m cautiously optimistic and even potentially optimistic that after what could be seen as a lost decade of VR and AR, we might be on the cusp of something else, with the Apple Vision Pro leading the way. To be clear, a 3,500 consumer device is way too expensive. I’m not saying that this is it. It might take five years to go where it needs to be powerful enough and cheap enough for consumers. But at the same time, I must say I like this approach. Going consumer too early with a device that is really not great, that’s trouble. How do you create a new world? How do you convince people? Yeah, you have volume, but then how much is it really based on good use cases?

Bertrand Schmitt

I must say I’m quite optimistic that between the competition with Meta and some new upstart we might talk about later on, there might be a renewed focus and therefore renewed interest from content producers, app developers in this space. Also let’s not forget the Vision Pro should be pretty easy to program. If you are used to the Apple ecosystem, it should be… I don’t want to say piece of cake, but very streamlined. The same tools, the same SDKs, the same approach. I forgot to say, but any iPad application will work out of the box. Obviously, you will want to fine tune that, but that’s a great start versus how hard it has been in some cases to develop for VR.

Nuno G. Pedro

I think a very positive piece of this is precisely that Apple does know how to manage developer ecosystems really well. They have very large developer ecosystem right now. The only guys that would compete with this would be Google, really. That’s very positive. It is also very positive that Apple is deeply involved in content development through Apple TV+ and through all the bets that they’ve made in the space. They understand content, they have spent money on content.

Nuno G. Pedro

I believe we referred in the previous episode, their first launches weren’t great or at least took a little bit of time to take off, but now they seem to be on a roll. It seems like as a streaming service, they’re hitting the right stride, and it’s impressive. We now have cult series out there, Ted Larsen and all this stuff, which is really cool.

Nuno G. Pedro

In terms of content development, developer ecosystem, management, tooling, the creation of tools for developers, ability of the operating systems to really take all of this stuff and take it to their hardware in the best possible way, there isn’t a better player to do this. Doesn’t take some of my doubts away, but it certainly gives them a shot. It certainly reduces some of the risks that you might have when you go into something like this, which is just creating a new category and just creating all this ecosystem around a new category.

Bertrand Schmitt

To be clear, I’m not too worried about Apple and Facebook. If it takes them 10 years instead of three years or in term of five years to get there, I mean, they have the money to spend, they’re printing cash. Obviously, as we have seen with Facebook, at some point, there is all the pressure coming up and asking knocking on the door, what’s going on guys? You have to be careful even if you are printing money that your investors are still happy. But I think they can do it.

Bertrand Schmitt

What is always more dangerous obviously, is for startups. If you go all in on a new innovation at the wrong timing, you cannot always wait five or 10 years for success like some of these big companies can. Maybe going to some of the competitors to the Apple Vision Pro, at least the current ecosystem, or at least what’s coming soon, what’s coming next.

Bertrand Schmitt

We have the Meta Quest 3 announced by Facebook days before the launch of the Vision Pro, obviously, trying to steal some thunder at only “$500.” A much cheaper price. Interestingly enough, the Quest 3 is the first non-pro Quest device with the ability to show the world in an extended reality perspective.

Bertrand Schmitt

In the past, you could see some pass-through video, but it was horrible. You would certainly not use it as much as you can. I think they are going to go for it and leverage some of the Quest Pro technology. That will be the big difference versus the Quest 2, is that it will be easier to use within your real world.

Bertrand Schmitt

That has been an issue of VR for a while, is this inability to deal with the real world, forcing you to remove your headset the minute you want to do something else. You want to drink something, you want to check your phone, you want to do this, that, you have to take away the headset, limiting your immersion factor.

Bertrand Schmitt

That’s the big thing coming up with the Quest 3. Again, $500, launching in fall. Meta has launched the Quest Pro early this year or late last year. Interestingly enough, it looks like it’s doing pretty bad. Launched at $1,500, price reduced to $1,000 in three months. Doesn’t happen often. Now there are rumors that production is stopping.

Bertrand Schmitt

Personally, I’ve been very surprised by the Quest Pro. Very focused on extended reality, but not great at that, not really focused on VR, where today practically is the money, and at the same time not going far enough like Apple is doing to really give a shot at providing a very singular mixed reality experience.

Nuno G. Pedro

I have the Meta Quest 2. I used it a little bit more, but I have a bunch of devices in my closet that I’ve used maybe once, twice and then never again. The Quest I use once in a while. I’m not super impressed by it. It’s a great, as you said, entry device, the Quest 2. I don’t know. We’ll see. I think there’s a lot of devices, but ecosystem building for them matters. As you mentioned before, Facebook and Apple are probably the best positioned players to really take it to the next level, and of the two, Apple is probably the best, so we’ll see.

Bertrand Schmitt

To this point, I want to share some numbers. Apparently, based on internal talks at Facebook, there has been 20 million Quest headsets sold, and that would include… By only 2023, that would include the Quest 1, 2, and Pro. The vast majority is coming from the Quest 2. That’s the type of numbers we are talking about, 500,000 for a Vision Pro, 20 million for a Quest Two, mostly. In term of success, if we compare to other devices, they’re still very small.

Bertrand Schmitt

The other piece of the puzzle, as you talk about device in your closet, the big issue is people don’t use their headset or at least maybe the first cohorts were using more the headsets, but the latest cohorts were not really using their headsets. You buy it, you test it, you have fun a bit about it and then you stop using it.

Bertrand Schmitt

Maybe super quickly to finish with some other headsets and we can talk about the bigger picture. Varjo, a company from the Nordics, has been quite successful, I would say, selling B2B headsets. They’re probably the market leader in term of devices used for professionals and typically their headsets were typical of price points for professional at 6,000 or plus.

Bertrand Schmitt

Interestingly enough, their headset would typically include most of the stuff we see coming from Apple. Eye tracking, hand tracking, mixed reality, very high quality screens providing very good resolution. One thing we forgot to say is that what’s typical of the Meta Quest devices, pro or not, is they have very low resolution. When you are used to computer screen high resolution, phones high resolution, going to hyper-pixelated screens on your eyes is very odd. I think that’s part of the reason it has been an issue people are not so much enjoying.

Bertrand Schmitt

Varjo has been very successful on the enterprise side. But very successful is, of course, not so much success in term of volume when you are dealing with B2B. They launched a prosumer type of device, the Varjo Aero, at 2,000 that has attracted a pretty big fan base if you are into flight simulation or racing simulation. It’s probably the headset of choice today if you want to do that in VR, simply because it provides a much higher resolution than the Meta Quest devices. In term of immersion, it’s simply night and day.

Bertrand Schmitt

Another player just launching, Bigscreen Beyond. It’s lightest by very far VR device. We are talking about, I believe, 127 grams, so very light. We didn’t talk about Apple device, but it looks like we’re talking about 1 pound, 500 grams. It would be way lighter than the Apple device.

Bertrand Schmitt

I certainly believe comfort is a big issue and part of the comfort question is how heavy is your headset. Having something so heavy on your head limits how long you are probably going to spend time. Yes, you can create great headband and stuff and try to distribute the weight properly, but at some point, your neck is going to suffer, your shoulders, your head.

Bertrand Schmitt

I think that’s a very interesting device. It’s also custom made to your face, which Apple is not doing, by the way. For me, that’s a device to follow because it can be really transformational and it can point to where is the future going in term of VR.

Bertrand Schmitt

Maybe one type of device we don’t talk much, best exemplified by the XREAL Air or the Rokid Max. These are portable movie theaters, so these devices are presenting themselves as AR device but at the end of the day, these devices are just used to watch movies or play games, and play games from your Nintendo Switch, from a handheld device, or from a PC, but it providing you a very big screen. If you are traveling, if you’re in a plane, if you’re in a hotel room, that might be a great portable replacement for a movie theater at cheap price. We’re talking about 300-500 bucks.

Section 4 – The inflection point for Software and Applications? Developer is king…

Nuno G. Pedro

Well, the king is going to be content and the king is going to be software and the king is going to be applications for these things. Maybe switching to that, and maybe actually starting with the big apps and the big content pieces. Everyone says when there is a big shift in content platforms that the X-rated guys, the triple X people lead the way. Probably that was true early on, that there was a lot of innovation-driven into this by that market. Probably less true now.

Nuno G. Pedro

We have a lot of things that dominate the charts today on the Oculus Quest 2s of the world, etc, are around gaming, around meditation, around wellness, around fitness. Things are probably looking relatively obvious, but at the same time, we haven’t had really a title that has taken over the world. Probably the reason for that is that there isn’t a device that has taken over the world. Even the Quest 2 seems to have done quite well, but they are still small by comparison to what mass market looks like. What do we see ahead of us in terms of what will come?

Nuno G. Pedro

For me, gaming needs to lead the way for sure. I think just content consumption, unless there’s some formats that we’re missing right now, will be there and it will be important, but gaming is going to be the way to go. If we have games that really take it to the next level in terms of these experiences, then we might have some one, two, three, four leading titles, really large titles that lead the way. What’s your view?

Bertrand Schmitt

That’s always a great question. I certainly believe it would be way more gaming than X-rated content, that’s for sure. For gaming, interestingly enough, probably the most sold title in VR has been a game called Beat Saber, and it’s a pretty fun game. But what’s interesting for me is that it’s a totally different game than you would get on mobile and PC or anything. I mean, you couldn’t really play that game in this environment.

Bertrand Schmitt

Like it happened in mobile for instance, what are the most successful games in mobile? Typically these are mobile native games, not PC imitation type of games. Yes, at some point you can reuse a brand, you can adjust a brand, you can do stuff, but in terms of major success, it’s typically stuff created natively for mobile and taking advantage of mobile.

Bertrand Schmitt

It’s a touch screen, it doesn’t require controller, it’s a multiplayer, for instance, that sort of stuff. Beat Saber, you use your VR controllers as virtual sabers and actually it feels good because the controller is something relatively big that you hold in your hand. Typically it could be a saber. It is well done from that perspective.

Bertrand Schmitt

I will say the only exception to that I believe are simulation. If we are talking about car simulation, flight simulation, I believe that there can be a translation from what’s happening on PC and going to VR. Because here, at the end of the day, the goal of this simulation is to be as close as possible to the real world. It’s not as if it has to be native for VR. What it has to be, however, it has to use some native controllers. I mean, native to the simulation. You want to have a real wheel to drive your car, you want to have a real yoke to fly your plane.

Bertrand Schmitt

But that 2D screen, if you can get a 3D immersion, if you can get triple screen, yes, you probably want that. As long as the immersion in VR is not going to be so poor versus what you get on a 2D screen. Let’s not forget that VR is more resource intensive, if we want it to look as close as possible to reality and not a simplification of reality. That’s definitely one expectation I have.

Bertrand Schmitt

The main market that we have seen so far for VR, it has been gaming, and that has been one issue for XR is that basically there was no use case for XR except if you are a high-end designer, you need to design your car or some complex engine and make sense to try to visualise it in AR. Even here, question mark, do you need AR or is VR enough?

Bertrand Schmitt

But here, what’s interesting with Apple approach, Nuno, is that it looks like they’re absolutely not going for gaming. I mean, out of that one hour or 40 minutes presentation, there was maybe three, four seconds to gaming. Even that gaming was not a native experience at all. It was just, “Hey, this iPad game is working inside our headset,” and using a regular, not a VR gaming controller. That was very odd.

Bertrand Schmitt

For me, that’s very interesting. That will bring some diversity to the use cases that we experience today, and that’s very exciting. I want to highlight a few type of use cases I believe could be interesting, at least if you have headset at the level of the Vision Pro. First, that could be obvious, but it’s watching 3D movies. Do you remember your 3D TVs?

Nuno G. Pedro

Yeah, it never scaled.

Bertrand Schmitt

Yeah, it appeared out of nowhere and in six months everyone is talking about 3D TVs. Two years after everyone forgot about 3D TVs. I’ve rarely seen a mass consumer market phenomenon going up and down like this. You could argue this might be the right device to experience 3D movies. For me that’s interesting.

Bertrand Schmitt

Obviously, as usual, we see first the typical 3D movies that are already done today but maybe going forward there might be way more interesting 3D movies done more natively for the medium, so not shot for 2D. But it will not be an easy one because to invest if your total world audience is 500,000 people, that might not be enough to support any SaaS investment.

Bertrand Schmitt

Another interesting one is sports. I kept hearing that some demo that Apple did of sporting event where they record the event live or not, but they are putting a bunch of cameras at one spot and you can basically experience virtually this spot as if you were there. Was really amazing.

Bertrand Schmitt

There were examples of doing that during some basketball event, NBA event and people were like this is amazing, this is like having the best seat and experience it as if I was ready in real life. That is interesting. I’m not myself much of a sports guy, so I’m not sure what to think about it, but it felt interesting.

Bertrand Schmitt

Another use case I found interesting was doing meetings. As you know, the big issue with VR is that typically you don’t see yourself. Quest with the Quest Pro came up with some ways to check the state of your face and to try to represent that on your cartoon-like avatar. But I’m not sure it’s a way to really mix people using your headset with other people with this cartoonish avatar.

Bertrand Schmitt

Apple approach to scan your face and create a very realistic avatar is very interesting to me. If it’s good enough to do meetings, that might be an interesting use case because suddenly you can do your video meetings while working and people keep seeing you perfectly. You are not just forced to be seated in front of your laptop or screen to do your Zoom calls.

Bertrand Schmitt

I see that as an interesting use case. Maybe after a few days I will say it’s crazy, but I’m willing to try. Apple presented some stuff like working with your Mac on your Vision Pro. I have trouble to believe anyone would want to do that, to be frank. But who knows, maybe when you travel and obviously, they present a lot of use case around doing mixed reality stuff.

Bertrand Schmitt

You see some stuff inside your living room and you play with it or you do some stuff with it. You can imagine having a chessboard on your living room table that is virtual. That’s probably the spot where I have most trouble to see anything. It looks cute, looks interesting in principle. I’m sure there will be some great chess programs you can use. But seriously, that’s the part where I’m not really sure.

Nuno G. Pedro

I think all the aspects that you mentioned will be potential killer contents or killer apps. The fully immersive opportunity is very compelling. There’s other input and output elements out there that are still being developed. We won’t talk about them today like holograms and holographic technologies and all those things.

Nuno G. Pedro

I feel this paradigm as the highest likelihood of winning. If we’re thinking about the next 20 to 30 years, this paradigm of glasses, goggles, head devices is likely going to win for the next 20 to 30 years rather than holograms. But at the same time, what’s going to be the killer app, killer content, who knows?

Nuno G. Pedro

We’ve seen some pretty cool stuff already developed. I mean, there have been award winning movies, short movies using a lot of these technologies. My hypothesis continues being that there’s going to be a significant game that will take off and then there will be other things.

Nuno G. Pedro

There will be content that will come on and at some point there will be diversity, which is effectively what happened with the App Store launch. If we go back in history with the App Store launch in 2008, one year after the iPhone launch, we started having diversity of things that people can use.

Nuno G. Pedro

Multiple apps, multiple things that serve different purposes, utility. Maybe that’s what’s going to happen with this. Maybe the key thing is that there will be so much content and so many apps out there that the proliferation of the devices will be immediate. Maybe it’s the other way around. The proliferation of devices will happen sooner.

Nuno G. Pedro

Honestly, at 3.5K Pop, I don’t think that will happen with the Vision Pro. It might happen at 600, $700 in a follow-up device. It’s not out of the realm of possibility that within a cycle of two to three years, we might have a device that finally takes off and that’s exciting.

Bertrand Schmitt

For me, what’s interesting is that Apple might have been with Vargau, the first player to start with a pro version of their headset. Definitely, they put some thoughts into it. If you remember the first iPhone, it was not Pro. The first iPad, it was not Pro. Actually, the pro models came later.

Bertrand Schmitt

We are starting with a pro model. I’m very hopeful that it means that actually there are cheaper devices down the road and hopefully Apple will get the market feedback about what’s really necessary or not to include in that non pro version. Or just simply Apple will wait, that the prices of components and manufacturing will just go down over the years to launch, I don’t know, in two, three years, a non pro device, and maybe we’ll get a light version as well.

Bertrand Schmitt

Apple doesn’t typically do light, but who knows? That would be an interesting development. Maybe another interesting use case might be to record, finally 3D videos. Enjoy 3D videos in return by yourself. There have been example of you can record with the Vision Pro, anything, anytime while wearing it and then you can re-live it.

Bertrand Schmitt

That might be an interesting new way to create your own 3D videos and apparently the expense was very impressive. So that’s another potential use case and I guess we will see Sony and others building devices just for that.

Bertrand Schmitt

I would say in term of price, another issue is not just the price, but when you think about it, most people absolutely need a phone, you need that device. You cannot survive the world without that device in your pocket because you need to communicate, you need to know where you are, you need to take pictures. That’s why it has scaled to billions of devices and also because the price point went cheaper and for $200 you get a smartphone these days.

Bertrand Schmitt

It’s based on need. Second need is your computer. You need a computer in most cases at least if you’re a white collar, to do work. It’s not an option, you need that computer. Interestingly enough, if you look at console, if you look at tablets, you don’t need them to survive. You can do without a tablet but my point is that you start to get into devices that you don’t absolutely need. You can get by without them.

Bertrand Schmitt

The console, you can play games on your mobile, on your PC or Mac, so the console is again, another step forward. That’s a bit the question for me of the headset. Is it another device that you don’t really need, but it’s nice to have? Then it can still be big. Tablets are big, but it won’t be as big as a smartphone. I know a lot of special investors are looking for the next killer big thing to change dramatically everything.

Bertrand Schmitt

Will it really be that or will it be more a niche success? A niche can be, we are talking about hundreds of millions of tablets and for console, similar numbers, maybe lower numbers or are we talking even smaller? That’s really a big question. I have trouble to think that it will be smartphone-like in the next 10 years, 20, 30 years hard to say.

Bertrand Schmitt

Everyone has seen Ready Player one movie obviously, if it’s as easy and as high quality to use and cheap, maybe everyone get one. But for the next 10 years I would be very dubious.

Section 5 – A non sci-fi future?

Nuno G. Pedro

Yeah, and maybe that’s a good segue to the non-Sci-Fi future. What do we think is going to happen over the next 10 years? It’s actually interesting, I don’t think the Vision Pro is as big of a deal as maybe you think it is. Maybe it will lead to something else and that will be cool, but then I’m more optimistic on the timeline.

Nuno G. Pedro

I think over the next 10 years we will have the first fundamental shifts on input output that will not make mobile devices disappear over the next 10 years and on that I would agree with you, but would make some of these devices mainstream. I think these devices will be essential and will be the core devices for input and output.

Nuno G. Pedro

Then the discussion can be is it in the next decade or is beyond the next decade. Stake on the ground, I’d say probably within the next decade it will happen. I don’t know if it’s Apple, apple has a shot at getting it right. I don’t think it’s the Vision Pro. It’s the next probably iteration or two iterations down the road might have a better chance at that. But it will fundamentally shift the paradigm of input and output.

Nuno G. Pedro

Over time, mobile phones will become a thing of the past, in my opinion. Again, I’m a guy who grew up on mobile phones. I made a part of my career on mobile. We’ve discussed that before. But at some point this will shift and the mobile phone paradigm is still very limited, in my opinion.

Nuno G. Pedro

Is this going to be a lateral thing, like tablets consoles with very niche, limited market? Maybe it will continue being like that over the next few years. It’s still actually subpar compared to those two markets. But at some point, again, bet would be that maybe five years, six years down the road, we’ll see a hockey stick on some of these devices.

Nuno G. Pedro

The limitation I see on this, just to be very honest. There’s always this discussion around are these devices pushing us more and more to a world where we act like individuals, where we’re surrounded by screens? Like we already talk about mobile phones and screens when we’re having dinner because we have them in front of us and we’re not looking at each other’s each other as we should, etc.

Nuno G. Pedro

Maybe the paradigm will evolve to something that’s a little bit less intrusive, that people will get used to. I think when the Google Glass came out, it was just weird to see that in people’s heads, but people get used to it. Maybe there will be a shift, there will be a shift in how we interact and talk to each other. But at the end of the day, I feel that this non-Sci-Fi future is probably more upon us than we think it is. A little bit more optimistic on that.

Bertrand Schmitt

Yeah, to be clear, optimistic that this goes into the hundreds of millions a year. In 10 years from now, I just don’t see the billions in less than 20, 30 years simply because of price of opportunities. We are not just SF or San Francisco as a world. I must say I’m quite optimistic that we’re at a turning point with renewed efforts from Meta, the fresh competition from Apple, investment for both Meta and Apple that need to give results.

Bertrand Schmitt

They cannot afford to just close shop in three years on all their virtual reality investment, whatever they want to call it, at the same time, for sure, you cannot go faster than in a way, technologies allow you to do. That is clear, but they have the money to keep going and keep pushing.

Bertrand Schmitt

I will say I’m quite optimistic on the short run for more specific use cases like better screen, better input, output. To be clear, I think the $500 ones are basically only about us today to do games. I’m quite optimistic we’ll see some very interesting use cases.

Nuno G. Pedro

Even more interesting than that, just stacking up a little bit on what you’re saying. We didn’t even talk about B2B. We’ve been talking really mostly about consumer.

Bertrand Schmitt

Exactly.

Nuno G. Pedro

Just B2B by itself, we’ve seen some of the stuff already be being used in a very niche way for specific applications, industrial applications, manufacturing applications, taking away curbs of learning from some specialised workers. At the end of the day, we could do like a full couple of episodes just on B2B, AR, VR and use cases there. That alone could drive markets. That alone could drive markets.

Bertrand Schmitt

I totally agree. That’s some of the niche use cases I’m thinking about. I think there are use cases where current headsets were not just good enough at all, but we start to go to a level where between the price point, yes, a bit more expensive, but higher quality, between use cases that these companies better understand, better promote. We are getting to a point where the business opportunity will start to be pretty interesting.

Bertrand Schmitt

I think there are a lot of situation where you want to remotely control devices, where you want to cheaply train workers, where some of these devices are going to get more and more interesting. I’m pretty actually hopeful now that on the short run we will see some niche use case that would drive some opportunities in terms of business model.

Bertrand Schmitt

I can see in the 5-10 years time frame on top of the specific niche use cases, some bigger mass adoption and in 10 years, yeah, we talk about 100 million device. If we have not just 100 million device, but 100 million device that are used regularly, that will be a big win because let’s not forget today that’s a big issue.

Bertrand Schmitt

When people talk and try to compare to smartphones for instance, I’m always reminding people, yeah, for us a day, 6 billion people every day. At some point what you want to beat is really the number of hours used by humanity every year. That’s where VR devices have been falling short big time so far. But again, I’m hopeful that we have use cases in front of us on the short run that will make business models possible for a lot of companies beyond Meta and Apple. We can afford to burn money on the short run.

Nuno G. Pedro

I’ll go back a second and then I’ll bring it to a little bit of a summary from my perspective, but just going back a second, there’s no way something like this, for example, multiplayer gaming or social networking, metaverse type social networking, etc. There’s no way it can scale if there isn’t critical mass of devices. If there isn’t, it doesn’t exist.

Nuno G. Pedro

Effectively, at the end of the day, the big limitation we have today in a lot of even these contents and apps, even scaling, is there is no scale to be had. You need network effects that can’t happen without significant scale of device adoption. Maybe just bringing it to a frame and summarising my perspective is I think we are on the verge, maybe of the big movement.

Nuno G. Pedro

If I had to guess, maybe Apple has a shot at it. I don’t think the Vision Pro is it. It’s maybe the beginning, it’s exciting, but it’s not it. I don’t know if it will be a variant for the down-road from the vision probe. As I said, they sure have a shot at it. Can Facebook make it work first at scale, maybe? I don’t know. They have certainly a lot of deployments right now, devices.

Nuno G. Pedro

We’re on the verge of it. We’re not there yet. So, short term, maybe I’m a little bit more pessimistic than you, Bertrand, longer term in terms of timeline to get to real mass adoption, maybe I’m a little bit more optimistic.

Bertrand Schmitt

Sounds good, Nuno. It’s still very early on. It’s exciting times. Let’s see how it goes. But I would say overall, as usual, except if you are one of these big conglomerates, big tech companies, you have to be extremely careful about finding the right business model because we are still early on. I totally agree with you, it’s still very early for that mass adoption and that virality to happen. You need to find business models that don’t require that and you still can be successful as an entrepreneur.

Conclusion

Nuno G. Pedro

Very clear. This concludes our episode on AR, VR, MR, XR, all the Rs reality with other things that are virtual on it. We discussed what it actually is. What is AR, VR, MR. We talked about potentially the last decade that we just had, or the last decade and a half that we had. It wasn’t fully lost, but certainly there’s a lot of failure in there, despite a lot of investment.

Nuno G. Pedro

We talked about the inflection point that maybe the Apple Vision Pro brings to bear, and talked about other devices that are on the market, the role that Facebook, for example, has had with their Quest series. We talked about an inflection point that is seemingly happening around software applications and content. Finally, we ended up with some discussion around the future, the next 10 years and beyond, and the non-Sci-Fi vision of that future. Thank you.

Bertrand Schmitt

Thank you, Nuno.

Nuno G. Pedro

Thank you, Bertrand.