Is the metaverse going to suck? A chat with Matthew Ball

Maybe it won’t be a dystopia?

Let’s talk about the metaverse.

You probably can’t stop hearing about it. It’s in startup pitches, in earnings reports, some companies are creating metaverse divisions, and Mark Zuckerberg changed Facebook’s name to Meta to signal that he’s shifting the entire company to focus on the metaverse.

The problem, very simply, is that no one knows what the metaverse is, what it’s supposed to do, or why anyone should care about it.

Luckily, we have some help. Today, I’m talking to Matthew Ball, who is the author of the new book called The Metaverse: And How It Will Revolutionize Everything. Matthew was the global head of strategy at Amazon Studios. In 2018, he left Amazon to become an analyst and started writing about the metaverse on his blog. He’s been writing about this since way before the hype exploded, and his book aims to be the best resource for understanding the metaverse, which he sees as the next phase of the internet. It’s not just something that you access through a VR headset, though that’s part of it. It’s how you’ll interact with everything. That sort of change is where new companies have opportunities to unseat the old guard.

This episode gets very in the weeds, but it really helped me understand the decisions some companies have made around building digital worlds and the technical challenges and business challenges that are slowing it down — or might even stop it. And, of course, I asked whether any of this is a good idea in the first place because, well, I’m not so sure. But there’s a lot here, so listen, and then you tell me.

Okay, Matthew Ball. Here we go.

Matthew Ball is the managing partner of Epyllion and the author of a new book called The Metaverse: And How It Will Revolutionize Everything. Welcome to Decoder.

Glad to be here.

You are also the proprietor of an excellent Twitter feed about the metaverse. Do you think of Twitter as your primary platform?

I do. It is my most used app. TikTok is creeping up there — and of course my Screen Time doesn’t register Fortnite — but Twitter is definitely my primary channel and where I learn the most.

You have been tweeting about the metaverse for quite some time, and you obviously have a big audience on Twitter. From a media nerd perspective, why turn it into a book?

Thanks for the tee up. I started writing about this fascinating topic in 2018. The term comes from the early ‘90s, but the ideas span back to the ‘30s. This truly century-old idea was finally practical, that is to say, we could start building it and trying to realize it. Over the following years, I got smarter in the area, received more input from other people, and more projects came to bear.

Then suddenly last year it became the word du jour. Not only did Facebook rename themselves, but Google also did a reorg, Amazon started redoing job descriptions, and many of the fastest-growing companies in media tech — Roblox, Unity, Epic — wrapped themselves around the theme. Yet there was very little actually articulating what it is, why it mattered, and what the challenges were.

I was really excited about crystallizing that, distilling my thinking into something more concrete, updating the things that I got wrong, making sure that it was comprehensible, but the most important thing was actually social. Every time we have a platform shift, we have an opportunity to change which companies lead, which philosophies, which business models. I think many people are coming out of the last 15 years dissatisfied with the lack of regulation, the take rates, the role of algorithms, monetization, and which companies lead — and who leads, frankly. The best way to positively affect that outcome was to be informed about what was next. That is the goal.

We have to start at the beginning. There are a couple chapters at the beginning of the book where you talk about that long history and how it has built up to this moment. The third chapter is called “A Definition Finally,” which is great because I feel like the definition of the metaverse really does need that “finally” moment. What is your definition of the metaverse?

I cheat here a little. It is more helpful to describe it similarly to defining the internet as TCP/IP, the internet protocol suite. The description is what is more helpful.

It is a massively scaled and interoperable network of real-time, rendered, 3D virtual worlds that can be experienced synchronously and persistently by an effectively unlimited number of users, each with an individual sense of presence. It has the technologies, capabilities, and standards to support what is essentially a parallel plane of existence that spans all virtual worlds and the physical world. From a human outcome, it means that an ever-growing share of our time, labor, leisure, wealth, happiness, et cetera, will exist in virtual spaces.

One of the key pieces of that definition is “3D virtual worlds.” I have heard other definitions of the metaverse that are a little bit more expansive, that get you to a place where Wordle is the metaverse. We are all doing it together once a day, so we exist in the universe of Wordle, however that universe is defined. You are saying this has to be 3D; it effectively has to be a video game. You get to a place where Fortnite, Roblox, or any number of other massively multiplayer online games is the metaverse. Does that count for you?

It is really a question of “what is” versus “what connects to and is part of” it. My building that I am speaking to you from right now is not the internet, nor really on the internet, yet it is part of the internet in one way, shape, or form. Wordle, of course, is mostly locally run on your device. You would not really call it an internet service, but some of it is delivered.

When you are talking about the metaverse as a new computing platform, for me, 3D is a requirement to do many new things, to elevate human existence — especially in key categories such as healthcare, education, and so forth — but the term really does not matter. What is in and out does not matter. It is likely we never say “metaverse.” In China, they have adopted the term “hyper-digital reality.” We may talk about the 3D internet, or we may just use the term internet. What matters is the real-time rendered element, which basically means the world as it exists is legible and changeable to software, and the advent of graphics compute. It does not need to be a game, it is just an expression.

I understand what you are saying. It is the description that matters, and this word may go out of fashion. Let me just push on that description and definition a little bit.

Right now you can log into Fortnite and run around with a bunch of friends. It is cross-compatible with many different kinds of devices, so it does not matter what hardware you have in your house. You are in a persistent online space where lots of other people are. Are you saying that because Fortnite does not connect to Roblox, it is not the metaverse?

This would be a little bit like asking, “If AOL ran on multiple different devices and a few different networks, is that the internet?” We could say it is, but if you talked about just AOL services in particular, you would be talking about a proprietary platform. You would not be talking about a unified experience that spans into industry with myriad different outputs, servers, or domain registrars.

The metaverse is really describing that unified experience, rather than a single expression, much like we would not say Facebook is “an” internet or “the” internet. When you are talking about Fortnite, there are certainly a bunch of things that do not fit there. It is not actually a persistent experience, and there are very few people who can connect to it at one point. Nominally, there are 100 people in a match, but they use a bunch of cheats so that there are only really 12 people that matter. It also does not connect into anything that isn’t purely game-like and leisure-oriented.

The definition of the internet at its most basic levels is a network of networks. You are connected to the network at university, work, or home, where you can go out and connect to Amazon’s network of servers to browse, then leave Amazon and connect to Facebook’s network of servers to do stuff there. You are saying the metaverse is the same thing as that overarching network of networks; it is the connectivity between multiple, different 3D worlds.

That is right.

What I would push on there is that the internet did not have to be built that way. The AOL example is very interesting, because AOL did not want it to go that way. The value plummeted when AOL went from being a provider of first-party services — like chat rooms, groups, and email — to an ISP that connected you to better versions of those services run by other people.

What is the push for Epic Games or Roblox to enable that connectivity? Historically, the people who own those experiences faced a raft of competition the second they gave them up. They kind of became dumb pipes and disappeared.

Let’s pause for a second. Of course, that was not the necessary outcome for AOL. We know now that no matter how successful AOL might have been in expanding its geographic footprint in connectivity, the largest opportunity for them was in horizontal software and services. There is a world where AIM, AOL Instant Messenger, becomes one of the world’s most significant communication platforms, like WhatsApp or Snapchat. There is a world in which its search engine turns into one of the world’s most dominant ad networks.

Microsoft is a pretty good example of that. They have never had a smaller share of computing devices, hardware, or operating systems, but their horizontal business is far more valuable than ever.

When you are talking about the incentives, first of all, we are already seeing this progress. The Roblox founder and CEO has been talking a lot about their explicit designs for interoperability. They have open-sourced some of their scripting languages, and he is even talking about embracing NFTs to take some projects off of Roblox.

Last week, the Metaverse Standards Forum was established by the Khronos Foundation — 28 companies such as Qualcomm, Epic, Meta, and others — specifically to solve this problem. Coming together is the easiest part. It is not forcing anyone to make a concession yet, to pick something that they did not advocate for, but is all in service of expanded network effects.

The belief is if consumers can buy 3D objects that can be used in more places, or encounter history that has more persistence and utility, it will grow much like the world economy, did through trade. There were individual instances of compression with some markets, some products, and some countries, which suffered from time to time, but the network was much stronger.

I will say you are right that the internet could have gone a different way, but we did have many competing inter-networking standards. There was a point in time in the early ‘90s where the Department of Commerce and the Department of Defense disagreed and pushed different standards. The idea that Comcast could email IBM, could email Telefónica, could email China Mobile, was really not the consensus. We had the protocol wars, but network effects and utility won out in the end.

The idea of a Metaverse Standards Forum is very funny to me. When covering consumer technology, you come up against standards bodies all the time, and they are hyper political. I would not say that Bluetooth is an example of the tech industry making something great that everyone loves, but it is pervasive in its way. The beginning of a standard and that early energy is great.

At some point, dollars are going to get allocated across whatever the metaverse is, and owning the early access points seems really valuable. This race and amount of hype we are in now, is it really about initiating the customer into whatever the metaverse is, to make sure that every time they buy something in another 3D world, you will get your 30% cut? Or is it, as you were saying at the beginning, that the technical ability to start building an early version of what you might consider the metaverse a net good? Should we just start doing it and see what happens along the way?

I think the latter is more likely, but it is more of an organic process. If you take a look at one idea that we have long believed would have utility, a federated universal identity in digital space, Microsoft has tried that multiple times. The .NET Framework was the last big time they tried, but no one wanted it. It was rarely deployed, for many of the reasons you just mentioned. I do not want to use Microsoft’s account system.

What happened to be the best way to build the de facto standard for identity was Facebook, which started as a college hot-or-not. The best way to build a or the metaverse for Epic was not by trying to build it. It happened to be a battle royale game that was not even intended to be a battle royale. That is to say, this process starts from building something tangential, that is 3D-oriented and social, that connects into another thing. Then you start to get organic alignment around that standard set.

You are right to be skeptical when someone says, “This is the thing, let’s all do it.” It rarely happens that way. It is actually more power-based.

You have described the metaverse as this parallel reality that you can live in and transact in, that will grow an economy that mirrors the world economy, because we will figure out some way to have scarce digital goods. I will come to the blockchain portion of this conversation later, but that is what you are describing.

In science fiction, where the word metaverse comes from, that vision is always dystopian. In the book, you refer to Neal Stephenson’s Snow Crash a lot, and you point out that the metaverse in Snow Crash made life in the real world notably worse. The heart of tension for me is the idea that we will build a parallel world and end up as so many brains transacting on other people’s platforms. I have an instinctive recoil from that which makes me skeptical of the entire enterprise, because I think life in the real world is actually rich and rewarding. I can go out and touch grass, and Apple, Google, Facebook, Epic, or whoever cannot get in the way of me doing that. Fundamentally, what makes this not the dystopia that it is always described as?

I agree with a lot of that and disagree with some of that. The literature for the metaverse in its antecedence is dystopic. One of the important reasons why that is the case is because the point of most fiction is human drama, especially science fiction.

Put another way, utopias tend not to make for much human drama. This is true that when you look at Neuromancer, The Matrix, Ready Player One, or back to the 1930s with Philip K. Dick and Isaac Asimov; these virtual planes of existence are not described favorably. Why? Because even when they are not negative in and of themselves, they lead to some disengagement with reality, and that is the problem. The technology is amoral, the consequences are not.

When you take a look at the actual examples to build these things, whether that is multi-user shared hallucinations in the ‘70s, Second Life and other metaverse-style experiences from the ‘90s, or Roblox and Fortnite from the 2000s, the tone is very different. It is not dystopic, it is creation, exploration, identification, and collaboration. Those are all very important.

At the end of the day, I don’t know that scarcity is that important, and this is actually where I think I disagree with many of my peers in the investing community, especially in relation to the blockchain. I don’t really get virtual land, certainly not scarce virtual land. The two brilliant things about the internet are network effects and zero marginal costs. Trying to create a next-version of the internet that constrains networks through money and introduces scarcity that need not be there, for a virtual plane of existence that does not actually need to simulate the real world, I don’t get and frankly don’t believe in.

We have done a lot of interviews with various Web3 folks on the show. I would say some of the themes there echo the themes you have brought up. There are a lot of people who, having built or invested or experienced the last 15 years of the internet, are dissatisfied with where we have landed. Can we build a new kind of internet that more effectively rewards creators and is not just about engagement metrics?

You talk about the metaverse and say, “Okay, I want to have digital goods. I want to buy and sell things here that create a world economy that rivals the real-world economy.” How do you do that without scarcity? Are we going to DRM all the virtual clothes? There is an element here that you need to create some sort of scarcity if your goal is to buy and sell 3D digital objects at a rate of transaction that mirrors the real world.

It is really interesting. This is where we get into a fundamental break between how different believers in the metaverse actually imagine the value. Just as I am not a believer in scarce virtual land that costs thousands if not millions of dollars, there is probably a pretty low ceiling to virtual goods and apparel. They are usually in support of experiences. It is either the experiences that drive the underlying value — as is the case in Fortnite — and not the items per se, or it is what we would consider graphics-based computing or simulation at large.

Let me make that less abstract. Jensen Huang, the founder and CEO of Nvidia, now the seventh-largest company globally, believes that the economy of the metaverse will eventually exceed that of the physical world. We are talking 51 percent, which would be $50 trillion per year in spending right now. He is not at all interested in virtual clothes or leisure. He is talking about real-time 3D simulations running the world’s best development platform, which is the world. A building or infrastructure, where goods flow and why, how you programmatically advertise in 3D space, often for physical things, certainly does not require scarcity of the odd avatar.

Explain that a little more directly. When you say the best development platform in the world is the world itself, do you mean the 3D environment that you are in?

I mean the physical world, the one that you are standing on and exist in right now, which has many of the attributes I mentioned such as persistence, maximum capacity, et cetera. I will give two examples that are perhaps helpful.

Nvidia redesigned its headquarters with a real-time, rendered 3D simulation to understand every design choice. “What happens when you put a piece of window in one spot, or use one construction material or another? At exactly 3:22 p.m., November 22, what is the climate implication in the conference hall? How do you simulate the flow of energy, of heat, or the refraction of light to drive energy to operate the building?”

We are seeing that premise being used to operate airports in real time. “Do we really want to move the flight from gate 82 to gate 80 because it is close by? Should we actually move it farther away for operational efficacy and safety reasons in case there is a flash flood, fire, or terrorist event?” We are talking about making the entire physical world with an augmented layer on top of it that is legible to software in real-time, impacting production flows in a factory or the flow of people in a facility and so forth.

Connect that to the metaverse for me. This is a concept that is often called digital twins. You build an operational, digital twin of an airport or your office building, and they can proceed down different timelines based on different choices to give you a sense of what might happen if you make changes in the physical world. Do they interact? If someone is going from the digital twin of your office to the digital twin of the airport, is that where you think the metaverse is?

I think the idea of simulating physical environments more directly, more accurately, is very powerful. The idea that there will be some layer of commerce in those digital twins that is independent of what is happening in the real world seems like the big step.

There are two things to unpack. Number one, digital twins are not the metaverse. If the internet is a network of networks, of different autonomous systems exchanging information consistently under common protocols, then a digital twin is like an office network. It is the Vox ethernet.

It is the interconnection with other digital twins, other simulations, for the exchange of information — your user identity, your payment history, or your avatar if you so choose — that collectively produces the metaverse. In this instance, there is not necessarily any utility or purpose for you, the consumer, to explore the digital twin of the environment you are in.

You might wear augmented glasses in 2037, in which case a version of that digital twin is being overlaid selectively to you, but I don’t agree with the premise that we are going to navigate an airport by putting on a headset or taking out our device.

Are you saying you don’t agree with the premise that there will be pervasive augmented reality?

No, I do. My point is the digital twin, at least foreseeably, is a B2B application, not something that you, the consumer, is going to log into and explore. There is very little practical value right now in you saying, “I want to go navigate MIA, the Miami airport, in a 3D digital twin.” It is not interesting or useful. That does not mean it isn’t super valuable to the operator.

As you describe this, there are a bunch of very hard, technical problems to solve to make this all work. If I build a digital twin on Nvidia’s platform of the airport and someone builds another digital twin on another platform for the office building, it is not just me, the builder of the digital twin, that needs to want to inter-operate.

The platforms need a core capability to inter-operate. If I want to jump from Roblox to Fortnite, those companies have to agree that my avatar can go between the worlds. If I buy a gun in one video game and want to go to another video game where that gun is 100 times more powerful, I might just wreck it for everyone. Some of that is a very difficult technical problem, some of that is cultural, and some of that is straight up business and politics. Have you seen the beginnings of solutions to those problems?

You’re right. Most technology problems are only masquerading as technical problems, and are actually business and/or societal problems, as in “can we agree?”. In the gaming community, I see limited benefit from taking your gun or avatar from one environment to another. That is not to say that there isn’t some utility, particularly with cosmetics with no functional value. It is easier, but at the end of the day, how important is it that I can wear a banana peely skin in Call of Duty? Probably not that important. The technical impediments, not to mention the commercial and creative ones, are pretty high.

When you take a look at industrial simulation, the utility there is a lot higher and the technical solutions are already in place. You mentioned Nvidia’s omniverse platform, which is not really a platform in the same sense as Roblox or Minecraft; it is actually more of a middleware simulation DMZ. It is actually where DeSo and Boeing take their simulations and interconnect them, with Nvidia’s machine learning upscaling, downscaling, translating, and then operating that simulation.

There is a lot of work to do if you want to talk about the progress. We do have some standards groups, but there is an old xkcd joke that basically says when 14 people disagree about 14 competing standards, you get a 15th standard that no one uses. So I don’t want to be too optimistic there.

What you see with Epic is one potential example. They launched their Epic Online Services, a live services suite where independent game developers can access Epic’s 500-million-account user base with 3.5 billion user connections — and at this point $30 billion in invested avatars and skins. This is just like The New York Times tapping into Facebook’s account system to speed up the user flow. Not to say that they don’t prefer their own account, but they recognize there is utility in getting some information.

You and I can go make a game and then access Epic’s avatar suite and its users, therefore driving from smaller developers, who are less endowed technically and financially, to consolidate around their conventions, their file types, and their engine to tap into their networks.

I feel like we are bouncing back and forth between where the money is now and where the money will be in the future. To some extent, this is making my head spin. You are saying the money in the future is not just avatars, skins, and items. It is some massive B2B market where the real world is being simulated at a level of high fidelity, and some revenue will be created there as different businesses find different things to do for each other. The money right now is very much in Fortnite skins, right? How do you go from one to the other?

I don’t mean to oscillate between the two points. My point is rather that when people express skepticism as to whether or not standards and interoperability can be achieved, it is important to say that progress is happening. We had cross-platform gaming in 2018, we now have common account systems and entitlement systems for Epic, and we have the omniverse platform for Enterprise.

The fundamental tension you are talking about stems from the fact that, for decades, game engines, 3D simulations, have essentially been good enough for leisure and not much else. Unreal, for example, is a non-deterministic physics engine. That means that if you throw a grenade eight times, you might get seven different answers, somewhat.

It is only recently that the fidelity and sophistication of the simulation, and the investment that Epic has made into vertical solutions, make it practical enough for deployment in healthcare, military, education, and automotive. We are very early on that deployment curve. You need to get it right, then you need people to adopt it and so forth.

That is one of the reasons why we struggle with this odd juxtaposition of talking about the trillion-dollar metaverse economy while turning over and saying, “Right, but we are talking about $200 billion in gaming spent mostly on cosmetics.”

I just keep coming back to the notion that the metaverse is the inter-connection between these worlds. That is where the value multiplier is. You can build all this stuff as one-offs, and all you have really ended up with is AOL and CompuServe. If you connect those things together and to 100 different networks and servers, then you have multiplied the value of all of it. Everyone rushes into it because it is so compelling that you cannot say no. Suddenly we end up in 2022, and every now and again I’m like, “Maybe we should turn it off.” It eats the world in a way that seems remarkable.

The immediate, compelling use of the internet was obvious to everyone, in the sense that if you wanted to look something up, you could just do it faster. Wikipedia comes into existence and suddenly the Encyclopedia Britannica seems unwieldy, old, and not up to date anymore. The other day I wanted to figure out how to cook something, so I watched a YouTube video and that was the end of it. I knew how to do it and we were off to the races. Whrere are the compelling, immediate uses of the metaverse that showcase that multiplicative effect, beyond just getting to the Boeing simulation faster?

To start with, I would personally disagree that the utility of the internet was self-evident. I mean, we have the classic Paul Krugman example in 1998.

Well, I am not saying some people weren’t wrong. I’m kidding around when I say that I was just smarter.

No, I agree with you. One of the weird things is that transition point was actually relatively late. Even as late as 1996, there were fewer than 50 million Americans who would use the internet in a month, and most of that use case was pretty frivolous. When I was in high school, Wikipedia was seen as deleterious, that it actually worsened education. I think that is part of it.

What we are seeing here is network effects. I don’t mean to be evasive, but we are talking about combinatorial innovation that is not yet present and therefore remains speculative. Take a look at the world economy, as an example. It is not that having independent nations and industries wasn’t hugely profitable, it was that the utility of all investments of all products in all markets went up.

In the social era, we easily take for granted that anything we create works everywhere. I create text, audio, video and I can take it anywhere. I can take a photo with my iPhone, it stores to iCloud, and I don’t have to say, “Well, darn, now I can’t put it on Facebook.” I can put it on Facebook, right click, save as, upload it to Snapchat, screenshot it on Snapchat, and put it into TikTok.

The utility of global commerce and trade, the utility of having common file formats, is really profound on the internet. It is so hard to create in 3D. Then you have this issue where the thing you want to do in 3D is a different system from your partner. Unity and Unreal actually use different XYZ coordinates, if you can believe it.

It is kind of intuitive at this point to say we have had hundreds of billions of dollars in 3D assets invested, and all of those essentially get deprecated after their first use. That means that we either need to remake them, or we just will never use them. That is part of the premise here.

I will give you a counterexample. Emoji is a big standard. It is run by a consortium, but it is rendered differently by every phone, by every platform. So the smiley face emoji…

I am an Android guy, I know it well.

It’s the grimace emoji, right? On Samsung phones, for a long time, it looked like it was smiling. Samsung owners were sending people grimaces when they meant they were smiles or vice versa.

You have a 3D file format, and everyone has agreed, “Okay, this is the one.” How do you make sure it is rendered across all these systems? Over time, will Samsung have to realize, “A lot of people are confused by our emoji, we should come together with Apple and make sure they look the same”? Google had to go from blobs to faces, which was very controversial in the virtual world, I will point out.

I like the blobs.

People love the blobs, and Google got rid of them because Apple is dominant and they needed to conform to what Apple emoji looked like. Do you see that playing out with 3D objects? Will an outfit or briefcase in Fortnite eventually come to dominate what it looks like everywhere else?

The example with emoji is a good one. It shows where slow-moving standards bodies, even when they are successful, end up being corralled through standard participants. They are not overtly saying, “Here is what the standard should be,” but drive all of the other members along. That actually helps with standardization.

When you are talking about 3D objects, there is a large contingent who believe that the consumer-facing 3D objects are less important. Bringing your briefcase from one environment to another is less important than having the environment itself be useful or repurposable for more developers. As an example, take the investment that Disney has made into Hoth, and make that into a virtual biking course used by Peloton, a dating simulation on Tinder, or a theme park in Fortnite. That is probably more useful.

When it comes to your question of visual cohesion, it is not just a question of how you want to express it. What dimensions do you need? What pixel density do you have? The technology for machine learning, particularly from Intel, to up- and downscale is pretty strong. You can take a 2D object and 3D-ify it. You can say The Verge makes virtual shoes that don’t separate between the sole and the fabric, but our system can actually separate the two for different designs. A lot of that is going to be interpretive software that takes what is not standardized and modifies it.

I feel like this is beginning to unlock for me in an important way. Unreal has moved into Hollywood, and it has moved into cars. You see this graphical engine appear in more and more places where graphics need to be rendered. So The Mandalorian renders the background on giant LED monitors behind the actors in Unreal, and now that same virtual world is available for Peloton to say, “We are going to bike through this environment.” Is that somehow an open platform for that kind of development?

You are quite right. Let me frame it a slightly different way. Entertainment is such a good example. Disney will spend $100 million producing backdrops in virtual environments for a film. Those are essentially all deprecated. They are increasingly used for the next film, but that is about it. What does that mean?

Well, if Peloton wants to build a Star Wars biking sim, they need to build it all. The business case might not be there. In addition, Disney might say, “Well, we have to make the thing, then we have to brand approve the thing, so we need to charge a lot.” So a lot of this does not happen. Once you start to standardize these 3D assets, you start to say, “We have made this investment and now we can use it wherever we want, or at least more extensively without building it anew.”

You take that from consumer leisure to, “Well, Ford has dimensionalized its next Ford Escape, so now we can simulate it in other enterprise environments, such as a car park for parking simulations.” A Hummer vehicle can use its lidar sensors to map the local area, then you can pre-drive that environment, like you would in a video game, to make sure that you can make the path. Making all of this information more repurposable starts to have extreme combinatorial effects, either by making new creations easier or cheaper.

Who controls the access and the connections between those things in your view of the metaverse? That seems like a very powerful vision, but then I start to pull the thread. If Disney has rendered out the world of The Mandalorian, I’m like, “I want to make print versions of The Verge for The Mandalorian.” I can imagine all these things we could do, but it feels like I still have to go get permission. The asset may be cheaper, but over time, content creation gets cheaper and cheaper anyway. Where does the technical part of availability come from? That seems like the hardest problem that we have been talking about.

There is no simple answer. These environments are managed centrally, and their permissions are going to be managed deliberately to start. If we have learned anything from the era of Shutterstock, TurboSquid, Quixel, or 3D asset databases, it is that the most valuable stuff, the IP, is not easily or cheaply licensed.

This is where we get into one of those fundamental questions of decentralization versus centralization. There are good arguments to be made that the last 15 years were too centralized, because the internet protocol suite has too little in it. We can get into that one way or another, but there are many forms of centralization that have nothing to do with technology per se. Revenue leads to greater investment and better products. IP centralizes or drives habit and retention. Brand keeps people inside of a system that they trust more than another.

This is the case even if you believe that the metaverse is a big, disruptive, next-generation internet, or if you believe in the wide deployment of blockchain and Web3 to democratize more of the stack. OpenSea is a great example of how we may still end up with no technical barriers to switching, but enormous habit and brand-based, or IP-based, stickiness to a few.

I feel like we have arrived at the Web3 portion of the conversation, so let’s talk about it. The ideas are in parallel, right? The amount of Web3 hype that has happened over the past 18 months is right next to the amount of metaverse hype. It feels like everybody wants to conflate them for some reason. Certainly, it is trendy in the business world to conflate them, to juice your stock price in some insane way.

They are not necessarily connected, but it does feel like the game of, “What are some use cases for Web3?” is best answered by, “There will be scarce digital objects in the metaverse.” There is a connection there. The open, technical questions of how these 3D worlds might work and how you might transact in them are actually answered by the blockchain, by Web3 technologies. Do you see that connection as directly? Do you think it is just a quirk of timing? Do you think there are other possible solutions?

I think that there are a few different things that we can unpack here. First and foremost, I and others, like Mark Zuckerberg and Tim Sweeney, describe the metaverse as a successor state, or quasi-successor, to today’s internet. Web3 is so named because it succeeds Web2. If both things come after the current thing, it makes sense that you have conflation.

In addition, there is a good reason to believe that the philosophies at minimum, or perhaps the technology at maximum, of blockchain are essential or important to the metaverse. Which is to say, property rights are probably going to be important, as they are to most economies. The ability to tap into decentralized or wide networks of contributors to provide extra GPU cycles, broadband, or just time and assets, which are currently hard to accumulate from individuals — Patreon only scales so much —are good reasons to believe that it is important to have a thriving metaverse, one that we want rather than one that is just technically possible.

I understand why the two are conflated, but I would say that they are separate. When you are talking about a good technological solution, when you talk about interoperability, you need a standard. You need someone to effectively take custody of an object and you need everyone to agree that they trust it.

The big problem that we have right now is EA and Activision do not have a good system to exchange anything. They certainly do not want to use one another’s new thing, should it exist. When other aggregators like Steam have tried in the past, no one opts in because the platform is already powerful enough.

Irrespective of whether or not blockchains are actually the ideal solution, they clearly have some revenue attached, speculative or not. They are proving themselves to get a wide collection of different deployed solutions. At the end of the day, it is not always important whether something is perfect, insofar as whether or not everyone uses it. The GIF file format is awful. We have known that for decades and yet everyone uses it, and so that ends up being the thing. That to me is part of the case.

One of the very hard problems with all of this is the amount of compute that is required. We are going to render a bunch of persistent virtual worlds that have unlimited maximum capacity, then potentially we are going to run blockchains to manage scarce digital goods inside those virtual worlds. That is a lot of compute; it is more compute than we have right now. Do you see that coming down because of Moore’s Law? Is TSMC going to figure out the next process node and we are just going to get there? Is it an agglomeration of other kinds of compute? Who builds this stuff? Where does it come from?

There are three dominant theories here. One is just Moore’s Law, slowing or not, continues to improve, and as part of that we get better at compression. We start to prune out the inelegant data formats and architectures, just like moving off of GIF to MP4 for lighter performance.

The second school is really organized around more efficient resourcing. This is the cloud argument. There are problems with it, but the argument would basically be that it is kind of stupid that we put the most intensive computing at the individual user, whose device has to be affordable, lightweight, and replaced every two to three years, versus the power plant approach of saying, “No one should have a generator in their home. We should deliver it from industrial scale.”

Then third are the bigger punts. There is a large contingent of people, Intel or TSMC, who are starting to believe that quantum computing — another idea that has long been considered fanciful — is no longer a crazy thing to believe in and ends up being essential.

The last and the most fun is decentralized computing, not necessarily in the blockchain sense, but in the solar panel sense. I am sitting talking to you right now. I have two consoles with incredible GPUs both sitting unused. There may be someone in my building right now who could use that. Right now they either do not have it, or they need to rent it from a data center that is expensive and far away, thus producing latency. Do you have a model potentially on blockchain or not? Is that a more effective system of renting out excess capacity, like a solar panel, or like Elon imagines Teslas will do in a self-driving car?

I love that idea and have heard variations of it for a decade now. I used to run SETI @ home on the computers at the college computer lab that I managed.

It is in my book. It is so fun.

It’s all right there. We have been chasing it for a minute. That requires that your personal power bill might go up and down in ways that you cannot predict. Your bandwidth might get strained in a way that you cannot predict. It would be sad if right now our call was diminished in quality because someone was running the GPU in your PS5 at 100 percent.

On top of that, at least in this country, the bandwidth required to do that is actually not evenly or equitably distributed. Some people have really fast connections, and many people have bad connections. There is virtually no competition for those connections whatsoever. You can make that bet, but you think about how it would play out in practice and it just feels like a lot of people will be selfish, first of all. That seems like a thing you can count on. Then second, the infrastructure to actually pull that off does not really exist.

I agree with you. I characterize it as the fun one because it remains the elusive one, just like when we talk about peer-to-peer servers for multiplayer games. It is a fun idea, but no one has figured out how to do it.

There are some technical solutions, of course, one of which could be that you do not necessarily need to congest the neighborhood if you geographically constrain who your GPUs are available to. You can also have different bidding. One of the problems I talk about in the book is the fact that we actually have very poor systems in TCP/IP to manage the prioritization of traffic once it leaves our network. I am not talking about paid peering or net neutrality, but literally the ability to differentiate if it needs to be there in 10 milliseconds or 50 milliseconds.

These are actually more fundamental issues. We do not have an effective way to split GPUs. It is not like you can say, “I need 80 percent of it but the remaining 20 percent can go.” I will say that there are some systems for this that are being deployed. J.J. Abrams and Ari Emanuel are on the board of a company called Otoy. They have a blockchain-based system called the Render Network, and it is designed to do exactly that.

An architectural firm that perhaps does not need its high-end GPUs overnight can rent those out on a bid-ask, blockchain-based system, and Hollywood studios do use them. This is not the expectation of every single person’s sitting devices used minute to minute, but we are starting to see it work on a more regular basis for industrial use cases with high-end, low-supply hardware. I put this in the, “if you woke up in 2045, it might be answered” bucket.

Let’s wrap up here by talking about the companies that are building this stuff now and where they are. You run an ETF called Meta that invests in various metaverse companies, and you obviously pay very close attention. Let’s start with the obvious candidate here, Meta.

In your book, you call it Facebook, because it is too confusing to call Meta “Meta” in a book with a metaverse, which I appreciate. Facebook obviously rebranded itself to Meta, Zuckerberg is all in on this pivot to the metaverse. In VR headsets at least, they are the market leader; the Quest 2 is a really good consumer product. Though I do not know if it is a metaverse product, since it is a pretty closed system. But they are ahead. How do you think they are doing and where do you think they go next?

I would say that the Oculus device is actually pretty open. They support side loading, and they do not require a central identity system. You can use alternative payment solutions for side-loaded apps, which is not even side loading, it is just not app store direct. The Oculus is unique in that it is effectively the only mainstream console that uses open standard rendering collections, WebGL, OpenGL, WebXR. Those are pretty significant. No one else did it. PlayStation 3 did, but PlayStation has never done it since.

The truth is, if you were to talk about number of users, amount of spend, number of developers, amount of developer profits, and cultural impact, they are frankly nowhere near leads like Roblox, Fortnite, Minecraft, or Unity on the B2B side. They also have a much harder path to doing that.

One of the challenges Facebook has in particular is that the economy is slowing down. Apple’s ad changes have had huge effects on Facebook’s revenue. They are trying to manage this big pivot and this bet on the future. People might buy fewer Quest consoles, and they are investing less in future hardware. Do you think they are going to be able to make it through?

The AT&T shift from Apple is particularly brutal. The estimated cost of that is $10 billion in operating cash flow in 2022. That happens to be exactly what Facebook Reality Labs was spending on their many projects, the various XR devices, the wearables, their operating system, and the Horizon Worlds platform. Anyone finding out that they are going to have $10 billion less in cash flow is going to have to trim budgets, especially in special projects with limited revenue and probably a negative 80 percent gross margin overall.

I think the biggest challenge — one that Mark has consistently underestimated, it seems — is that the timeline for those new devices, that would allow him to get out from the hegemony of Apple and Google, is probably farther out than was ever imagined.

2015 was the first time Mark said publicly that they imagined by the end of the decade, last decade, wearable headsets would replace the smartphone. They have reiterated that this decade, but as you and your colleagues have reported, they have now delayed the first edition three times. We may not see consumer AR hardware until 2025 or 2026, and he has called it the hardest technological challenge of our era, putting a supercomputer into lightweight wearables.

If that is their biggest opportunity to have hardware, to have their own operating system, and they are already sitting behind when it comes to what I call integrated virtual world platforms — Horizon versus Roblox or Fortnite Creative Mode — and they are simultaneously experiencing decline, not necessarily secular, of the core business, the timing starts to feel tight.

You said that it is the hardest technological challenge. I always think about it as a stack of problems, especially for AR glasses. You need a camera that can see the world around you in sufficient fidelity. That has to go to a processor that can interpret that data and spit out something good to put over top of it to augment reality. You need a battery that can power that processor and that camera. You almost certainly need persistent connectivity. Then most importantly, you need a display solution that actually works, which does not exist yet. Do you think Facebook is on the road to solving any or all of those problems?

I would add two more problems. It has to actually fit and weigh little enough that you are comfortable wearing it, and it has to not melt your face while you do it. Every single thing that you just mentioned trades off with one another. You want another two sensors that are good for UIX, it drains the battery and the GPU power, increasing the cost and the form factor, generating more heat.

Put another way, we take for granted that today’s most computationally powerful consumer devices, consoles, really just need to manage for a few constraints. The size, not really; the new PlayStations are four times bigger than the first PlayStation. They do not need to manage the battery, as they have constant access to power. They can put fans in there so that the overheating problem is not that bad. And they know that the build of materials has to cost between $400 and $700. When you are talking about these devices, you have several new problems: size, heat, you cannot have a fan, you need battery power, and the GPUs are smaller. All of the other things get harder despite that.

We see that Facebook is investing in its own semis and you are right, it’s the stack. All of these things need to be solved. We know that Apple is planning up to 12 or 14 cameras. I think the current Oculus has 6. Well, maybe you do need 12 or 14. Every time you put another pair in there, you are going to find that the GPU you thought was going to power experience X just cannot. It is incredibly hard.

I think that set of challenges is very difficult for Facebook. When we talk about hardware, we have to go to Apple next, which is very good at hardware. They are very good at performance chips that run a long time on batteries. There are lots of rumors about Apple’s headset out there.

But they are pretty bad at ecosystems and playing nice with others, and with interoperability. As you have mentioned with their ad tracking stuff, they are pretty good at locking things down. They are good at preventing innovation from taking place; game streaming does not exist the way it could because Apple will not allow it on their platforms. OpenSea cannot transact NFTs because they would have to pay Apple a 30 percent cut. How do you think Apple is doing?

One thing that is fun to put on the side of this is that six days before Epic Games sued Apple, Tim Sweeney, the founder and CEO, tweeted out that Apple had outlawed the metaverse. His point was exactly the cloud gaming one. I cite The Verge a few times in there with these fun quotes that basically say, “Arguing about what Apple does or does not allow is irrelevant because they can change the rules any time they want.”

The Apple constraint here is really profound. They have incredible hard, soft, and often accidental power, and they do work hard to prevent many standards and solutions coming into place.

You just teed up my favorite example, which is what happens with NFTs. Let’s keep in mind that they allow you to buy fungible tokens, ETH, on Coinbase, but you cannot buy a non-fungible token, an NFT, on Coinbase. If you choose to fractionalize an NFT into a billion fungible tokens — you could actually increase it so that there are more fractionalized tokens than there are Bitcoin tokens — that is still not allowed, even though you might own one trillionth of an NFT.

This just reflects the extent to which they are contending with not just business model disruption, but control of their own ecosystem. Outlawing is not wrong, but I think we will see how that turns out. When it comes to new hardware, it is obvious. If AR and VR are going to be things, Apple will be at least a player, but it is more likely that they have the most performance, best-looking, lightest weight, and preferred early additions. The advantages there, especially at scale and cost — development cost or production cost — are simple.

In the book you have a section about how the metaverse need not actually take place in headsets. It could be expressed in all kinds of ways. As we talk about these companies, their metaverse bets are very much headsets.

Facebook wants to be first to headsets at scale, because then they can just leave the iPhone and the complications of Apple’s platform behind. Apple does not want to have the iPhone disrupted, so they are racing towards a headset. I think Tim Cook wants to shift the AR headset as his last big reveal before he moves on in 10 years. Right now, to do a non-headset metaverse, you are kind of stuck behind whatever Apple will allow, because they are the most pervasive computing platform that exists.

That is quite right.

Is there a way around that? Do we just hope Amy Klobuchar can find the votes for her anti-trust bill, or is there a business model or industry solution that solves that?

This is where we get into some of the interesting answers. Is there a way around it? Are there alternatives? Yes and no.

Cloud gaming is a potential answer, but we should keep in mind exactly how many ways Apple stymies them. It probably works 95 percent of the time for 40 percent of users. That is not a good technical solution for a social platform, but it can work. Doing it from the browser is not a great experience. Apple, for security reasons, valid and not valid, also constrains your ability to send notifications. That is not great if I am trying to tell you to log onto Fortnite. First of all, you cannot have an app, and secondly, you don’t ever get the notification.

The other way to do it is browser-based rendering, but Apple has historically constrained WebGL, so the non-application alternative of using a browser, what they call the open web, doesn’t really work. The way in which Apple constrains WebGL is because Safari does not support it comprehensively, whereas I can download Chrome for iOS, and I am really just using the Chrome wrapper on the Safari engine. Their technical decisions for Safari mean what Google can and cannot do is inherited, and the app stores hegemony over software means that I cannot download true Chrome.

We are finding out this is why Tim sued Apple. He says that Apple has outlawed the metaverse rather than gotten in its way. A properly motivated Apple can effectively stymie most things. There is a reason why Web3 games are either based on non-real-time collecting and trading, or really primitive browser-based games, like Axie Infinity visually. You cannot pull off complex rendering without most of WebGL or a native app, and Apple will not allow it.

You mention the open web, which means we should talk about Google next. Google is Google. They have multiple competing projects. They have just restructured some things, and they have announced some little things. Are they a player?

That is a great question. Google has spent quite some time focused here. Google Glass was a famous disaster, but they have released another two versions of Google Glass, or enterprise editions. They made a billion-dollar acquisition last year, a $200 million acquisition the year before. Clay Bavor, an SVP in charge of essentially all special projects, plus AR and VR, and has been for some years, was realigned to directly report to Sundar.

It is clear that they are focused here. The problems have always been that their software is never considered best for consumer applications, their hardware has never really taken off, and their efforts in gaming have barely been funded. Many of their best potential plays, Niantic and others, were divested, spun off, or allowed to competitors.

If Android is and remains the most used ecosystem globally — it is the second highest revenue-generating games platform globally — they are likely to benefit, but the big opportunities with new hardware, a virtual world platform, or managing the standards, all seem tough. Even when you take a look at Google Cloud, it is estimated to be losing $5 or $6 billion per year. AWS has more profit than Google Cloud does in revenue. Even with the tangential argument that increased computing power is going to be good for Google, their business currently loses money every time a new server gets stood up. They are harder to see.

You mentioned AWS, so let’s keep going down the list. Amazon has some pretensions here, in the sense that they have a big hardware division that invents a bunch of stuff all the time. They have the most pervasive voice assistant, which I think is an interesting side light into the idea of a secondary world that you can interact with in different ways. Are they a player? Do you see them making an investment?

I would guess that they are number one in virtual assistant hardware, but I would also guess that Siri and Google Assistant are the most-used virtual assistants. They have the other benefit of having the device everywhere; mobile is better tailored.

Amazon is really interesting. The computing and data center business is going to be an extraordinary beneficiary. How much that moves into value-added services in machine learning and others has yet to be known. Snowflake is a good example of other companies building value-added services on top of the pure racks.

The bigger challenge is one I find really interesting. Amazon has spent a lot of time focused on more traditional media categories than it has in gaming or interactive, even though the latter seems a lot closer to their core business on the AWS side, and their success rate has been mixed. Jason Schreier at Bloomberg has estimated billions were spent into Lumberyard, their game engine. That was given over to the Linux Foundation earlier this year. Luna, their cloud gaming service, seems to have had less of an impact than Google Stadia did.

That is a very quiet burn. I just want to put that out there.

There is a good question of whether or not it is a quiet burn because they have been a lot quieter as well. Part of the problem that doomed Stadia was much bigger and more public ambitions, and much greater out-of-the-gate spend. Amazon is best in the world at the slow burn strategy and they remain committed to it, though I have not seen any big leaps.

While Amazon Game Studio has had some success with New World and others more recently, it is operating as the publisher. They are not developing the titles themselves, and they are not using AWS in an innovative or new way. As you take a look at Amazon’s interactive business, they have rewritten many job descriptions to focus on the metaverse in name. They are a big proponent of the Unreal ecosystem. They are trying to advance certain standards. But externally a lot of it still feels like more potential and conjecture than it is, as yet, a product.

I want to ask about two more here. Microsoft CEO Satya Nadella has said the metaverse is already here, so he is buying Activision and the Xbox seems to be growing. They just keep buying everything, but they do not have great hardware. The HoloLens is not a huge success; they just shuffled that team and fired Alex Kipman, who was in charge of the HoloLens. Are they on track, or are they just going to be a horizontal software provider, which has been an enormously successful strategy for them as you pointed out?

I talk about this quite a bit in the book. There is this fascinating aspect in which the company has absolutely thrived under Satya by becoming horizontal, shedding the stack requirement and rich vertical integration.

But when Satya took over, the games business was being called on for divestment. Yet the first acquisition he did was of Minecraft. He did something really unique at the time; he committed to keeping it fully horizontal, available on all platforms, not exclusive to Xbox, and keeping it agnostic to the end point, not even preferring Xbox hardware.

It was about five or six years before he did another large acquisition, that of LinkedIn. Then you have Activision Blizzard, the most expensive big-tech acquisition in history, at $75 billion. In the opening graph, the last line, he says, “It is for the foundations of the metaverse.”

In many ways, Minecraft presaged everything that he was going to do with the strategy at large, and they have been very focused here. The number of different pieces they have is actually really exciting. I talk about Microsoft Flight Simulator as perhaps the most technically impressive consumer-deployed, persistent live digital twin or metaverse-style experience that any of us can do.

This is a company where, putting aside the fact they were public about the metaverse before Facebook was, it feels like execution of bringing the pieces together — which is the same for Google and Amazon, but less clear — could be extraordinary for them. I think that is why you have always seen this commitment, and why he is so quick to bet FTC scrutiny, DOJ scrutiny, and $75 billion to build it.

I could keep doing companies forever. It’s a fun game, but I want to actually end on the regulatory scrutiny piece.

This space is unregulated, in a way that if you make the comparison to the early internet, it is very different. The early internet was a government project. There was the idea that we would keep regulators away from it. Even that decision to keep regulators away is, itself, a regulatory decision, and then you had all of the public investment into the internet around the world.

That is not happening here, right? This is all a purely private company kind of investment. Regulators seem like they have no idea what to do here, in the same way that even regulators have no idea what to do with crypto, but they have a lot of ideas. Here it is just silence. Where do you think that comes into play? Where do you think the government comes into play here with the metaverse?

The interesting thing about regulators leaving their hands off the internet is, of course, that the internet came from government. Many of its foundational bodies, the internet engineering task force that stewards most of TCP/IP, was developed by DOD and then relinquished, but is still strongly influenced by government. One of the reasons why governments left it was because there were pretty strong and important self-regulating bodies that worked together effectively that they had helped to create.

You are right that we do not see this here, but I actually think it is changing pretty quickly. Yesterday the EU released their Think Tank’s Policy Memorandum. The chief negotiator of the EU for the Digital Services Act has been very critical and very vocal about what they need. The South Korean government has established the South Korean Metaverse Alliance, an effectively required body that is also effectively mandating national standards.

Their perspective seems to be that the standards group will force things that many do not want and are individually disadvantaged by, but to the national benefit. Of course in China, which is a whole other issue, I do not think it is a coincidence that just after Tencent unveiled its Hyper Digital Reality vision — which is their essential trademark for the metaverse — they began the biggest ever crackdown of the space.

I think the US is probably the furthest behind, in at least formal recommendations. I think that in many territories — Southeast Asia, China, and the EU — governments seem very focused on this now in a way that surprises and inspires me. The fact that it coincides with regulation designed to fix the problems of the past 15 years raises the specter of accidental damage to an area that does not really exist yet. I am more hopeful that it actually sets us on a clearer path, rather than 15 years of catch-up.

Let’s end with a look to the future. I think one of the things that you and I would both agree on is that this is not going to be a light switch. The metaverse is not going to just turn on one day; it is going to happen to us slowly over time. I am curious. In that big picture, what is the sign post for you that the metaverse is more likely than not, or that it has arrived in a real way? What would be the indicator for you?

The indicator that I would pay attention to is the early demographic transition. Seventy-five percent of those ages 9 to 12 in most Western markets use Roblox, and just Roblox, on a regular basis. That is not to say that they do not use other things. We know fundamentally that Gen Y games more than Gen X, Gen Z more than Gen Y, and Gen A more than Gen Z, and that trend is not turning around.

I think the big things that I am getting excited about are the industrial applications, the deployment in what we call ACE — architecture, construction, and engineering. The challenge with those is that lead times are long. You have to convince businesses to use new technology to solve problems they are not used to solving. They have to then deploy them, and they have to get good at using them. They need to start to share with the city and with other partners.

Once we actually find a way to make development of the real world more productive, to live-operate businesses and infrastructure together — which can be as simple as lighting systems in a smart city with proper civil engineering — that is what gets exciting to me.

Matt, this has been incredible. I could keep going for another hour. Thank you so much for being on Decoder.

Thank you.

(Except for the headline, this story has not been edited by Leader Desk Team and is published from a syndicated feed.)