Navigating the Metaverse with your AI Concierge

Navigating the Metaverse with your AI Concierge

This article is the fourth in a series focusing on the Architecture of the Metaverse. ArchDaily has collaborated with John Marx, AIA, the founding design principal and Chief Artistic Officer of Form4 Architecture, to bring you monthly articles that seek to define the Metaverse, convey the potential of this new realm as well as understand its constraints.

Of all the attributes that will define the Metaverse the single most important is that of experience. As we move more deeply into the Anthropocene Era humans seem to be shifting their interests from collecting things to collecting experiences. As the demand for experiences grows more intense and detailed, the need for content, and the clever and effective use of that content, will rise exponentially.  From a more detailed perspective, the management and quality of those experiences will determine the initial success of the Metaverse. This is where the concept of a Responsive AI Concierge comes into play.

These experiences will have a large measure of curation in them. When you walk down a quiet side street, there most likely will be an experience waiting for you, whether it is a building, a flower, a spontaneous band of mice playing Mozart, or a portal to another experience ….  the Metaverse is not a static space. It will constantly evolve and shift in concert with the users who are present. This level of curation may be quite passive and barely noticeable, or highly scripted like a traditional bus tour. It may be predictable, as in searching for an art museum that has existed in a particular location for decades, or completely unpredictable, as in a small pop-up museum whose entry point not only exists in 2,000 places simultaneously but also may only present itself to you for ten minutes and then disappear. Managing these experiences on the scale of the Metaverse is a daunting task well beyond the efficacy of human involvement.

The following story illustrates a vision of the metaverse, through a prototype we call the Icon, where large structures provide a context for an overlapping of physical and virtual experiences. Where you can meet other people in real time and space, but where, integral to your visit, there is also a digital twin where you can enjoy this experience with others anywhere in the world.

Imagine …. walking into a vast and intriguing indoor space.  This is a space that seems almost limitless in geography, and its capacity to draw you in is powerful and compelling. There is an experience awaiting you …….. well, in fact, there are multiple experiences waiting for you.

The trip had been arranged a year ago, you bought tickets to The Icon and selected the aspirational theme of Intergalactic Exploration. There was an extensive questionnaire that probed deeply into what your goals and expectations were for your visit ….. it even asked about what your favorite colors are and your taste in music.   There was an option to let the system review your online history to get a sense of how you navigate the world and what you care about.  You were able to control how intrusive the system could be in order to balance your privacy with your desire for a personalized experience.

You were staying for a week and did not quite know what to expect.  The images of other people’s experiences seemed out of this world as if they had actually traveled in space to some distant world, or in your case worlds.

Over the span of a few months, you were tempted by “enhancing” your experience.  First, with a series of virtual background visits to some of the places you might see, some “real” like Mars, some completely imaginary.  After that, you were enticed to make it more physical.  There was a multitude of accessory options for purchase through the Portal’s online store, but eventually, you decided to buy a week’s worth of outfits, which included an advanced set of AR glasses, boots, and individualized/creative spacesuits to walk on foreign planets.

The particular experience you selected is participatory and curated, it is not simply a case of showing up at an analog amusement park and standing in long lines for a few rides.  From the moment you pass over the threshold of the themed plane that takes you to this wonderment of technology, you are immersed in an experience. At the boarding station, you changed into your first outfit. Not all of the available experiences invite you to arrive in customized outfits, but this week everyone seems to be arrayed in dazzlingly imaginative variations of space travel, some have changed the color of their hair or altered other body parts.

20,000 people arrive every day at The Icon.  They come for an experience, and most of these experiences are curated.  It is a daunting task to orchestrate 20,000 experiences a day which vary in length and intensity.  It is an effort that if attempted by humans, would be akin to 20,000 planes landing at an airport all within an hour of each other. 

In order to provide the highest level of user experience, in a part virtual / part real environment, a project like The Icon could be managed by a Responsive AI Concierge. It learns about you from the moment you first log on to an online portal to inquire about its existence. For those people who want to engage in a fully or semi-fully curated experience, there is a need to gather an appropriately sized amount of information from each user.

In our example, there are eight steps that the Responsive AI Concierge goes through to create a context for your experience:

1. Visitors – First determine the frequency and broad type of visitors. Are you coming as an individual, couple, family, or group of friends? Are you a gamer, or an artist, are you visiting once, or a recurring visitor?
2. User Profiles – Develop a comprehensive background based on preferences and interests. Of special relevance is what kind of personality each user wishes to present as … an explorer, an artist, or a scholar? In some ways, this could be seen as selecting what kind of character you want to play in a movie, but it sets the stage for the kind of experience you want to have.
3. User Need States: This aspect determines how active or passive the user wants to be in the experience. Is the user looking for a relaxing and quiet experience or an intense thrill-seeking experience?
4. Macro Themes: These are the narrative overlays that give the experience a specific theme. The range of themes is only limited by human or AI imagination. These might be an existing theme from the library or a customized theme. Imagine your children customizing a theme around their parent’s anniversary.
5. Content Sources: Content can come from a variety of sources, but three categories stand out. Branded partners are long-term relationships with well-known and established organizations.  Companies like Disney, Netflix, or Epic Games might become content providers, which create their own attractions.  Commissioned content might come from independent artists with specific intentions relative to unique narrative themes. Finally, users may create content during an onsite experience that is shared with other users in real-time or recorded.
6. Lenses: This is a dynamic between Immersion and Absorption. Immersion relates to being fully engaged in physical or mental activities such as world-building that involve the creation or production of new content. Immersion can create a sense of an intense relationship and a sense of belonging to a community of users. Absorption relates to learning from a distance, where the user builds an understanding of a subject without actively engaging with others from the community.
7. Narrative Types: Each user will want different levels of curation and scripting in their experiences. This will range from non-curated wandering to simple linear scheduling, to a branching experience, where at multiple points the user is presented with options to do different things, making a choice will then trigger additional option points. A fully interactive narrative type will allow the user to navigate freely in a seemingly organic fashion, while the AI Concierge will continuously tempt the user with unique experiences.
8. Experience Options: Ultimately the AI Concierge creates a set of specific experiences tailored to each user or group of users.

The Icon does not exist yet, but it dramatically extends the prototype of a traditional theme park. Theme parks around the world are based on relatively nonparticipatory entertainment experiences.  While they can be extraordinarily engaging, they lack the opportunity for a uniquely customized experience, and cannot adjust themselves in real time to the needs of their users.  The metaverse will be highly interactive and participatory, it will invite you to immerse yourself fully into a physical/virtual environment.

Always watching … always there for you …. is it intrusive spying or is it a clever mirror with an economic incentive. The precise balance of these two forces should be up to the user.

We are entering into a new realm of experience design and curation.  In many ways, these new levels of customization offer an unprecedented opportunity to change the nature of learning, entertainment, and leisure.  We will need to set boundaries and conditions for privacy, safety, and control, as well as, provide an open platform for innovation and immersive and imaginative user-based experiences.  The future is ours to imagine, with the help of an attentive AI Concierge.

“Navigating the Metaverse with your AI Concierge” is written by architect John Marx, AIA, the founding design principal and Chief Artistic Officer of Form4 Architecture, an award-winning San Francisco-based firm that designs prominent buildings, campuses, and interiors for Bay Area tech companies such as Google and Facebook, laboratories for life-science clients, and workplaces for numerous other companies. In 2000-2007, Marx taught a course on the topic of placemaking in cyberspace at the University of California, Berkley, and in 2020 he designed his first project in the Metaverse for Burning Man: The Museum of No Spectators. The following year, John Marx led a design team charged with creating a $500B portal to the Metaverse.

Editor’s Note: This article was originally published on May 25, 2023.