AI, Mixed Reality, and Clipper Ships

How should industry disruptors be incorporated into our designs?

Rob Badenoch

September 14, 2023

It’s been a few months since Infocomm 2023, and we’re still buzzing over the impressive showing, with strong industry attendance and some fascinating new developments.  As usual there was much on display, and ample opportunity to connect with colleagues and clients alike. TAD was present in force, conducting tours, seeing new faces, and gathering insights into what the future may hold.

Each year seems to have its own, unofficial theme: Digital Experience. Immersive Design. Hybrid Work. But this trip I came away thinking of something completely different: Clipper Ships. Sleek sailing vessels which were popular during the end of the 19th century, then rendered obsolete. What on earth do clipper ships have to do with the state of our industry in 2023? More on that in a minute.

“With such wide adoption of remote and hybrid work, it now feels like we are on the cusp of another big shift in the industry."
Rob Badenoch
TAD Partner

As consultants whose work is often long-standing, we must consider factors that will affect our industry and our clients both now and in the future. There is always risk that decisions made today will be jeopardized by an unforeseen development in the wider world. Changes in markets, culture, and user expectations happen often as a result of global forces, and it’s crucial that we step beyond the bounds of our (not so) small AV industry to consider them.

Sometimes it is the unseen element that ultimately proves to be the next big thing which transforms our lives.  Many of us remember our first time seeing Zoom in a tiny booth at the outskirts of industry shows well over a decade ago.  Few could predict the extent to which they would become a transformative force in the world.  It is from this perspective that we take careful note of the fringes and the outliers.   

Sometimes shifts in our Industry are caused by external events.  An obvious example: COVID’s impact on workplace has accelerated the evolution of collaboration technology, vastly increasing adoption of videoconference systems.  Another example: The climate crisis has propelled the adoption of smartbuilding strategies, as municipal sustainability regulations motivate our clients to seek data analytics driven by IoT sensor systems. 

It is not all about transformation, however.  At Infocomm this year we’ve seen improvements, both incremental and substantial, in many product categories.  Among them:

We are excited by the new advancements in immersive audio tech.  With increasing adoption of Dolby Atmos in both video production and music, we have opportunities to include wavefield synthesis and 3D audio pattern control in our projects.  The products and applications available from Holoplot are of particular interest.  These tools are also applicable in business-oriented collaboration environments, solving for acoustically challenging spaces in unique ways.  

The new paradigm of hybrid has placed emphasis on corporate studios, as our clients look to bolster their communications through high quality media.  We are including virtual production environments in some facilities, which expand beyond big screens for studios to also include authoring environments and creative development workflows for technical staff.  These include new applications for hybrid presentation, with platforms such as Immersive Canvas, which uses Unreal Engine 5 to create spaces that cross between live in-audience and virtual remote participation.  

A client's high-quality media moment, to be used for events and hybrid meetings.

And then there is AI. Truly, this was an important theme at Infocomm this year.  Applied to camera tracking systems, AI now does much more than machine vision and facial recognition. This year at Sony we saw it performing combined camera-follow using face, clothing, and skeletal features, greatly increasing reliability and accuracy with moving performers. Applied to audio, we learned of AI algorithms in development at Microsoft which go beyond noise cancellation to recognize individual voices. Soon this may allow Teams Rooms to selectively amplify a talker while cancelling the voices around them. (And for those who opt in, identifying the talker on screen based on their voice as heard in a room.)  Perhaps one of the more compelling uses of AI was in the detection and selection of audience reactions in a videoconference.  Huddly had this on display with their Crew camera system, demonstrating reliable switching and framing based on facial activity not just of the talkers, but also of those who showed reactions to what was being said.

 

These applications of AI make improvements and refinements to existing technologies such as cameras and loudspeakers.  This is quite useful in our project work, but not quite to the level of industry-wide disruption. 

 

Outside of the Infocomm show, Apple’s entry into the mixed reality (MR) space, the Vision Pro, has prompted some interest.  By projecting a virtual screen and, more importantly, providing a comprehensive MR development platform for third parties, Apple seeks to completely upend our relationship to displays, room audio systems, and in-person interaction.  If successful, we can safely call that a disruption.  But will Apple be successful?  They certainly have a track record of identifying already-existing technologies, bringing their own version to market, and then disrupting the world with their version.  This has been evident with the Macintosh, iPod, and iPhone: all products brought to a market where prior versions already existed.  (The home PC, pocket MP3 player, and Blackberry, respectively.)  Will the same be true of the Vision Pro, coming to a market that already has VR via Oculus, and MR via Microsoft Hololens?  Time will tell, but until then it’s interesting to consider that our role as designers of physical spaces could eventually be turned on its ear.

A demonstration comparing camera tracking systems. Can you tell which makes the best use of AI?

All of this brings me back to clipper ships. When searching for the next big thing in technology, thinking on what global event or upstart new tech will disrupt our industry, I’m reminded of these sleek sailing vessels that were refinements of existing technology.  Sharp lines, multiple masts, and innovative hull design led to the fastest ships ever constructed… right before the arrival of steam power rendered them completely obsolete. 

What capital planning, architectural design, and consulting decisions are being made today that are in jeopardy of being obsoleted by an unknowable paradigm shift?  As manufacturers continue to refine the next generation of AV products, we look to technology trends, world events, and developments outside our industry for clues.  Mixed reality platforms like the Vision Pro strive to radically alter the experience of hybrid collaboration and communication. If there is a nonzero chance that Apple could be successful at this, how should we adjust our thinking towards design of spaces for work and play?  It is our role to advise and design for our clients current needs, while anticipating what’s next to come.  Otherwise, we risk building a clipper ship, providing impressive sailing performance that is made irrelevant by the arrival of a new paradigm.

At TAD, we delight in these challenges, seeking the right compromise between an ambitious vision for the future, versus the pragmatic needs of the present.  Infocomm 2023 has left us more excited than ever at what’s to come.