Visualizing the soundscape of Dolby Atmos
- Client
- Dolby Laboratories
- Partners
- LudoWic
- STMPD Studios
- JUR
- Deliverables
- Digital Installation
- Event installation
- Generative 3D visualization tool
The challenge
High-quality audio is essential in music as it enhances emotional impact, clarity, and detail, allowing listeners to fully connect with the artistry and nuances of a performance. Dolby Laboratories have created the surround sound technology Dolby Atmos, reinventing how entertainment is created and experienced. Dolby Atmos allows creatives everywhere to place each sound exactly where they want it to go, for a more realistic and immersive audio experience.
Dolby partnered with recording electronic artist LudoWic to give a live performance as part of Amsterdam Dance Event 2024, and asked CLEVER°FRANKE to create visuals to go along with the musical performance. The robust sound quality provided by Dolby Atmos inspired us to design a visual element to fully engage the audience in this event, reflecting the spatial and temporal dimensions of soundscapes and elevating the live music experience.
Value delivered
With this project we wanted to explore how data can play a role in generating creative art. Leveraging the latest audio technology we created a multi-sensory experience, seamlessly integrating LudoWic’s music with visuals that respond to Dolby Atmos’ spatial audio capabilities. Using 32 screens and TouchDesigner software, we created an immersive backdrop to this unique and once-in-a-lifetime performance.
Background
Dolby Atmos delivers a fully immersive sound experience by positioning audio in a three-dimensional space, allowing sound to move freely around the audience and creating a sense of depth that results in a more immersive experience.
Dolby partered with Dutch recording artist LudoWic for a live performance in Dolby Atmos sound during the yearly electronic music festival ADE (Amsterdam Dance Event) 2024.
CLEVER°FRANKE was asked to design visuals for this musical performance, transforming the Dolby Atmos sound into a fully immersive, multi-sensory journey that transports the audience to a new and unique soundspace.
Phases and toolkit items used
Solution
We generated visuals on screens around the stage to support LudoWic’s performance and provide a visual representation of Dolby Atmos’s extensive audio data.
The visuals allow the audience to experience sound as a physical space that we can all intuitively feel, yet rarely get to see. Using the music of LudoWic as a guide, we visit a new and unfamiliar place that’s constructed and deconstructed in real time, reminding us of how audio is tangible and interacts with its physical environment.
DolbyAtmos
Dolby Atmos sound has immersive depth, clarity and details that other forms of sound lack. Unlike speaker outputs like mono, stereo or surround sound, Dolby Atmos introduces height channels to an audio mix — speakers can be placed not just in left, right, center and back, but anywhere within a three-dimensional space. This offers a dynamic environment for movement and depth, allowing us to experience sound through speakers the way we would in the real world.
Dolbyaudioobjects
Dolby audio objects are the fundamental building blocks of a three-dimensional sound experience in Dolby Atmos. Each object carries its own audio data along with precise metadata that determines its position and movement in a 3D space.
In practice, each object is rendered in real time by the Dolby Atmos system to adapt to the specific setup of speakers in a theater, home system, or even headphones. This dynamic rendering ensures that the experience is consistent, spatially accurate, and engaging, no matter how many or what type of speakers are used.
Soundasspace
We created dynamic, real-time visuals influenced by the music in LudoWic’s live performance to represent the unseen audio objects. The Dolby Atmos aspect of the performance contains multiple audio objects with pre-defined positions and movements around the venue, creating a vast soundscape. During the performance, audio is sent through these fixed channels, with certain instruments anchored to their physical locations to reduce cognitive dissonance.
To highlight the three-dimensionality of the sound, audio objects are presented in real time via circular light sources that move through the space. Their intensity and diameter are dictated by the volume emitted from each active audio object.
Immersivescreensetup
This three-dimensional soundfield, populated with audio objects, was conceptualized as a physical world typically heard but rarely seen. To give the audience an understanding of this Dolby world, a specific screen setup behind and around the stage was devised: by placing 32 screens of varying sizes, we created windows into this invisible soundfield, where the audience could see generated visuals that complemented the musical performance and enhanced the three-dimensionality of the audio.
Expansivevisuals
The visual style not only captures the essence of the music, but also transcribes sound parameters in a way where the audience can see and not just hear the music.
We opted to utilize flat shapes in multiple overlapping layers to create a sense of depth. This approach to what could feel like an infinite space challenged audience perception, creating a distinct and immersive sensorial experience.
Texturedsonics
We crafted a visual framework that merged the essence of LudoWic’s music with the power of Dolby Atmos.
LudoWic’s music blends chaos and order, where noise becomes texture and disintegrating sounds form rhythmic, harmonic layers. Analog imperfections add life to his compositions.
Dolby Atmos introduces infinite sounds within a finite space, with some appearing momentarily and others drifting in from afar. As we navigate this sonic landscape, we discover new, unexplored dimensions.
Storytelling
The performance included six individual tracks, all with their unique visualizations. We based the visuals on the sonic identity of the track, paying attention to things like transitions, crescendos, and moments to emphasize or minimize.
After outlining a story for each track, we generated visuals that matched the feel of the music to fully draw the audience into the Dolby world that we envisioned.
Technology
The project's main technical challenge was determining the positions of Dolby Atmos audio objects in real time, as their positions could not be directly retrieved during the performance. To solve this, we calculated object positions using two data sources: animation sequences and metadata recorded via the Dolby Atmos Renderer, and real-time timing data (beats) transmitted through the tdAbleton package from LudoWic's Ableton project. By aligning the beat data with pre-processed sequence data stored in CSV files, we could determine the current step in each sequence and calculate the objects' positions accordingly.
We gave the audience a visual reference of where the sound was moving in the space, aiming to achieve cognitive consonance. The real-time positions of the Atmos objects were visualized in a separate virtual world, and were shown in between the music visualizations. This so called “Atmos world” showed the motion paths of these objects, creating the visual bridge between the music and the data.
LudoWic'sperformance
Results
I’ve had multiple people tell me in the last few days that the live performance with LudoWic was the best show they’d experienced at ADE. It felt so special in the room.
Jane Gillard
TV & Music Partnerships, Dolby UK
This project set out to explore the role of data in generating creative art. By harnessing Dolby Atmos' advanced spatial audio capabilities, we crafted a multi-sensory experience that seamlessly fused LudoWic's music with responsive visuals. Through a setup featuring 32 synchronized screens and TouchDesigner software, we built a dynamic visual environment that adapted in real time, enhancing the depth and immersion of the performance. This approach not only redefined the possibilities for live music but also showcased the powerful potential of integrating data-driven visuals with cutting-edge audio technology.
