Culture Curve: Dynamic Shift

In just ten years, Nina Dunn has become one of the most prolific practitioners in her field, producing work for The National Theatre, ENO, Cameron Mackintosh, The Royal Opera House, Bill Kenwright, Edinburgh International Festival, Durham Lumiere and the Culture Mile.

Earlier this year Nina broadened her creative portfolio to deliver a stunning, self-perpetuating video artwork for client Culture Mile. Named ‘Dynamic Shift’, its content was informed and triggered by the sounds of people going about their daily lives in London’s ‘Culture Mile’, surrounding the Barbican. The resulting artwork, which played out on a curved exterior of the Barbican, was a symphony of image and light that transformed an otherwise unremarkable concrete wall into something intriguing.

Delivering the artwork was no mean feat! “I’m used to working in controlled environments, such as performance spaces and arts venues, where there’s generally a captive audience,” Nina explains. “The challenge with a stand-alone installation, where the audience’s motivation is not easily defined and where there is little control over the street environment, is that the work has to be powerful and meaningful enough to attract attention.”

With a brief to create a ‘congregational’ location at the Barbican Centre, encouraging visitors to see beyond the concrete façade of the building to the culture and life that it encapsulates, the project was split into a test stage and the final artwork. Nina says: “As the first test played out, I realised the artwork needed an extra layer of context to resonate more strongly with its location and audience.”

Nina decided to use the soundscape of the Culture Mile to trigger and influence the video output. “Our sound designer, Sebastian Frost, recorded a set of four, five-hour audio feeds between the hours that the artwork would play out. These included people talking in a pub; commuters going through the gates at the Barbican tube; the audience arriving at the Barbican’s foyer, clapping in the auditorium and activity in and around Smithfield market.”

Nina combined these with a live feed from the artwork’s location and used them to trigger graphic content in real-time. To achieve this, she used disguise gx media servers, supplied by Bluman Associates, in association with Notch - an interactive and video content application that enables content to be made and broadcast in one, unified real-time environment.

Nina tweaked numerous parameters of the audio inputs, including cutting back on the ambient noise in the recording of the Tube station. “This made the impulsiveness of a tube gate opening and closing much stronger visually. In many ways the graphics we started to generate began to look more and more like the sounds we were emulating - the possibilities seemed endless. You don’t even have to render the graphics, but if you do you can export them as executable files and keep the inputs at the disguise end of things.”

Nina and her team used the recorded sound feeds as if they were live mixed with the live feed from two mics positioned at the Barbican wall - the location where most passers-by would notice that the artwork was responding in real-time to surrounding activity.

Guaranteeing that these graphics worked well in a single visual composition also presented a challenge. “It’s not just about creating various layers that react live to an input,” Nina explains. “They have to interact with one another in a visually compelling way. Additionally, LED lighting fixtures, controlled by video content through disguise, provided up-light to the concrete voids, making for a seamless presentation of the whole artwork.”

What impressed the client was that they felt embedded in the creative process from the start, Nina enthuses: “We didn’t have to keep going back to the original renders, edit the content and wait a day for the new version. We made things larger or smaller in real time, even if it was 3D rendered. We’d move the camera in 3D space, or change the lighting, and it would update in real time on the wall.”

The first time the creative team saw the artwork at full size was on the first rehearsal day. “I wanted to remain empirical and ensure the artwork was a true reflection of the sound recordings and the live mic feed at the wall,” stresses Nina. “We fine-tuned some elements to ensure no graphic dominated - e.g. when the Barbican hall started to fill up or when the audience applauded, but ultimately I wanted the visual output to be a true assessment of the flow of activity in the Culture Mile at any given time.”

What also fascinates Nina is the responsive of disguise and Notch. New software releases allowed Nina to pre-compose four different graphics into a single thread, enabling her team to send out a single Notch block instead of five separate ones.

Tobias Melin, Coder/Software Developer on the team, developed and incorporated a custom OSC script to scan and catch any tweets containing the #culturecurve hashtag. Once a tweet was received by the server, this was then passed through to disguise as an OSC command triggering the playback of tracks on the disguise timeline.

“We created five fixed states to respond each time the hashtag appeared on Twitter. These described which sound was dominant and where it came from, offering the tweeter a hint on how the artwork was generated.”

Nina says she plans to combine more standard content making workflows with those of Notch and disguise. “It will allow me to take a more narrative approach and increase control over some of the graphic / animated elements. I’d also like to layer rendered graphics with live reactive graphics, embrace automatic cueing, yet retain traditional workflow as a bedrock. I don’t feel one is exclusive from the other. I can also use Notch for non-live graphics by pre-rendering fluid assets and embellishing more formal content as a stand-alone programme.”

For the future, Nina is determined to bring this workflow further into her theatre practice on an upcoming dance project: “I hope to use audio, lighting, video and BlackTrax as influencing inputs, because it further layers of possibilities and results in a design that is truly and innately responsive to the rest of the elements in a show.”