blog 2023 5 min read
What’s in store for the future of motion capture?
While motion capture has been around for nearly 30 years, the technical requirements for its implementation have largely limited its use by performers and productions with requirements such as restrictive suits, wires and extra budget. In our latest webinar, disguise Product Manager Camilla Powell welcomed Niall Hendry, VP of Partnerships at Move.ai, and Adam Samson, Creative Technologist at creative studio FutureDeluxe, to explore the benefits and myriad of applications for our brand new, AI-driven real-time markerless motion capture solution named Invisible. Here are five key takeaways from the discussion:
1. A new partnership for seamless motion capture
The Invisible solution is a complete technology bundle for real-time markerless motion capture developed in partnership with disguise and Move.ai. The solution – which can track up to two people at once - extracts natural human motion from video using advanced AI technologies. This allows you to transform human motion into a virtual world or environment which can be a live broadcast watched by millions, a concert crossing into the metaverse, or the latest feature film shot using virtual production.
The Invisible solution leverages Move.ai’s patented Invisible software to extract natural human motion from video using advanced AI algorithms and integrates that 3D data stream into our Designer software in real-time. Once within the disguise workflow, users can employ Unreal Engine to render out frames for use in XR, live, virtual production, and immersive experiences.
“We realized the tools we can bring together really show off the best of each other’s capabilities,” says Niall Hendry in the webinar.
Learn more about the partnership
2. Markerless mocap achieves a faster, easier, and more cost-effective process
By eliminating the need for markers, suits or long calibration set ups, the Invisible solution allows users to mirror up to two performers’ movements into a character rig within disguise’s Designer software in real time. This simplified process reduces prep time, speeds production and gives performers a freedom of movement they’ve never had before.
In addition, real-time on-set feedback enabled by an environment where the virtual characters and worlds are in plain sight, enables the director and other creatives to see how the performers are interacting with the environment and if retakes are needed - giving performers a chance to tweak their movements for a better result.
“Real time means you’re going to get 95% of the final quality, which means you have a very different idea of what you’re in the process of making while you’re making it,” notes Adam Samson.
3. Motion capture dazzles audiences
After successful showcases at NAB and the Cannes Film Festival this year, productions of all sizes and budgets that might have shied away from the complexities of motion capture now see how easy it can be. XR production, location-based experiences, live events, filmmaking, broadcast, fixed installations and the metaverse all find advantages with Invisible’s quick and easy motion capture capabilities.
For example, you can add accurate shadows for talent in XR and broadcast productions. These shadows mirror the talent’s movements in real-time and in-camera. You can also create interactive, motion-triggered graphics to bring impact and make weather, election and sports broadcasts come to life.
If you want to attract visitors to your installation such as those seen in theme parks, museums and branded stores, you can use motion triggered effects, live digital puppeteering and virtual characters to make it exciting. Imagine having an installation where people can step onto a live size virtual piano and trigger effects for each key they stamp on? This is all now possible.
For live events, you can transport performers into a game universe with avatars that replicate their movements. Create a rich social experience and the potential for new audiences and new ways to monetize events crossing into the metaverse. Create a rich social experience and you have the potential to reach new audiences and find new ways to monetize events crossing into the metaverse.
Filmmakers can also use motion capture to create virtual characters while preserving cinematic quality on set.
4. Greater access to new creative opportunities
Adam Samson from Future Deluxe sees the Invisible solution empowering a lot of individual voices. “With better tools coming together and more accessibility to those tools, creatives are able to take their ideas and push them further,” he says.
As AI continues to make its way into every corner of our lives, Samson believes “there is a lot of room to come up with really good combinations of computer intelligence and human intelligence, make bigger things with them and remove the headaches that happen in the creative and production process.”
5. This is just the beginning
Our Invisible solution will continue to evolve to offer users new and improved features to meet their needs.
In the short term, Niall Hendry forecasts advances in reducing latency, improvements to data that deliver even closer to finished quality video and speed postproduction, and the ability to track more people in an LED volume.
“Being integrated with disguise and pushing that [partnership] forward is definitely highest on our agenda,” he reports.
Get started with Invisible
The Invisible solution is a small, medium or large bundled offering, depending on setup size and numbers of performers, supported exclusively by disguise.
It includes disguise MX Move data processing hardware, purpose-built to run the Invisible software; FLIR Blackfly S GigE machine vision cameras; TAMRON 2.8mm, Goyo 3.5mm or LCM 4mm fixed focal length camera lenses, depending on the size of the LED volume; and a Netgear MS510TXPP network switch supplying PoE for the cameras.
A minimum of one disguise vx or gx media server is also needed for graphics processing and one disguise rx II for real-time rendering.
