Small studio, big cinema

Smart Virtual Production with Unreal 5 for a better actor experience

Size does not matter (anymore)

As a small film studio, physical space and financial budgets are limited. Virtual production helps a lot with these issues, but the studio is still limited by the size of its premises when it comes to shooting scenes where actors have to walk long distances.

Filmstudio Basel

Filmstudio Basel offers smart film productions ranging from corporate videos to event documentaries and cinema/TV productions. With Virtual Production they can bring any location into the studio — reducing both costs and carbon footprint.

UX Design

Prototyping

Software Engineering

What is Virtual Production?

Virtual Production is a technology in which LED panels or back-projections are used as a backdrop for a film set, on which video or computer-generated imagery is displayed in real-time.

This allows studios to shoot a scene in any environment without needing the entire crew on location, making film production considerably more cost-effective and sustainable.

A currentless treadmill

We already successfully partnered with Walkolution on other projects. The company produces motor-less treadmills that are the perfect base so that actors can walk naturally without actually changing their location. The key feature of the treadmill is that it is purely muscle-powered, solving a major problem we saw early on: a traditional electronic treadmill would rob actors of their self-determination in acting, as it runs at a fixed speed.

With the Walkolution treadmill, the actors can decide when to start and stop walking, or even walk and run at different speeds, increasing the actor experience tremendously.

The challenge: digitising the treadmill

Of course, a muscle-powered treadmill would not be enough. The virtual scene rendered by Unreal 5 had to move in perfect sync with the actor. First, we built a sensor to measure the actor's walking speed, using a classic Lego wheel attached to a rotary sensor driven by the Walkolution surface.

In order for Unreal 5 to communicate with the treadmill, it needs input parameters it can actually understand. The Walkolution is not digitised at all by default, but we still needed digital input data for the game engine. For this, we connected the sensor to an Arduino Leonardo microcontroller.

A treadmill as a game controller

The Unreal game engine needs to interpret the output of the rotary sensor as a movement input to reposition the virtual camera. To solve this, the microcontroller emulates an Xbox gamepad for Unreal 5. The velocity information from the treadmill is mapped to the gamepad's analogue sticks.

Slow walking on the treadmill corresponds to gently pressing the analogue stick forward, while fast walking corresponds to pressing the stick all the way. With a bit of tweaking, we managed to link the treadmill's movement to the Unreal Engine's camera in real time.

Real-world and virtual camera synchronization with nDisplay

One of the key components for this project was Unreal's nDisplay technology. Its support for virtual reality components allows us to synchronise cameras on set with virtual cameras in the engine.

Using motion tracking devices such as the HTC Vive, we can track the movement and orientation of the real-world cameras and replicate that movement in the virtual environment. The result is a seamless integration of real-world and virtual footage, creating a more immersive and believable final product.

Prospects: enhancing the actor experience

Freeing yourself from studio size limitations is on its own something small studios will look forward to. But the benefits and future possibilities of this project don't stop here. In the traditional world, any visual effect that is supposed to go along with the action of an actor would need to be synchronized manually as a post-processing step. This can be a source of errors and quickly become expensive. In order to keep post-processing effort to a minimum, actors must be able to determine for themselves when a visual effect is supposed to happen, as they have the best feeling for the timing of their acting.

What if the actor could trigger events in the scene with their feet? An explosion, for example? The technology we built in this project with Filmstudio Basel is a step towards that direction.

Virtual production in practice

Technologies

Learn more

It was a great pleasure to work with Centigrade again.I was very excited to utilize our product in such a creative and experimental environment and it gives our company the opportunity to further expand the use case of the walkolution.

Dr. Eric Söhngen Founder/Managing DirectorWalkolution GmbH

Working in a small studio may limit your technical options, but it doesn't stop you from coming up with great ideas. With Centigrade's creative approach, we were able to develop a solution to our space constraints and improve the experience for our actors. All this in just two days while having a lot of fun — hackathon style.