Immersive & Interactive: Virtual Production In Video Games

Virtual production in films and video games share an intricate and intimate relationship. In fact, a famous real-time creation tool, Unreal Engine, which makes virtual production possible, was first used in a first-person shooter game, Unreal.

We have been filmmakers for a long time, but a story we both loved made us consider 3D video game as a medium. There’s an Indian novella, Sultana’s Dream (1905), by Rokeya Shahkawat Hossain, a futuristic story where a little girl Sultana wakes up in a feminist utopia. It’s a sci-fi dream, with solar energy, flying cars, and a society where all beings in nature are equal. Hossain described a circular system, predicated on equality, compassion and learning as opposed to the pyramid system of patriarchy we still suffer from. It set us thinking: how to adapt her radical vision for a contemporary audience.

Unlike films, video game is an immersive and interactive experience. Also, most people watch films just once, while a game can be played many times over, sometimes with different outcomes. Much like how cinema changed the lives of people in our generation, video games are doing the same for the current generation. As per Netflix’s data, most of its user base is above the age of 38. The youth have other interests now, and we’re interested in their interests.

Video games in the past decade have been mostly competitive and often violent, catering to a certain kind of player. We wanted to change the politics of the video game: change a player’s interaction, make it more inclusive, make it more conscious. So we came up with the idea of a first-person-shooter with a camera, where the player picks stories and progresses in the game with a camera (instead of a gun). Virtual production made perfect sense for a story like this because it’s set in a world so progressive that it doesn’t exist in the real world—but now it can, in the virtual one.

We’re creating an immersive 3D space that imagines a gripping and fun world, founded on mysticism and ecology. Immersive, yes, but not hyper-real. It’s a stylized world, a beautiful world. It’s also mapped on a real piece of conserved forest land in Palolem, Goa. The real-life impact of the game really fired our imagination. We were helped by several mentors who made us realise the key differences between video games and cinema. Here’s the most crucial one: unlike writing for cinema where the audience is in front of you, in a video game the player is right behind you. It’s a huge difference. It’s the biggest challenge we have ever faced: “Oh, shit, she’s gonna jump! Oh hell, she’s gonna turn left.” So you just want to finish your part before the players overtake you!

Besides, everything counts in a video game. If the players jump, what is its pace, velocity and weight? Can we have a little twirl in their walk? For example: there are generic animations for a woman or a man standing, so we mixed the different styles of ogres and bears standing to create Moushi, a fisherwoman, who moonlights as a DJ. Why do all of that? Simple, we want to create unique ‘Indian’ characters.

India is a new player in the video gaming industry with loads of talent. The major challenges are referencing and localised assets. Original work is difficult to move in the infancy of an industry. People are slow to accept that to produce something original, it takes passion, sweat, rigour and time. We want to look at mysticism, handloom, craft, folk music, costume, culture. We want to return to our roots to visualise a future.

We’re also certain that our engagement with virtual production will continue beyond video games. Our next feature film, for instance, is a period piece set in the 1920s. We plan to stitch up photos from that time on a virtual set and ‘mimic’ live action. That prospect excites us greatly—to use and move still pictures in such a way that it transports the audiences to, say, Marine Drive in 1924.

Even though the possibilities of AI fascinate us, we are worried that it may be used to churn mechanical work. We had a writer who had to write dialogues for Bot-Kabir, a character in the game, and they produced something which made us go, ‘Umm, it’s … okay, but is this how you think a mystical poet in the future would speak?’ They said, ‘You know, I didn’t write it myself, I put it on ChatGPT!’ So, we wonder if that’s also one part of the AI-powered future. We needed the machines to clean the house, while we wrote poetry. But it looks like we are working the backend, while AI writes poetry.

(Views expressed are personal)

(This appeared in the print as 'Immersive & Interactive')

Q is a storyteller who has just finished his new film Zewel

Hina is a film editor and a producer whose films have premiered at festivals worldwide