Rhiju Talukdar is one of the co-founders at Dot Films. He writes, directs and doubles up as one of Dot’s researchers and self-styled experimenters. He also moonlights as a musician and card-game designer over the weekends. He was also one of the founding team members of Blue Vector, Dot’s parent agency, having dabbled in all major facets of content and branding right before.
Almost all your jobs are safe. And even if one job goes extinct, 3 new ones are getting created in its stead. That had to be said before this piece spiralled into masochistic sensationalism that a lot of AI-related content has been casually adopting for years now. For the foreseeable future, AI isn’t going to overturn society as we know, but only augment existing human abilities, and it doesn’t take an AI expert to figure this out, an enthusiastic hobbyist can tell you the same thing.
For the bigwig studios, AI is already helping, but it’s not making them better filmmakers than they already are. There’s the obvious sexier use-cases of bringing back dead actors with deepfakes and even in helping productions make the most optimised casting choices, but it isn’t redesigning the filmmaker’s fundamental process. Some would argue that even writing’s being taken over by the robots but that still has a long way to go before there’s a Neural Network Writer’s Guild.
In essence AI is affording more time to filmmakers by taking away the labour of a lot of the menial stuff so that the focus remains the story, characters, and creating unimaginable worlds. Fortunately, as of now, the degree of monetisation of these activities hasn’t reached the extent where it’s completely out of reach for the indie filmmakers. Companies like Adobe are harnessing the power of the Subscription Model like it’s their business, which it is. This is making AI-magic a plausible tool for any filmmaker, irrespective of their portfolio.
Here are some areas of production and pre and post-production where AI can lend an artificial and cost-effective hand, or at least create a taste for it.
Music, at scale
The importance of music has never been greater, partially because it has never been so easy to use it so expressively before. Also, social media probably has something to do with that but that’s for a separate discussion. So, while the Zimmers and Elfmans of the world may not be on our contact lists and we may not have the best lawyers to help us siphon favours from global artists, we do have our friends and acquaintances who are music producers. But asking them to score your 15-minute short film for free is just pushing it given how music can make or break a film. Products like Boomy and Soundraw can help create a soundscape which accurately conveys the mood, but you’ll need to have some existing idea of music and how you want it to work for your film.
Voice-overs and subtitles
An easy entry point for filmmakers into AI-usage is voice and text. A simple Google search instantly spoils you for choice, offering you a laundry list of services, some free, to get high-fidelity, believable AI-generated voiceovers or subtitles. An extension of this usage also happens be ‘virtual presenters’ wherein a model reads out what you’ve written while safely steering clear of the Uncanny Valley. But like other applications, this too isn’t a magic wand and requires some practice before getting just right.
AI in VFX
Unbeknownst to them, anybody using any VFX or 3D software reaps the benefits of AI with every new update that they download. A gross oversimplification of how this works is this: better processing has allowed for more AI technology to be used in the background, which in turns leads to faster performance. From rendering to ray-tracing, AI is involved in almost every facet of the technical process and deserves more acknowledgment than we give it credit for. So indie filmmakers who involve some VFX in their work, all they need to do get the AI benefit is upgrade their systems really.
Environment Generation
A lot of filmmakers will know about the virtual sets used in Mandalorian and how it’s a gamechanger for filmmaking. Combined with good 3D artists who know their way around Unreal Engine, you can create fantasy sets out of the very depths of your imagination at the fraction of the cost of creating a real set. It still may not be the perfectly affordable replacement for the humbler green screen, but given enough time and innovation, smaller, more modular variations could soon become a reality. Till then tools like Nvidia’s AI landscape painter like GauGAN 2 can help you literally ‘draw’ out outdoor locations which come closest to the descriptions in your script. Not ideal for shooting purposes but definitely a crutch during the storyboarding phase.
Editing and some
Editing is a lot of things, including the terrible bits. Behind a few seconds of playtime could be hours of doing one task over and over again for hours. Rotoscoping is one such task and it’s in mundane never-ending activities like these that AI excels. Comixify (still in Beta) and similar tools can help you rotoscope, scale resolution and even slow down your footage. This is ideal for filmmakers who can’t afford reshoots or don’t have full control of their productions, which just happens to be most indie filmmakers. Besides ‘editing’ it even lets you restyle your footage to something more experimental for when its appropriate.
Better Visualisation
An idea that’s visualised is more than an idea, it’s motivation to start. Newly inducted household names like Dall-E and Midjourney let creators conjure up concept art, storyboards, moodboards, set ideas or even character designs with a few simple sentences. For young filmmakers who don’t have friends in high creative places, this could mean better pitches, better direction, and fewer dead ideas. As long as you can navigate past the nightmarish outputs after some practice.
To test these new off-the-shelf AI, and just for fun’s sake, we attempted a little experiment at Dot Films. We prompted a writing AI called Jasper with the beginning of made-up odd stories. Then we took that input and thought of interesting frames to better convey the story, getting Dall E and Midjourney to generate frames for these visual prompts. A rich synopsis of sorts. The whole exercise wasn’t something out a sci-fi movie, in fact it was kind of a reminder how AI uses that which already exists and remoulds it in some unfathomable mysterious way to give us something that’s close to useful and platonically a cliché. While we didn’t create stories that just needed telling, it did give us fantastic inspirational material to start with. There was a lot of word-wrangling that we had to do to ensure good outputs because the AI isn’t going to bother to try to understand you, but you definitely need to try to get under its skin. If nothing, it helps validate that the things imagined are too ludicrous, undoable, or just hard to imagine off of a page.
AI by itself doesn’t seem to be a new member of the film crew but it will invariably become more integrated with our workflows and soon enough we may hopefully start to see some dependencies grow, for the established filmmaker and the little guy as well. Right now, when while things only just taking off it could be prudent for us to get comfortable with these new technologies because the growth of the applications is going to be exponential, even if sometimes some of it does feel ‘gimmicky’ these days.