AI At NAB – Old School Meets New School

AI At NAB – Old School Meets New School

Artificial Intelligence which has come of age in the film and broadcast industry was given center stage at NAB 2024.  For some creators, it is like a Sword of Damocles, hanging over the future of their livelihood. To others it is the end of the rainbow, an ally to take over mind numbing tasks. AI has for some time been used in Visual FX and blossoming new editing tools. It has now found its way to the nitty-gritty of filmmaking to help cinematographers and other artists revolutionize their workflow.  Prior to NAB, the issue was given voice at the Editors Lounge, the best prep for the conference.

It was here that some cast AI in the role as a savior. Shahrzad Benzadi of MTI Films talked about the magic that AI can perform in film restoration. MTI, which has restored such classics as Lawrence of Arabia, Taxi Driver, The Godfather (PT2), Bridge Over The River Kwai and Scarface, can now call upon AI to recreate the missing frames of film classics degraded by mold, age and improper storage. We are in the midst of an industry wide crisis of epidemic proportions, a great number of classic films shot on silver nitrate, that have not been restored, are lost forever. Many modern film stocks have also suffered. After only 5 years, Steven Spielberg was shocked that the blue of the water in Jaws had faded while the blood became more saturated. Looking for a job in the film industry? You can’t go wrong with restoration.

On AI as a threat, colorist Alexander “Zander” Schwab (We’re Doing Good) said, “If you sit in the room with directors and DPs (director of photography) you’re going to get a sense of how they feel about AI; the amount of care that they’re taking for their imaging and how dialed in they are to create their visions. The last thing that they’re embracing right now is having AI touching anything that they’re doing. This industry is being held up by relationships, studio with director, director with the cinematographer, cinematographer with you as the colorist. If you’re not interested in having relationships. I would say you know good luck making Dune 2 with AI.” 

Perhaps the centerpiece for AI and the movie industry at NAB 2024 this year was Generative AI and The Role of The Cinematographer chaired by Gary Adcock, one of the main tech gurus for the movie and television industries.

 He was joined by a panel of prominent cinematographers who were more upbeat about AI’s entrance into the production world.  Many viewers are unaware of the preplanning that happens before filming starts. Especially the story or photo boards that are created to show the director and or the producer where the position of the camera, its angle and what the framing must be. 

 Alicia Robbins (Greys Anatomy,Bridgerton Season 3), “I’m a horrible artist. I could not draw at all to convey how I would put this scene together, that went to all of the producers. Everything had  my horrible little hand drawn renderings.  I had a friend actually push that through some AI technology, that would show me what it could look like. It was way better, and I would much rather send that to the producers than my stupid little stick drawings.” 

Dave Stump describes the agony he experienced  rotoscoping on Tron. He then goes on to describe how AI revolutionized the workflow on the movie Commandante, which allowed VFX Post Production to start during the shooting of the film. “I was moonlighting when I was working 20 hours a day, I plopped rotoscope cells on the original Tron. And it is such mind numbing work that you just have to get up and walk away every now and then and recover your sanity. So, It shouldn’t be any surprise that one of the first low hanging fruits was Rotoscope; because there are all kinds of cues that an AI bot can use to determine the shape and cut it out. And unsurprisingly, everybody, everywhere is going after that.” 

“I have been keeping a list of software companies that were an outgrowth of what we did a year and a half ago on Comandante where we used CopyCat, the Nuke roto utility. We trained CopyCat on a few frames and then taught it to fill in. It’s actually amazing. You can do the rotoscope for a 2, 000 frame shot by doing the first 100 frames and the last 100 frames and training it up and saying, Okay. This is what it ought to look like. Do the middle to match. And it’s astonishing.” That’s great news for us as filmmakers. That’s terrible for thousands of 20 year olds working in Surinam or in Vietnam (who we farmed out the roto to in the past). If a task gets too expensive, we’re going to build a computer to do it.”  Gary Adcock went on to point out that AI itself, especially generative AI where you describe the miracle the bot is to perform, “is almost all programmed in standard English, not in foreign language.”

“And maybe one of these days people aren’t going to care that the creator is a computer who did it. But we’re not just creating things, we’re curating them,” Alicia Robbins added. “And that is a place where a cinematographer, I think, is absolutely going to be needed with AI.  I do think there’s a lot of value in the way Scorsese created it. He put this together, the person behind the actual story that you’re watching. And I guess, we’ll see what happens with society there.”

In all the years I shot performers or nature some of my most memorable moments were what I called “happy accidents.”  Like the blur on a slide trombonist’s hand that glowed in the dark.

When I asked Gary Adcock’s his response was, “The happy accidents you mentioned frequently show us opportunities that we didn’t know existed. They show us what was possible, that we didn’t know was possible. And to go back to what we were saying earlier, AI is fundamentally a feedback loop. It can really only revert to taking what we already know to be true or possible or interesting.

“So I think whether or not viewers appreciate the value of it, I think we’re not going to be able to actively develop more content or art or creativity simply on the back of AI. You need those happy accidents, you need the humanity and the interaction of the crew on set to push things forward.”