From Workshop to Workflow: Exploring AI Video Production in Practice
On 19 and 21 May, the Immersive Lab organised an ‘AI video production workshop’ as part of the TETRA project General World Models.
During the workshop, participants explored one central question: How can artificial intelligence become a practical part of a professional video production workflow? To make this question tangible, Immersive Lab created the short film ‘Where I Ended’ in three different ways:
- A cloud-based AI workflow, created by one person
- A local real-time AI workflow, created by three people
- A non-AI workflow, created by three interns
By comparing these three approaches, the workshop showed that there is no single “correct” AI workflow. Each method offers different levels of control, speed, consistency and creative flexibility.
Three Ways to Make One Film
- The non-AI version followed a more traditional digital production pipeline, including green screen filming, motion capture, character rigging, animation, Unreal Engine environments, visual effects and compositing.
- The cloud-based AI version explored image generation, image-to-video tools, visual consistency, character references, prompting strategies and iterative shot development.
- The local AI workflow used a Promptable AI Camera, combining live camera input, Unreal Engine environments, custom LoRA models, prompts and locally running diffusion models to generate visuals in real time during filming.
What We Learned
One of the key insights from the workshop was that AI video production is not simply about generating images or clips. Strong results depend on workflow design, preparation and iteration.
For example, generating foreground and background elements separately can give creators more control over composition and consistency. Character reference sheets, multi-angle views and carefully selected start and end frames can also help maintain continuity across shots.
At the same time, traditional production skills remain essential. Editing, sound design, colour grading, visual refinement and quality control still play a crucial role in achieving a coherent final result.
Want to Dive Deeper?
The workshop only scratched the surface of these production methods. The full documentation provides a detailed breakdown of the three workflows behind Where I Ended, including tools, production steps, prompting strategies and practical lessons learned.
