New kid on the block: Sora

OpenAI recently took an impressive step in the field of text-to-video with Sora. OpenAI’s promise? Sora would generate highly photorealistic videos up to 1 minute long. Based on the examples shared, it seems to be living up to these high expectations and to be a major improvement over other text-to-video applications such as Runway Gen-2 and Pika.

From text prompt to feature film?

Current text-to-video models have demonstrated difficulties with temporal consistency and understanding physical laws such as gravity. Sora shows progress in addressing these issues. Apparently, the model has also learned cinematic grammar as an emerging capability without being specifically trained for it. How long will it take before individual creators can generate entire series or films from text prompts?

… and then straight towards Artificial General Intelligence?

The last sentence on the landing page is perhaps the most intriguing: “Sora serves as the basis for models that can understand and simulate the real world, a capability we believe will be an important milestone in achieving AGI (Artificial General Intelligence).” Runway also recently announced new long-term research into general world models: systems that understand the visual world and its dynamics.

LLMs are gradually showing their limitations. Sam Altman admitted that a new breakthrough is needed. These examples reveal that understanding and simulating the real world, its physical laws, dynamics, and interactions may be the next big innovation in AI research.

First experiments in the Immersive Lab
Researcher Keerthanan conducted a quick test in the lab using Luma AI and was able to extract a 3D mesh from a Sora sample video. Here you can see the result:

AI video workshop on March 13, 2024

Sora will not be available yet, but during the next workshop we will introduce you to some other applications that are already accessible, to give you a feel for the possibilities of AI for video.