Broadly speaking, the video generation techniques available today fall into three categories:
Text-to-Video
Video-to-Video
Image-to-Video
Industry Updates & News:
Top-tier AI video generation application (Part 1)
OpenAI has officially launched Sora, its advanced AI video-generation tool capable of creating realistic videos from text or image prompts, available exclusively to ChatGPT Pro and Plus subscribers, with features like a storyboard interface and remix options for customization.
As of March 2024, OpenAI has been exploring partnerships with Hollywood, aiming to revolutionize the entertainment industry with AI-generated video content.
Runway, an innovative artificial intelligence startup, is at the forefront of developing generative AI video tools. Their flagship product, Gen-2, is reshaping the landscape of video content creation.
Update:
They just revealed their new Gen-3 Aphpa. Runway's Gen-3 Alpha, a cutting-edge AI video generator trained on both videos and images, offers improved control, faster creation times, and higher fidelity compared to its predecessor, Gen-2, supporting tools like Text-to-video, Image-to-video, and Text-to-image.
Act-One
Runway's Act-One is revolutionizing AI-assisted video creation by enabling creators to capture and transfer facial expressions from a single video performance onto AI-generated characters without complex motion capture equipment. This makes high-quality character animation accessible to creators at all levels.
Luma Ray2
Luma released Ray 2, a new AI video generator model that creates realistic videos with natural and coherent motion, available now on the Dream Machine platform
Dream Machine 1.5 is here. Now with higher-quality text-to-video
Hailuo AI is an AI-powered productivity suite developed by MiniMax, a Chinese artificial intelligence company based in Shanghai, that offers text-to-video generation, intelligent programming assistance, and other AI capabilities to boost efficiency.
Kling 2.0 is the latest version of Kuaishou’s advanced AI video generator, delivering major improvements in prompt adherence, dynamic action, and image-to-video fidelity, making it one of the most powerful tools for creating high-quality, AI-generated videos from text or images
Meta has unveiled Movie Gen, a groundbreaking AI-powered suite of models that can generate, edit, and personalize high-definition videos with accompanying audio using simple text prompts.
Top-tier AI video generation applications (Part 2)
Google Veo 2
Google just released Veo 2, an advanced AI video generation model that offers enhanced realism, improved physics understanding, and extensive camera control options, setting a new standard in video creation technology.
Pika
Invideo is a comprehensive video creation platform that offers an AI-powered solution for creating engaging videos quickly and easily.
It has now introduced a GPT feature, enabling the creation of video content right within ChatGPT. Users can write scripts and then directly connect to Invideo to bring their creations to life.
Hedra is a platform that provides a personal AI studio for creating generative images, videos, and audio content.
Case：
motion control
OpenSource Models:
Tencent HunYuan Opensource VideoGen model
Chinese internet giant Tencent has launched HunyuanVideo, an open-source AI model with 13 billion parameters, designed to generate high-quality videos from text prompts, offering state-of-the-art video quality and motion.
Genmo Mochi 1
Genmo AI is at the forefront of video generation technology, offering models that create realistic motion adhering to physical laws and providing detailed control over video content through textual prompts.
Image-to-Video Solution
AnimateDiff
AnimateDiff is an AI plugin based on Stable Diffusion that creates short video animations from simple text prompts. Users can choose from styles like comic book, fantasy, anime, photorealistic, and 3D.
Recent updates have improved the output quality, making it comparable to premium video editing tools. This tool allows anyone, even with no technical skills, to create high-quality short videos, opening up new opportunities for creating short video content.
Example: Below is a short animation made by director Karen Cheng on X using AnimateDiff.
Video-to-Video Solution
Video-to-video editing tools are great at changing how characters look and altering the style of a video. They can turn real footage into animated scenes or replace characters seamlessly. This technology gives users more control over the final video compared to text-to-video methods.
Move AI
Domo AI
Runway Gen3, Luma, Kling 🎥 Video Prompt Maker
Expert in video prompts. Create videos in any style you can imagine with Text to Video. Instantly generate high-quality prompts, and spark your creativity. Perfect for Runway Gen-3, Kling, Luma AI, Sora and Pika.
VideoGen-Eval v1.0
The Dawn of Video Generation: Preliminary Explorations with SORA-like Models
