This is my second experiment using AI to create a short film or trailer, and I would like to explore how far I can go using only AI. I used LeonardoAI. Basically, Leonardo AI uses Stable Diffusion, and there is no text-to-video generator, but you can generate the image and then animate a 4-second-long video.
I had the original idea to make a 5-minute short film, but I faced a lot of limitations. The first and main problem was that I couldn’t make robots fight. So there were a lot of images, but when I started to animate them, I was disappointed because I couldn’t make them actually hit each other.
Then, I was evaluating the option to make characters talk, but I didn't like this option due to several limitations. The third problem was I couldn’t make a bullet shot, so I had to go with a firearm. You can have good results on images, with beautiful moves, etc., but when you start to animate it, in some cases, it just doesn't work.
The basic workflow is to generate images with the prompt and then upscale them using LeonardoAI Universal Upscaler, which gives amazing results on the images, but when you convert them to video animation, it reduces the quality a lot. Anyway, in close-ups, it works pretty well, but in general and master shots, sometimes it's a mess, and you can’t see the character's face.
I tried to use different video generators like RunwayML, but I found out that there are a lot of censorships, so I couldn't animate fights due to the policy restrictions.
I had the idea to make Samantha talk, but it was almost impossible to make POV shots from her perspective; otherwise, I would have to use external software for VFX. It works pretty well with general shots like factory aerial master shots and with close-ups, but all the rest require a lot of improvement and have a lot of limitations. I can't wait to try SORA, but I think it will have the same problems with characters: complex movements, continuity, expressions, emotions, and talking.
Anyway, LeonardoAI is a great tool, and I totally recommend using it and experimenting with it.
Check other AI for video creation