Page 1 of 6

Viridian's Runway Gen3 AI Animations

Posted: Fri Aug 02, 2024 9:59 am
by Viridian
Thanks to Duncan for tipping me off for the image-to-video function. It's still in the early days with virtually no control and the usual amount of utter randomness, but here are some pleasing results.

Re: Viridian's Runway Gen3 AI Animations

Posted: Sat Aug 03, 2024 7:51 am
by Acidtester
Very promising! I can’t wait to see where this goes from here!

Re: Viridian's Runway Gen3 AI Animations

Posted: Sat Aug 03, 2024 8:00 am
by Viridian
This is a non-quicksand teaser. Getting mud textures is an absolute bitch still, and I'm afraid that Runway still can't do it - I'm guessing the training sets simply don't have enough mud textures and everything forces water splashes. However, the cohesiveness of the outputs is brilliant, which means cinematic scenes as close to life-like as possible when combined with high-quality photorealistic AI images.

So, here's Susan and Sanders in the opening scene of sixgunsloaded to-be-released story, animated and lip-synced with RunwayML's current tech. It doesn't use keyframes. However, it is FAR better at retaining the features of the original image, and can create additional items in the image including backgrounds, objects and text.

Runway's always been good at generating backgrounds, and Gen-3 is REALLY good at creating environments, especially now that can create 10-second sequences instead of 4-second chunks.

It's only missing the ability to consistent create mud/quicksand textures to make this work.

Re: Viridian's Runway Gen3 AI Animations

Posted: Sun Aug 04, 2024 8:10 am
by Viridian
Now presenting what is probably the cutting edge of what Gen-3 Alpha can put together (in a day).

Re: Viridian's Runway Gen3 AI Animations

Posted: Sun Aug 04, 2024 8:18 am
by PA Jack
This video is impressive, looks like an amazing promise to make any quicksand vignette that you want to.

Re: Viridian's Runway Gen3 AI Animations

Posted: Sun Aug 04, 2024 10:19 am
by MDLambert
Kudos! that was quite impressive

Re: Viridian's Runway Gen3 AI Animations

Posted: Sun Aug 04, 2024 1:13 pm
by cerberus
That was cool, I was just getting into it and it stopped, dang! That's quite a bit further than I thought AI Video had got. I'll be keeping an eye on it from now.

Re: Viridian's Runway Gen3 AI Animations

Posted: Mon Aug 05, 2024 2:59 am
by Viridian
Update: using Final Frame, I've managed to create extended scenes. Basically, generate the 10s clip in Runway, use Final Frame to get the last frame, and then generate it again. It does OK in maintaining a continuous sequence.

Re: Viridian's Runway Gen3 AI Animations

Posted: Mon Aug 05, 2024 7:49 am
by Acidtester
I love how you’re continuously pushing to find new ways to keep making better content.
I’ve watched the Safari Girl clip no less than 15 times so far.
It’s how I would’ve started a comic strip, with the landscape view of a jungle, then her peacefully strolling, then shit gets bad.
I can’t wait to see where you can go from here.

Re: Viridian's Runway Gen3 AI Animations

Posted: Mon Aug 05, 2024 8:41 am
by Viridian
FRESH OFF THE PRESS - Runway just updated 10 minutes ago. The image reference can now be used as the final frame. I repeat: keyframes are now a thing.