Viridian's AI Experiments

[Viridian]

And now the next level. Original images first generated with NovelAI, then run through Stable Diffusion, then rendered with AI voice animation on D-ID. Then with ambient sounds added in a video editor.

The only think lacking at the moment is how to animate the quicksand, but we've come a step closer to bringing out artwork to life.

[Acidtester]

It’s definitely a good start. Lots of potential.
The face on SG looks really cute when she talks.
Once you get the struggling and sinking to go with it, this could be the coolest thing ever!!

[Viridian]

It’s definitely a good start. Lots of potential.
The face on SG looks really cute when she talks.
Once you get the struggling and sinking to go with it, this could be the coolest thing ever!!

Seriously, the quality of AI voice acting is something - and these aren't the premium voices. While there some gaps in the intonation, it's actually amazing that the AI can string together coherent flowing sentences. The lip sync is pretty good, and the animation - while not perfect - still hits the goal of bringing the scene to life, especially given that this is not what the AI was designed for. It was intended for talking head videos, not quicksand scenes.

[Gitget]

WOW!

Very Good
I'm speechless.

I sure would like to learn how to do that.
Great though keep up the great work.

[MDLambert]

Very eerie, its cool but creepy lol. I love it

[Viridian]

Once you get the struggling and sinking to go with it, this could be the coolest thing ever!!

So, about that.

I ran some test videos through RunwayML. The results are extremely random - you have NO control over what the AI decides to do with the image. However, what it DOES produce, in the 1% that it works, is EXTREMELY smooth. But since there's no control over what is being done with the image, I have to work in reverse: pray for a workable video render, then create the voice lines to roughly match the lips, then add the ambient noise.

Runway can only output 4s clips. It can extend it by a further 4s up to 16s, but each attempt to generate an extension deviates further from the source. "Beach" was the only one that I could extend to 8s. Afterwards she turns into a mermaid or something. It's pretty trippy.

[sixgunzloaded]

Yeah, it's trippy, but quite amusing at this stage! I can only imagine that's what it looks like from a stoned person's perspective. lol
But hey, it'll learn, and quickly, just like it's been doing. I'm assuming this is still in the early stages for the motion apps?
It speaks volumes that AI is already this far along. You began this experiment roughly eight months ago(?) and look how far it's come in that short time.

It's amazing to hear voices put to our OCs and see them move - even just a little. Imagine where it will be a year or two from now!
I'm really glad you started this experiment. You're bringing dreams to life.

And thanks also, for the pantyhose demo! Even if it chopped half her leg off, that pic was so hot. lol It truly looked like the fabric was wet and heavy with mud. Fantastic!

[Viridian]

Yeah, it's trippy, but quite amusing at this stage! I can only imagine that's what it looks like from a stoned person's perspective. lol
But hey, it'll learn, and quickly, just like it's been doing. I'm assuming this is still in the early stages for the motion apps?
It speaks volumes that AI is already this far along. You began this experiment roughly eight months ago(?) and look how far it's come in that short time.

I personally particularly like the "Nature" test. Even with limited tools and only 3 seconds, it gets a nice TV drama snippet.

The challenge is what we can make with this combination of high quality still images, basic moving frames, short animations and voiceovers. A narrated visual novel is possible now. But how to execute?

This is new territory for QS artwork. I'm satisfied that I can generate an infinite number of the pics that we like. We need to go deeper.

[sixgunzloaded]

Woah, that one's extra nice!

[MDLambert]

Very cool stuff, I went to that site and tinkered around most of the time my images the AI would lift them out of the quicksand lol