Generative AI Reel
As the world of GenAI continues to develop at breakneck pace, I find it important to document the various methods I have used as an artist to make this tool useful to the creative process.
​
Note: The content displayed below predates the announcement of Open AI's Sora by 9 months and is based on completely open sourced technology
Gen AI Reel 2023 | SD 1.5+ControlNet
A big problem one encounters when using AI tools is the inability to direct how the camera and subjects move in a video. This due to the fact that AI models are unable to accurately guess the distances between objects and are often uncertain of where one object ends and another begins or how motion and physics work.
​
To find a workaround for this, I decided to use a workflow that uses Stanislav Demchenko's StableHoudini plugin to leverage 3D software so a user can generate depth maps that will then be processed through Stable Diffusion using Automatic1111 and SideFXHoudini.
​
This allows us to use 3D models, alembic files and 3D cameras to direct the generation of an image or video, giving us control over what action takes plaace, what environments look like, camera moves and even the focal length of the camera in use.
The video displayed below helps breakdown this concept further:
* Generative Video AI is a young technology that is still imperfect. At the time of the making of this video (2023), the ability to create videos that can maintain temporal consistency from frame to frame was still lacking with no viable commercial or opensource alternatives.