Media Pipelines

Tools for generating images, video, speech, music, and sound effects while organizing references and outputs.

Creative toolsAI

Diagram of three pipelines: image generation from ranked references, video generation on a PC from a Mac command, and audio models for speech, music and sound effects — Diagram drawn for this page from the three repositories' READMEs and docs.

Status: Exploration, running locally
Tools: ComfyUI, Wan 2.2, Gemini, Qwen3-TTS, ACE-Step 1.5, Stable Audio
Links: Image pipeline source Audio pipeline source

Local tools that generate images, video, speech, music and sound effects for the films, and keep references and outputs organised. Heavy models run on a PC with an RTX 3090, driven from the Mac.

Three kinds of output

Each pipeline ends in one kind of file: images, video clips, or audio for speech, music and sound effects. The tools keep the references that went in next to what came out.

Illustration of three file cards side by side: an image of a lighthouse, a video clip with a play button and timeline, and an audio waveform with headphones — Illustration of the three kinds of output. Not generated by the pipelines.

Decisions

Treat a film's look like a design system

In the Film Design System, characters, props and settings own ranked reference images. A shot is composed from those entities, a preview shows which references go in before anything is generated, and a keeper is promoted back into the canon.

Film Design System Generate screen: Captain Mara, the Brass Compass and Harbour at Dusk selected, a scene line, and a payload preview of the five reference images that would be sent — Composing a shot from the canon: the payload preview lists the five reference images before anything is generated. Demo canon; the reference images were made with ChatGPT and imported through the app. A local capture.

One command for a previz shot

previz takes a start frame and a shot prompt on the Mac, queues the job on the PC and brings the clip back, so the Mac can sleep in the meantime.

A generated frame of a large yellow vehicle touching down in a cloud of dust — A frame generated by Wan 2.2 for a previz shot.

Follow each model's licence

One video model's licence does not cover showing its outputs in several countries, so it is used for private previz only. Shots that get shown are made with Wan 2.2 or a commercial service.

Next project

Story Dice