Hermosa Research·Synthesis
Generative Media Pipeline Patterns
A catalogue of 77 generative media pipelines across 9 families and 226 sourced stages. It covers image, video, character consistency, audio-driven animation, and 3D, with the ordered models, controls, failure modes, and source for each chain.
- Published
- 3 July 2026
- Data as of
- 19 August 2026
- Research type
- [synthesis]Distils many external sources into a position. The rigor lives in source verification and per-claim attribution.[tentative]Reasonably supported, but parts are still unverified or liable to drift.
- Topics
- [generative-media][diffusion][image-generation][video-generation][3d-generation][ai-pipelines]
Method: We catalogued 67 chains across nine families and 194 stages. Each pattern names at least two distinct generative models, an ordered workflow, controls, failure modes, and a source covering the chain.
Background
Generative media workflows often split work among models. One pass establishes composition, another preserves identity, a third adds motion or detail, and a final pass restores or upscales the result. The order and handoff between those stages determine whether a result can be repeated.
This catalogue records 77 reusable chains across 9 families. Every entry names at least two generative models, orders the stages, identifies the control inputs and common failure, and links to a source that covers the chain.
Key findings
- 1Most chains separate exploration from finishing. A fast or flexible pass establishes the asset, then a slower or more specialised pass refines detail, motion, identity, or texture.
- 2Control inputs deserve their own stage. Identity, pose, structure, and style are more repeatable when supplied through references, adapters, masks, seeds, or control networks instead of prompt wording alone.
- 3Image-to-video starts with composition. Many motion workflows treat a strong still, first frame, or storyboard as the upstream asset, then pass it to a video model for movement and timing.
- 43D workflows often separate geometry from materials. Pipelines such as TRELLIS and Hunyuan3D generate or refine shape before producing texture, which makes each stage easier to inspect and replace.
- 5Audio-driven animation now extends beyond lip sync. Recent systems use speech or music to drive facial motion, gesture, or full-body movement. The dataset separates those workflows from face-only talking-head pipelines.
The pipeline catalogue
The 77 pipelines are grouped by workflow shape. Each record gives the ordered stages and models, controls, use cases, multi-model rationale, common failure, and source.
pipeline 1 of 77 · Draft to finish
Base + Refiner Draft-to-Finish
text -> image·SDXL base -> SDXL refiner -> 4x ESRGAN upscaler (or FLUX.1 schnell draft -> FLUX.1 dev refine).·Shared seed across passes; denoise strength on the refine step; base/refiner step-split ratio; optional control net on the draft.·One model rarely optimizes both global composition and fine detail at once; splitting draft from refine lets a cheap fast pass explore many options and an expensive pass finish only the keepers.·Over-denoising the refine step drifts away from the draft composition; a mismatched seed breaks continuity between passes.
Summary
A fast base model lays down composition and structure, then a high-fidelity refiner pass adds detail and fixes artifacts, optionally followed by an upscaler.
- Stages
- Draft: text-to-image (SDXL base, FLUX.1 schnell)
- Refine: image-to-image (SDXL refiner, FLUX.1 dev)
- Upscale: upscale (Real-ESRGAN 4x)
- Use cases
- Hero art and key visuals
- Batch concepting then finishing keepers
- Print-resolution stills
Browse all pipelines, stages, sources, and CSV/JSON downloads →
What recurs across the catalogue
The models change quickly, but three implementation concerns recur: explicit controls, defined handoffs, and validation at stage boundaries.
Use explicit controls
References, control networks, masks, seeds, and identity embeddings make important constraints visible and reusable. Prompt text alone is a weak place to store them.
Model the handoff
Each stage has a defined input, output, and transformation. The dataset maps those stages to workflow-node categories so the chain can be inspected or implemented as a graph.
Failures cluster at handoffs
Seeds, masks, dimensions, colour, identity, and camera assumptions can drift between stages. A pipeline needs explicit contracts for the artifacts and controls it passes forward.
When a pipeline is warranted
Every additional stage adds cost, latency, and another failure point. Add one only when it addresses a named limitation: a finishing pass for detail, a reference stage for identity, a motion stage for video, or a texture stage for 3D. Define the input, output, controls, and acceptance check for that handoff before adding the model.