HunyuanCustom Multi-Subject Video Customization

Tencent's HunyuanCustom fuses a LLaVA-based text-image understanding module with the HunyuanVideo DiT and an image-ID enhancement module (temporal concatenation) to keep one or more reference subjects visually consistent throughout a generated or subject-replaced video.

Properties

family: character-consistency
modality: reference image(s) + text (+ optional driving video) -> subject-consistent customized video
whyMultiModel: The multimodal conditioning explicitly combines a separate vision-language model (LLaVA) for text-image fusion with the HunyuanVideo generation backbone, plus dedicated injection sub-networks (video-driven injection module for subject-replacement mode); no single model handles both the semantic fusion and the temporal identity reinforcement.
steps: [object Object], [object Object], [object Object]
controls: one or more subject reference images, text prompt, optional driving/source video for replacement mode, dual-subject vs single-subject workflow selection
exampleStack: LLaVA text-image fusion -> HunyuanVideo DiT with image-ID enhancement -> video-driven injection module (subject replacement)
useCases: [virtual product/human advertisements with a consistent subject dropped into new video scenes][dual-subject consistent video generation][replacing a subject in existing footage with a reference identity while keeping motion]
pitfalls: dual/multi-subject workflows are more failure-prone than single-subject (identity bleed between subjects); the video-driven injection (replacement) path is sensitive to the compression/alignment of the source video and can introduce warping at subject boundaries
source: HunyuanCustom: A Multimodal-Driven Architecture for Customized Video Generation (arXiv:2505.04512)

Provenance

Source: https://arxiv.org/abs/2505.04512

Properties

family

character-consistency

modality

reference image(s) + text (+ optional driving video) -> subject-consistent customized video

whyMultiModel

The multimodal conditioning explicitly combines a separate vision-language model (LLaVA) for text-image fusion with the HunyuanVideo generation backbone, plus dedicated injection sub-networks (video-driven injection module for subject-replacement mode); no single model handles both the semantic fusion and the temporal identity reinforcement.

steps

[object Object], [object Object], [object Object]

controls

one or more subject reference images, text prompt, optional driving/source video for replacement mode, dual-subject vs single-subject workflow selection

exampleStack

LLaVA text-image fusion -> HunyuanVideo DiT with image-ID enhancement -> video-driven injection module (subject replacement)

useCases

[virtual product/human advertisements with a consistent subject dropped into new video scenes][dual-subject consistent video generation][replacing a subject in existing footage with a reference identity while keeping motion]

pitfalls

dual/multi-subject workflows are more failure-prone than single-subject (identity bleed between subjects); the video-driven injection (replacement) path is sensitive to the compression/alignment of the source video and can introduce warping at subject boundaries

source

HunyuanCustom: A Multimodal-Driven Architecture for Customized Video Generation (arXiv:2505.04512)