InfiniteYou Recraft to Animated Portrait

ByteDance InfiniteYou injects identity features through InfuseNet residual connections into a FLUX DiT to recraft a person's photo under a new prompt/pose/style while preserving likeness, then a downstream image-to-video model animates the recrafted still.

Properties

family: character-consistency
modality: single face photo + text -> identity-recrafted photo -> animated clip
whyMultiModel: InfuseNet's identity injection is specific to still-image DiT generation and carries no temporal component; a distinct video diffusion backbone is required to add motion, and InfiniteYou's own docs describe it as plug-and-play specifically for composing with other methods/backbones.
steps: [object Object], [object Object], [object Object]
controls: identity reference image, text prompt for pose/scene/style, base model swap (dev vs schnell) for quality/speed tradeoff, SFT stage controlling text-image alignment strength
exampleStack: InfiniteYou (InfuseNet) + FLUX.1 dev -> Wan 2.2 image-to-video
useCases: [personalized avatar photo then short animated intro clip][recrafting a single reference photo into varied scenes before turning into a talking/moving asset][identity-locked marketing creative across image and short-video formats]
pitfalls: identity similarity vs. text-prompt alignment is a direct tradeoff; pushing pose/style far from the reference photo degrades likeness, and any facial artifact in the recrafted still propagates and amplifies once animated
source: InfiniteYou: Flexible Photo Recrafting While Preserving Your Identity (arXiv:2503.16418)

Provenance

Source: https://arxiv.org/abs/2503.16418

Properties

family

character-consistency

modality

single face photo + text -> identity-recrafted photo -> animated clip

whyMultiModel

InfuseNet's identity injection is specific to still-image DiT generation and carries no temporal component; a distinct video diffusion backbone is required to add motion, and InfiniteYou's own docs describe it as plug-and-play specifically for composing with other methods/backbones.

steps

[object Object], [object Object], [object Object]

controls

identity reference image, text prompt for pose/scene/style, base model swap (dev vs schnell) for quality/speed tradeoff, SFT stage controlling text-image alignment strength

exampleStack

InfiniteYou (InfuseNet) + FLUX.1 dev -> Wan 2.2 image-to-video

useCases

[personalized avatar photo then short animated intro clip][recrafting a single reference photo into varied scenes before turning into a talking/moving asset][identity-locked marketing creative across image and short-video formats]

pitfalls

identity similarity vs. text-prompt alignment is a direct tradeoff; pushing pose/style far from the reference photo degrades likeness, and any facial artifact in the recrafted still propagates and amplifies once animated

source

InfiniteYou: Flexible Photo Recrafting While Preserving Your Identity (arXiv:2503.16418)