Wan2.2 Animate Character Replacement & Motion Transfer

Extracts whole-body pose and face keypoints from a source video, isolates the on-screen subject, encodes a separate reference character's identity, then fuses all three signals into a video diffusion model that replaces the on-screen performer with the reference character while preserving the original motion and expression timing. This is the current successor pattern to Animate Anyone / MimicMotion-style motion transfer.

Properties

family: character-consistency
modality: source video + character reference image -> pose/face control signals -> character-replaced video
whyMultiModel: A single video diffusion model has no mechanism to bind 'this exact pose/expression sequence' to 'this specific reference identity': a pose/face annotator (DWPose) produces the spatially-aligned skeleton and face-crop control signal the diffusion model cannot derive on its own, a segmentation pass isolates the subject mask, and a CLIP Vision encoder separately embeds the reference character's identity; only then does the Wan2.2-Animate-14B diffusion transformer fuse pose, mask, and identity embedding into the output. This is distinct from the tracked depth-pose-guided-v2v-restyle pattern (Lotus Depth/OpenPose -> LTX-2/Wan 2.1 Fun), which restyles the same on-screen person's appearance from a structure signal; here the CLIP-Vision-encoded reference image drives an actual identity swap of the on-screen subject for a different character, not a style change of the original person.
steps: [object Object], [object Object], [object Object] +1 more
controls: Mix (character replacement) vs Move (animate-from-reference) mode selection; LightX2V LoRA for 4-step acceleration; UMT5-XXL text prompt; mask preserve-background toggle; pose/face-crop conditioning strength.
exampleStack: DWPose (pose/face extraction) -> Segment Anything 2 (subject mask) -> CLIP Vision clip_vision_h (identity encode) -> Wan2.2-Animate-14B (video-to-video character replacement) -> Wan 2.1 VAE decode.
useCases: [Swapping an on-screen performer for a branded character/avatar in existing footage for ad or social content][Motion-transfer from a reference actor's performance onto a static character reference][Lip- and expression-accurate character replacement for dubbing-style localization]
pitfalls: Identity fidelity depends heavily on the quality and pose of the CLIP Vision reference image; fast motion or heavy occlusion in the source video degrades DWPose keypoint extraction, which propagates into warped limbs or misaligned expressions on the replaced character.
source: Wan2.2 Animate native workflow (ComfyUI official docs)

Provenance

Source: https://docs.comfy.org/tutorials/video/wan/wan2-2-animate

Wan2.2 Animate Character Replacement & Motion Transfer

Properties

family

character-consistency

modality

source video + character reference image -> pose/face control signals -> character-replaced video

whyMultiModel

A single video diffusion model has no mechanism to bind 'this exact pose/expression sequence' to 'this specific reference identity': a pose/face annotator (DWPose) produces the spatially-aligned skeleton and face-crop control signal the diffusion model cannot derive on its own, a segmentation pass isolates the subject mask, and a CLIP Vision encoder separately embeds the reference character's identity; only then does the Wan2.2-Animate-14B diffusion transformer fuse pose, mask, and identity embedding into the output. This is distinct from the tracked depth-pose-guided-v2v-restyle pattern (Lotus Depth/OpenPose -> LTX-2/Wan 2.1 Fun), which restyles the same on-screen person's appearance from a structure signal; here the CLIP-Vision-encoded reference image drives an actual identity swap of the on-screen subject for a different character, not a style change of the original person.

steps

[object Object], [object Object], [object Object] +1 more

controls

Mix (character replacement) vs Move (animate-from-reference) mode selection; LightX2V LoRA for 4-step acceleration; UMT5-XXL text prompt; mask preserve-background toggle; pose/face-crop conditioning strength.

exampleStack

DWPose (pose/face extraction) -> Segment Anything 2 (subject mask) -> CLIP Vision clip_vision_h (identity encode) -> Wan2.2-Animate-14B (video-to-video character replacement) -> Wan 2.1 VAE decode.

useCases

[Swapping an on-screen performer for a branded character/avatar in existing footage for ad or social content][Motion-transfer from a reference actor's performance onto a static character reference][Lip- and expression-accurate character replacement for dubbing-style localization]

pitfalls

Identity fidelity depends heavily on the quality and pose of the CLIP Vision reference image; fast motion or heavy occlusion in the source video degrades DWPose keypoint extraction, which propagates into warped limbs or misaligned expressions on the replaced character.

source

Wan2.2 Animate native workflow (ComfyUI official docs)