Diffusion-Transformer Garment-Preserving Video Try-On

A video of a person and a flat garment reference image are fed into a diffusion-transformer model that disentangles garment texture/print from pose and injects it frame-by-frame, producing a video of the person wearing the new garment with consistent fabric detail and motion.

Properties

family: character-consistency
modality: person video + garment image -> garment-preserved try-on video
whyMultiModel: Garment appearance (fine texture/print) and body pose/motion are different signal types that must be extracted, disentangled and separately conditioned on; a single frame-level try-on model cannot maintain garment fidelity across motion without an explicit garment-preservation encoder plus a video-native temporal backbone with distillation for practical speed.
steps: [object Object], [object Object], [object Object]
controls: garment reference image, person video, mask-aware loss region (garment area), sampling steps (distilled to 4)
exampleStack: MagicTryOn garment encoder -> MagicTryOn DiT (Wan 2.1 backbone) -> distilled 4-step sampler
useCases: [e-commerce garment videos showing fabric drape/motion instead of static try-on stills][fashion social ads without reshoot per SKU][size/fit preview videos]
pitfalls: Fast motion or self-occlusion (arms crossing torso) causes garment texture drift across frames; front-back consistency during turns remains a known failure mode versus single-frame try-on.
source: MagicTryOn: Harnessing Diffusion Transformer for Garment-Preserving Video Virtual Try-on (arXiv:2505.21325)

Provenance

Source: https://arxiv.org/abs/2505.21325

Properties

family

character-consistency

modality

person video + garment image -> garment-preserved try-on video

whyMultiModel

Garment appearance (fine texture/print) and body pose/motion are different signal types that must be extracted, disentangled and separately conditioned on; a single frame-level try-on model cannot maintain garment fidelity across motion without an explicit garment-preservation encoder plus a video-native temporal backbone with distillation for practical speed.

steps

[object Object], [object Object], [object Object]

controls

garment reference image, person video, mask-aware loss region (garment area), sampling steps (distilled to 4)

exampleStack

MagicTryOn garment encoder -> MagicTryOn DiT (Wan 2.1 backbone) -> distilled 4-step sampler

useCases

[e-commerce garment videos showing fabric drape/motion instead of static try-on stills][fashion social ads without reshoot per SKU][size/fit preview videos]

pitfalls

Fast motion or self-occlusion (arms crossing torso) causes garment texture drift across frames; front-back consistency during turns remains a known failure mode versus single-frame try-on.

source

MagicTryOn: Harnessing Diffusion Transformer for Garment-Preserving Video Virtual Try-on (arXiv:2505.21325)