Motion Transfer vs Image-to-Video vs Reference-to-Video: Pick the Right One
Four families of AI video model solve four different problems and are all marketed as 'AI video generation'. A decision tree, the inputs each one takes, and what each costs per second.
Most bad AI video output is not a quality problem. It is a routing problem — the wrong family of model was asked to do the job. There are four families, they take different inputs, and no amount of prompt tuning moves one into another's territory.
The four families
Image-to-video (i2v). Input: one image, optionally an audio track or a script. Output: that image, animated. It cannot invent a scene the image does not contain, and it cannot copy anyone's movement. Best for talking heads and bringing a photo to life.
Reference-to-video (ref2v). Input: several images of a subject plus a prompt. Output: a new shot containing that subject. It has no source video, so it can never reproduce a specific clip's choreography — it produces a dance, never that dance.
Motion transfer. Input: a reference video plus a character image. Output: your character performing the reference's movement. The only family that ingests motion.
Video edit (v2v). Input: the source video plus a reference image and a prompt. Output: the same video with something changed. The only family that preserves the original background, lighting and camera move.
The decision tree
- Does the movement have to match a specific clip? → motion transfer.
- Does the background have to survive? → video edit.
- Do you have a source video at all? If no → ref2v (you have references) or i2v (you have one image).
- Is there an audio track driving the performance? → i2v with native audio, or an audio-driven avatar model.
Representative endpoints and prices
Read live on 2 August 2026.
| Family | Endpoint | $/second |
|---|---|---|
| Motion transfer | fal-ai/kling-video/v3/pro/motion-control | $0.168 |
| Video edit | fal-ai/wan/v2.7/edit-video | $0.100 |
| Reference-to-video | alibaba/happy-horse/v1.1/reference-to-video | $0.140 |
| Image-to-video | alibaba/happy-horse/v1.1/image-to-video | $0.140 |
| Audio-driven avatar | fal-ai/bytedance/omnihuman/v1.5 | $0.160 |
The prices sit within a 1.7× band, so cost is almost never the right basis for choosing between families. Fit is.
Two traps worth naming
A parameter that quietly changes the family. Kling 3.0 Motion Control has a character_orientation setting. On video it behaves as motion transfer: the reference clip drives framing and the face-binding element works. On image the framing follows your still and that binding does nothing — you have effectively paid motion-transfer prices for something closer to image-to-video, and identity drifts.
Reference count. Reference-to-video models accept up to nine images (MiniMax H3 accepts more) and people hand them one. Supply front, profile and three-quarter angles. A single frontal photo gives the model nothing to work from the moment the head turns.
Duration is a quality setting
Across every family, identity holds better on short clips with restrained movement. Five to eight seconds is the range where output is reliably usable; past that, drift compounds — wardrobe shifts, faces soften, hair changes. If you need fifteen seconds, generating two shorter shots and cutting between them is usually better than asking for one long one.
The corollary: build to the family, not to the length you wish you had.