What Is Wan 2.2 Fun Control? How to Control Video Generation as Intended
In this step-by-step guide, I’ll introduce “Wan 2.2 14B Fun Control”. As a ControlNet for video, it helps you align character motion, timing, and more to your conditions. After a quick feature overview, we’ll run the official template and explain the ideas and steps behind an improved workflow.
What Is Wan 2.2 14B Fun Control?
“Wan 2.2 14B Fun Control” isn’t just for generating videos from prompts (text). By combining the “Control Codes” mechanism with multimodal conditioning inputs, it becomes much easier to generate videos that follow your specified control conditions.
Key Features of Wan 2.2 14B Fun Control
Wan 2.2 14B Fun Control supports the following control inputs.
- OpenPose（Skeleton／Pose Control）：Avatar motion control, dance motion transfer, turning motion-capture data into video, and sports movement analysis
- Depth（Depth Control）：Preserve an avatar’s silhouette, reproduce camera movement（parallax effect）, maintain spatial consistency, and create more 3D-like visuals
- Canny（Edge Control）：Preserve fine shapes/structure and details, and rewrite backgrounds
- MLSD（Geometric Line Control）：Control architectural/indoor structure, generate more stable backgrounds, and create more 3D-like visuals
- Trajectory Control（Path／Trajectory Control）：Generate video by specifying an object’s motion and path
Recommended output resolutions are as follows.
- 512：Comparable to
SD1.5 - 768：Comparable to
SD2.0 - 1024：Comparable to
SDXL
How to Use the ComfyUI Template “Wan 2.2 14B Fun Control”
Up to now I referenced ComfyUI_examples, but this time we’ll load the example from ComfyUI Templates and review it.
How to Load a ComfyUI Template
Load templates from the Templates button on the left side of the screen.
When you open Templates, you’ll see a variety of templates.
From the left menu under GENERATION TYPE, select Video.
You’ll see video-related templates, so select Wan 2.2 14B Fun Control.
When you open the template, any missing models will be listed. Download them as instructed and you can use the template as-is.
If custom nodes are missing, you’ll also see a warning. Install them via ComfyUI Manager → Install Missing Custom Nodes.
Required Models for “Wan 2.2 14B Fun Control”
This template uses the following models.
Diffusion Model (High) Diffusion Model (Low) Text Encoder VAE Lightning LoRA (High) Lightning LoRA (Low)✅A common mistake is putting the Diffusion Model into Unet. GGUF models go in Unet, but standard models like FP8 should be placed in the diffusion_models folder.
Input Assets for the ComfyUI Template “Wan 2.2 14B Fun Control”
The input image and OpenPose video are about halfway down the following page, so download them.
Nodes in the ComfyUI Template “Wan 2.2 14B Fun Control”
This template includes two workflows: a standard version and a Lightning4StepsLoRA version. ✅Right after loading, it’s set to the standard version.
Since the base structure is similar to the “Wan2.2 14B I2V” workflow, I’ll focus only on the control-related parts here.
Load the Reference Image
Load Image：Load the image that “Wan 2.2 14B Fun Control” will use as the reference.
Load the Control Video
- Load Video：Load the video used for control.
- Get Video Components：Convert the loaded video into sequential still frames.
- Canny：Extract edge lines from the loaded image.（Not used with this input）
Basic Video Settings
Wan22FunControlToVideo：Set the basic video parameters.
How to Run the ComfyUI Template “Wan 2.2 14B Fun Control”
Standard Workflow
Now load the reference image and OpenPose video you downloaded earlier into the “Load Image” and “Load Video” nodes.
Because this example uses an OpenPose video, the “Canny” node is bypassed. If you want to use Canny or Depth, convert it here instead. ✅Depending on the case, you may use the custom node “🔗comfyui_controlnet_aux”.
Run the workflow without changing any other settings.
After a moment, the character in the input image will move to match the OpenPose video.
Lightning4StepsLoRA Workflow
To use this, first right-click the Default workflow group and choose Bypass Group Nodes to bypass the whole group.
Next, right-click the Wan2.2 Fun Control fp8 scaled+ 4 steps LoRA group and choose Set Group Nodes to Always to enable the whole group.
As with the standard workflow, you won’t use the “Canny” node, so bypass it along with “Preview Image”.
Then load the reference image and OpenPose video into the “Load Image” and “Load Video” nodes to finish setup.
Compared to the standard workflow, you should be able to confirm that Lightning4StepsLoRA generates video in less time.
Results
If the hands look unnatural, it’s likely due to seed variance or because OpenPose didn’t capture the hands well enough.
How to Customize the Official Workflow
For this customization, it was more practical to split each process into separate workflows, so I divided it into three workflows. Including prep work, there are four steps in total. Here’s the OpenPose example.
- Prepare the input video：Get the video ready for conversion into a control video.
- Convert video to a control video：Create a control video from the input video with DCAI_Video2Control-OpenPose.
- Create the reference image：Generate a reference image from the control video with DCAI_Control2RefImage-OpenPose.
- Wan 2.2 14B Fun Control：Generate the video using the control video and reference image with DCAI_Wan22_14B_Fun_Control-OpenPose.
Using the person’s motion in the input video as a cue, we’re controlling an arbitrary image.
The input video was created using 🔗Amorn_mimi’s video on Pixabay as the source.
WorkflowsThis custom workflow set includes separate workflows for OpenPose, Depth, and Canny+MLSD. The workflows are published on Patreon, and only paid supporters can view and download them.
How to Create an OpenPose Control Video
Let’s start with OpenPose.
Prepare the Input Video
This time we’ll generate using 🔗Amorn_mimi’s video on Pixabay as the source. As-is, it isn’t ideal for Wan 2.2 14B Fun Control, so I adjusted it in a video editor and exported it. I’ll place the edited video on Patreon.
Next, we’ll export a control video using the workflow that converts a video into a control video.
If you adjust the input video’s size and frame rate in a video editor beforehand, you’ll get better quality.
✅Recommended export settings- File format：
MP4 - Codec：
H264 - Size：
1344×768（Optional） - Frame rate：
16FPS（Depending on the editor, it may be converted to15FPS.) - Length：
6:00
⚠️Wan2.2 Fun Control uses 16FPS. Typical videos use a variety of frame rates from 24FPS to 60FPS, but if you use a 30FPS video as-is, the generation can easily look like slow motion. If you match the video’s size and length to Wan2.2 Fun Control’s settings in advance, you can convert more efficiently with less waste.
⚠️Normally, a 5-second video at 16FPS has 79 frames. However, in Wan2.2 Fun Control, 5 seconds becomes 81 frames, so there may be no reference for the last few frames and the result can become unstable. Create the control video a bit longer—around 6 seconds.
✅If you don’t have “DaVinci Resolve” or “Adobe Media Encoder”, you can still do high-quality conversion for free using 🔗FFmpeg (it’s used internally by Adobe Media Encoder). I won’t cover the details here, but you can convert with the commands below.
ffmpeg -i input.mp4 -vf "scale=1344:768:force_original_aspect_ratio=increase,crop=1344:768,fps=16,trim=duration=6,setpts=PTS-STARTPTS" -c:v libx264 -crf 20 -pix_fmt yuv420p -an output.mp4ffmpeg -i input.mp4 -vf "scale=768:1344:force_original_aspect_ratio=increase,crop=768:1344,fps=16,trim=duration=6,setpts=PTS-STARTPTS" -c:v libx264 -crf 20 -pix_fmt yuv420p -an output.mp4Rename input.mp4 and output.mp4 to any filenames you like.
Here’s the finished video.
It’s zoomed in slightly more than the original.
Convert Video to a Control Video
Once the video is ready, open the workflow DCAI_Video2Control-OpenPose.json from Patreon.
Load the video into the “Load Video” node. When you run the workflow, an OpenPose video will be generated.
✅One thing to note here: the detect_face setting in the “DWPose Estimator” node is set to disable. For illustration-style generation, facial expressions tended to get pulled toward realism, so it’s disabled.
Here’s the finished video.
Create the Reference Image
Next, we’ll create the reference image. The closer the first frame (initial image) is to the first frame of the OpenPose video, the easier it is to get higher-quality results. So we generate OpenPose from the video’s first frame, then create the reference image from that result.
Generate the reference image using the workflow DCAI_Control2RefImage-OpenPose.json.
Load the control video into the “Load Video” node.
Load the following models.
- ControlNet Model：🔗OpenPoseXL2
- VAE：🔗sdxl.vae（Rename to
sdxl.vae.safetensorsafter downloading） - Checkpoint：🔗Nova Anime XL – IL v5B | NoobAI EPS 1.1
- LoRA：🔗DCAI Girl – NoobAI XL (ε‑Pred)
Since OpenPose wasn’t strong enough in this case, I increased the strength in the “Apply ControlNet” node to 1.50.
Then, until you get an image you like, click the New Random Seed button in the “Interactive Seed” node. Once you’re happy with the image, switch the mode in the “Control Bridge” node to Active, then click Run to run upscale and Face Detailer.
Here’s the finished image.
Now that the assets are ready, let’s generate the video.
Wan 2.2 14B Fun Control with OpenPose
Open the workflow DCAI_Wan22_14B_Fun_Control-OpenPose.json and load the image and video you generated earlier into Load Image and Load Video.
Required models and custom nodes are summarized in Markdown Note. If anything is missing, download and install it.
The prompt describes what you want the generated video to look like. In the official example, the prompt describes the situation, but you don’t need to describe the input image in great detail. It’s fine to specify only what you want to animate.
Use the “Interactive Seed” node and keep generating until you get a result you like.
If you don’t like the quality, try increasing the step count to 5 or 6 in the subgraph 👟 Steps & 🎚️ Shift.
A common starting point for High:Low is 2:2（steps4, swap_at2）, but also try 2:3（steps5, swap_at2）or 3:3（steps6, swap_at3）.
Also, it takes longer, but generating without Lightning4StepsLoRA（steps20, swap_at10）can produce higher quality.
Once you’ve generated a video you like, set the value of the “🛑 Stop Process” node to True and click Run to move on to upscaling and frame interpolation. If you don’t need that process, note that the “Preview Animation” node doesn’t save the video. Instead, connect the preceding “Color Match” node to the “Create Video” node in the Output group, then change fps to 16.
That’s it—you’ve generated a controlled animation. Below is a sample of the finished video.
How to Create a Depth Control Video
Depth and Canny in Wan22_Fun_Control aren’t ideal if your goal is to freely control a character like you can with OpenPose. Use Depth when you want to keep the original image’s silhouette while reproducing camera movement and maintaining spatial consistency.
This time we’ll generate using 🔗Dr. Samar Layek’s video on Pexels as the source.
Once the video is ready, open the workflow DCAI_Video2Control-Depth.json.
Just like with OpenPose, load the video into the “Load Video” node. When you run the workflow, a Depth video will be generated.
Next, prepare the reference image.
Generate the reference image using the workflow DCAI_Control2RefImage-Depth.json.
Wan 2.2 14B Fun Control with Depth
Open the workflow DCAI_Wan22_14B_Fun_Control-Depth.json and load the reference image and video you generated earlier into Load Image and Load Video. This workflow is the same as the OpenPose workflow; the difference is that the inputs and prompt are adjusted for Depth.
That’s it—you’ve generated a controlled animation. Below is a sample of the finished video.
How to Create a Canny+MLSD Control Video
Canny is a good fit when you want to preserve fine character shapes/structure and details—for example, when turning live-action footage into an anime-style video. In this example, we also mix in MLSD to preserve the original video’s perspective lines.
This time we’ll generate using 🔗Video on Pixabay as the source.
Once the downloaded video is ready, open the workflow DCAI_Video2Control-Canny+MLSD.json.
You can use this workflow the same way as the others. The difference is that it extracts Canny and MLSD and mixes them at 50%.
Wan 2.2 14B Fun Control with Canny+MLSD
Open the workflow DCAI_Wan22_14B_Fun_Control-Canny.json and load the image and video you generated earlier into Load Image and Load Video.
This workflow is the same as the others, so refer to the OpenPose workflow for how to use it.
That’s it—you’ve generated a controlled animation. Below is a sample of the finished video.
How to Create a Trajectory Control Video
To keep this article from getting too long, I plan to cover this in detail in a separate post.
Conclusion
“Wan 2.2 14B Fun Control” is a ControlNet that can take conditions like pose and depth in addition to text input, making it easier to steer motion and composition toward your goal. With the ComfyUI template, you can gather the required models and nodes and iterate more smoothly.
- Start with the template to understand the workflow：Select Video in Templates and load “Wan 2.2 14B Fun Control”.
- Choose the right control type for your goal：OpenPose for motion, Depth for silhouette/camera, and Canny for edges/details.
- Stabilize quality with prep work：Match the frame rate to 16FPS and keep the length around 6 seconds for easier handling.
Start by learning the full flow with the OpenPose workflow, then expand to Depth or Canny+MLSD depending on your goal.
Thank you for reading to the end.
If you found this even a little helpful, please support by giving it a “Like”!