comi-logic/Wan2GP
7
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

better Readme structure

Browse Source
This commit is contained in:
seko2521
2025-05-27 11:15:40 +02:00
parent c9aa2c3230
commit 728a5b4535
9 changed files with 1765 additions and 404 deletions
Download Patch File Download Diff File Expand all files Collapse all files

148
docs/CHANGELOG.md Normal file
View File

@@ -0,0 +1,148 @@
# Changelog
## 🔥 Latest News
### May 26, 2025: Wan 2.1GP v5.3
👋 Happy with a Video generation and want to do more generations using the same settings but you can't remember what you did or you find it too hard to copy/paste one per one each setting from the file metadata? Rejoice! There are now multiple ways to turn this tedious process into a one click task:
- Select one Video recently generated in the Video Gallery and click *Use Selected Video Settings*
- Click *Drop File Here* and select a Video you saved somewhere, if the settings metadata have been saved with the Video you will be able to extract them automatically
- Click *Export Settings to File* to save on your harddrive the current settings. You will be able to use them later again by clicking *Drop File Here* and select this time a Settings json file
### May 23, 2025: Wan 2.1GP v5.21
👋 Improvements for Vace: better transitions between Sliding Windows, Support for Image masks in Matanyone, new Extend Video for Vace, different types of automated background removal
### May 20, 2025: Wan 2.1GP v5.2
👋 Added support for Wan CausVid which is a distilled Wan model that can generate nice looking videos in only 4 to 12 steps. The great thing is that Kijai (Kudos to him!) has created a CausVid Lora that can be combined with any existing Wan t2v model 14B like Wan Vace 14B. See [LORAS.md](LORAS.md) for instructions on how to use CausVid.
Also as an experiment I have added support for the MoviiGen, the first model that claims to be capable of generating 1080p videos (if you have enough VRAM (20GB...) and be ready to wait for a long time...). Don't hesitate to share your impressions on the Discord server.
### May 18, 2025: Wan 2.1GP v5.1
👋 Bonus Day, added LTX Video 13B Distilled: generate in less than one minute, very high quality Videos!
### May 17, 2025: Wan 2.1GP v5.0
👋 One App to Rule Them All! Added support for the other great open source architectures:
- **Hunyuan Video**: text 2 video (one of the best, if not the best t2v), image 2 video and the recently released Hunyuan Custom (very good identity preservation when injecting a person into a video)
- **LTX Video 13B** (released last week): very long video support and fast 720p generation. Wan GP version has been greatly optimized and reduced LTX Video VRAM requirements by 4!
Also:
- Added support for the best Control Video Model, released 2 days ago: Vace 14B
- New Integrated prompt enhancer to increase the quality of the generated videos
*You will need one more `pip install -r requirements.txt`*
### May 5, 2025: Wan 2.1GP v4.5
👋 FantasySpeaking model, you can animate a talking head using a voice track. This works not only on people but also on objects. Also better seamless transitions between Vace sliding windows for very long videos. New high quality processing features (mixed 16/32 bits calculation and 32 bits VAE)
### April 27, 2025: Wan 2.1GP v4.4
👋 Phantom model support, very good model to transfer people or objects into video, works quite well at 720p and with the number of steps > 30
### April 25, 2025: Wan 2.1GP v4.3
👋 Added preview mode and support for Sky Reels v2 Diffusion Forcing for high quality "infinite length videos". Note that Skyreel uses causal attention that is only supported by Sdpa attention so even if you choose another type of attention, some of the processes will use Sdpa attention.
### April 18, 2025: Wan 2.1GP v4.2
👋 FLF2V model support, official support from Wan for image2video start and end frames specialized for 720p.
### April 17, 2025: Wan 2.1GP v4.1
👋 Recam Master model support, view a video from a different angle. The video to process must be at least 81 frames long and you should set at least 15 steps denoising to get good results.
### April 13, 2025: Wan 2.1GP v4.0
👋 Lots of goodies for you!
- A new UI, tabs were replaced by a Dropdown box to easily switch models
- A new queuing system that lets you stack in a queue as many text2video, image2video tasks, ... as you want. Each task can rely on complete different generation parameters (different number of frames, steps, loras, ...). Many thanks to **Tophness** for being a big contributor on this new feature
- Temporal upsampling (Rife) and spatial upsampling (Lanczos) for a smoother video (32 fps or 64 fps) and to enlarge your video by x2 or x4. Check these new advanced options.
- Wan Vace Control Net support: with Vace you can inject in the scene people or objects, animate a person, perform inpainting or outpainting, continue a video, ... See [VACE.md](VACE.md) for introduction guide.
- Integrated *Matanyone* tool directly inside WanGP so that you can create easily inpainting masks used in Vace
- Sliding Window generation for Vace, create windows that can last dozens of seconds
- New optimizations for old generation GPUs: Generate 5s (81 frames, 15 steps) of Vace 1.3B with only 5GB and in only 6 minutes on a RTX 2080Ti and 5s of t2v 14B in less than 10 minutes.
### March 27, 2025
👋 Added support for the new Wan Fun InP models (image2video). The 14B Fun InP has probably better end image support but unfortunately existing loras do not work so well with it. The great novelty is the Fun InP image2 1.3B model: Image 2 Video is now accessible to even lower hardware configuration. It is not as good as the 14B models but very impressive for its size. Many thanks to the VideoX-Fun team (https://github.com/aigc-apps/VideoX-Fun)
### March 26, 2025
👋 Good news! Official support for RTX 50xx please check the [installation instructions](INSTALLATION.md).
### March 24, 2025: Wan2.1GP v3.2
👋
- Added Classifier-Free Guidance Zero Star. The video should match better the text prompt (especially with text2video) at no performance cost: many thanks to the **CFG Zero * Team**. Don't hesitate to give them a star if you appreciate the results: https://github.com/WeichenFan/CFG-Zero-star
- Added back support for PyTorch compilation with Loras. It seems it had been broken for some time
- Added possibility to keep a number of pregenerated videos in the Video Gallery (useful to compare outputs of different settings)
*You will need one more `pip install -r requirements.txt`*
### March 19, 2025: Wan2.1GP v3.1
👋 Faster launch and RAM optimizations (should require less RAM to run)
*You will need one more `pip install -r requirements.txt`*
### March 18, 2025: Wan2.1GP v3.0
👋
- New Tab based interface, you can switch from i2v to t2v conversely without restarting the app
- Experimental Dual Frames mode for i2v, you can also specify an End frame. It doesn't always work, so you will need a few attempts.
- You can save default settings in the files *i2v_settings.json* and *t2v_settings.json* that will be used when launching the app (you can also specify the path to different settings files)
- Slight acceleration with loras
*You will need one more `pip install -r requirements.txt`*
Many thanks to *Tophness* who created the framework (and did a big part of the work) of the multitabs and saved settings features
### March 18, 2025: Wan2.1GP v2.11
👋 Added more command line parameters to prefill the generation settings + customizable output directory and choice of type of metadata for generated videos. Many thanks to *Tophness* for his contributions.
*You will need one more `pip install -r requirements.txt` to reflect new dependencies*
### March 18, 2025: Wan2.1GP v2.1
👋 More Loras!: added support for 'Safetensors' and 'Replicate' Lora formats.
*You will need to refresh the requirements with a `pip install -r requirements.txt`*
### March 17, 2025: Wan2.1GP v2.0
👋 The Lora festival continues:
- Clearer user interface
- Download 30 Loras in one click to try them all (expand the info section)
- Very easy to use Loras as now Lora presets can input the subject (or other needed terms) of the Lora so that you don't have to modify manually a prompt
- Added basic macro prompt language to prefill prompts with different values. With one prompt template, you can generate multiple prompts.
- New Multiple images prompts: you can now combine any number of images with any number of text prompts (need to launch the app with --multiple-images)
- New command lines options to launch directly the 1.3B t2v model or the 14B t2v model
### March 14, 2025: Wan2.1GP v1.7
👋
- Lora Fest special edition: very fast loading/unload of loras for those Loras collectors around. You can also now add/remove loras in the Lora folder without restarting the app.
- Added experimental Skip Layer Guidance (advanced settings), that should improve the image quality at no extra cost. Many thanks to the *AmericanPresidentJimmyCarter* for the original implementation
*You will need to refresh the requirements `pip install -r requirements.txt`*
### March 13, 2025: Wan2.1GP v1.6
👋 Better Loras support, accelerated loading Loras.
*You will need to refresh the requirements `pip install -r requirements.txt`*
### March 10, 2025: Wan2.1GP v1.5
👋 Official Teacache support + Smart Teacache (find automatically best parameters for a requested speed multiplier), 10% speed boost with no quality loss, improved lora presets (they can now include prompts and comments to guide the user)
### March 7, 2025: Wan2.1GP v1.4
👋 Fix PyTorch compilation, now it is really 20% faster when activated
### March 4, 2025: Wan2.1GP v1.3
👋 Support for Image to Video with multiples images for different images/prompts combinations (requires *--multiple-images* switch), and added command line *--preload x* to preload in VRAM x MB of the main diffusion model if you find there is too much unused VRAM and you want to (slightly) accelerate the generation process.
*If you upgrade you will need to do a `pip install -r requirements.txt` again.*
### March 4, 2025: Wan2.1GP v1.2
👋 Implemented tiling on VAE encoding and decoding. No more VRAM peaks at the beginning and at the end
### March 3, 2025: Wan2.1GP v1.1
👋 Added Tea Cache support for faster generations: optimization of kijai's implementation (https://github.com/kijai/ComfyUI-WanVideoWrapper/) of teacache (https://github.com/ali-vilab/TeaCache)
### March 2, 2025: Wan2.1GP by DeepBeepMeep v1
👋 Brings:
- Support for all Wan including the Image to Video model
- Reduced memory consumption by 2, with possibility to generate more than 10s of video at 720p with a RTX 4090 and 10s of video at 480p with less than 12GB of VRAM. Many thanks to REFLEx (https://github.com/thu-ml/RIFLEx) for their algorithm that allows generating nice looking video longer than 5s.
- The usual perks: web interface, multiple generations, loras support, sage attention, auto download of models, ...
## Original Wan Releases
### February 25, 2025
👋 We've released the inference code and weights of Wan2.1.
### February 27, 2025
👋 Wan2.1 has been integrated into [ComfyUI](https://comfyanonymous.github.io/ComfyUI_examples/wan/). Enjoy!

242
docs/CLI.md Normal file
View File

@@ -0,0 +1,242 @@
# Command Line Reference
This document covers all available command line options for WanGP.
## Basic Usage
```bash
# Default launch (text-to-video)
python wgp.py
# Specific model modes
python wgp.py --i2v # Image-to-video
python wgp.py --t2v # Text-to-video (default)
python wgp.py --t2v-14B # 14B text-to-video model
python wgp.py --t2v-1-3B # 1.3B text-to-video model
python wgp.py --i2v-14B # 14B image-to-video model
python wgp.py --i2v-1-3B # Fun InP 1.3B image-to-video model
python wgp.py --vace # VACE ControlNet 1.3B model
```
## Model and Performance Options
### Model Configuration
```bash
--quantize-transformer BOOL # Enable/disable transformer quantization (default: True)
--compile # Enable PyTorch compilation (requires Triton)
--attention MODE # Force attention mode: sdpa, flash, sage, sage2
--profile NUMBER # Performance profile 1-5 (default: 4)
--preload NUMBER # Preload N MB of diffusion model in VRAM
--fp16 # Force fp16 instead of bf16 models
--gpu DEVICE # Run on specific GPU device (e.g., "cuda:1")
```
### Performance Profiles
- **Profile 1**: Maximum RAM usage, minimum VRAM
- **Profile 2**: Balanced RAM/VRAM usage
- **Profile 3 (LowRAM_HighVRAM)**: Load entire model in VRAM (requires 24GB for 14B model)
- **Profile 4 (LowRAM_LowVRAM)**: Default, load model parts as needed
- **Profile 5**: Minimum RAM usage
### Memory Management
```bash
--perc-reserved-mem-max FLOAT # Max percentage of RAM for reserved memory (< 0.5)
```
## Lora Configuration
```bash
--lora-dir PATH # Path to Wan t2v loras directory
--lora-dir-i2v PATH # Path to Wan i2v loras directory
--lora-dir-hunyuan PATH # Path to Hunyuan t2v loras directory
--lora-dir-hunyuan-i2v PATH # Path to Hunyuan i2v loras directory
--lora-dir-ltxv PATH # Path to LTX Video loras directory
--lora-preset PRESET # Load lora preset file (.lset) on startup
--check-loras # Filter incompatible loras (slower startup)
```
## Generation Settings
### Basic Generation
```bash
--seed NUMBER # Set default seed value
--frames NUMBER # Set default number of frames to generate
--steps NUMBER # Set default number of denoising steps
--advanced # Launch with advanced mode enabled
```
### Advanced Generation
```bash
--teacache MULTIPLIER # TeaCache speed multiplier: 0, 1.5, 1.75, 2.0, 2.25, 2.5
--slg # Enable Skip Layer Guidance for improved quality
```
## Interface and Server Options
### Server Configuration
```bash
--server-port PORT # Gradio server port (default: 7860)
--server-name NAME # Gradio server name (default: localhost)
--listen # Make server accessible on network
--share # Create shareable HuggingFace URL for remote access
--open-browser # Open browser automatically when launching
```
### Interface Options
```bash
--multiple-images # Allow multiple image inputs for different starting points
--lock-config # Prevent modifying video engine configuration from interface
--theme THEME_NAME # UI theme: "default" or "gradio"
```
## File and Directory Options
```bash
--settings PATH # Path to folder containing default settings for all models
--verbose LEVEL # Information level 0-2 (default: 1)
```
## Examples
### Basic Usage Examples
```bash
# Launch with specific model and loras
python wgp.py --t2v-14B --lora-preset mystyle.lset
# High-performance setup with compilation
python wgp.py --compile --attention sage2 --profile 3
# Low VRAM setup
python wgp.py --t2v-1-3B --profile 4 --attention sdpa
# Multiple images with custom lora directory
python wgp.py --i2v --multiple-images --lora-dir /path/to/shared/loras
```
### Server Configuration Examples
```bash
# Network accessible server
python wgp.py --listen --server-port 8080
# Shareable server with custom theme
python wgp.py --share --theme gradio --open-browser
# Locked configuration for public use
python wgp.py --lock-config --share
```
### Advanced Performance Examples
```bash
# Maximum performance (requires high-end GPU)
python wgp.py --compile --attention sage2 --profile 3 --preload 2000
# Optimized for RTX 2080Ti
python wgp.py --profile 4 --attention sdpa --teacache 2.0
# Memory-efficient setup
python wgp.py --fp16 --profile 4 --perc-reserved-mem-max 0.3
```
### TeaCache Configuration
```bash
# Different speed multipliers
python wgp.py --teacache 1.5 # 1.5x speed, minimal quality loss
python wgp.py --teacache 2.0 # 2x speed, some quality loss
python wgp.py --teacache 2.5 # 2.5x speed, noticeable quality loss
python wgp.py --teacache 0 # Disable TeaCache
```
## Attention Modes
### SDPA (Default)
```bash
python wgp.py --attention sdpa
```
- Available by default with PyTorch
- Good compatibility with all GPUs
- Moderate performance
### Sage Attention
```bash
python wgp.py --attention sage
```
- Requires Triton installation
- 30% faster than SDPA
- Small quality cost
### Sage2 Attention
```bash
python wgp.py --attention sage2
```
- Requires Triton and SageAttention 2.x
- 40% faster than SDPA
- Best performance option
### Flash Attention
```bash
python wgp.py --attention flash
```
- May require CUDA kernel compilation
- Good performance
- Can be complex to install on Windows
## Troubleshooting Command Lines
### Fallback to Basic Setup
```bash
# If advanced features don't work
python wgp.py --attention sdpa --profile 4 --fp16
```
### Debug Mode
```bash
# Maximum verbosity for troubleshooting
python wgp.py --verbose 2 --check-loras
```
### Memory Issue Debugging
```bash
# Minimal memory usage
python wgp.py --profile 4 --attention sdpa --perc-reserved-mem-max 0.2
```
### GPU-specific Configurations
```bash
# RTX 10XX/20XX series
python wgp.py --attention sdpa --profile 4 --fp16
# RTX 30XX/40XX series
python wgp.py --attention sage --compile --profile 3
# RTX 50XX series (beta)
python wgp.py --attention sage --fp16 --profile 4
```
## Configuration Files
### Settings Files
You can save default settings in JSON files:
- `i2v_settings.json` - Image-to-video default settings
- `t2v_settings.json` - Text-to-video default settings
Load custom settings:
```bash
python wgp.py --settings /path/to/settings/folder
```
### Lora Presets
Create and share lora configurations:
```bash
# Load specific preset
python wgp.py --lora-preset anime_style.lset
# With custom lora directory
python wgp.py --lora-preset mystyle.lset --lora-dir /shared/loras
```
## Environment Variables
While not command line options, these environment variables can affect behavior:
- `CUDA_VISIBLE_DEVICES` - Limit visible GPUs
- `PYTORCH_CUDA_ALLOC_CONF` - CUDA memory allocation settings
- `TRITON_CACHE_DIR` - Triton cache directory (for Sage attention)

194
docs/GETTING_STARTED.md Normal file
View File

@@ -0,0 +1,194 @@
# Getting Started with WanGP
This guide will help you get started with WanGP video generation quickly and easily.
## Prerequisites
Before starting, ensure you have:
- A compatible GPU (RTX 10XX or newer recommended)
- Python 3.10.9 installed
- At least 6GB of VRAM for basic models
- Internet connection for model downloads
## Quick Setup
### Option 1: One-Click Installation (Recommended)
Use [Pinokio App](https://pinokio.computer/) for the easiest installation experience.
### Option 2: Manual Installation
```bash
git clone https://github.com/deepbeepmeep/Wan2GP.git
cd Wan2GP
conda create -n wan2gp python=3.10.9
conda activate wan2gp
pip install torch==2.6.0 torchvision torchaudio --index-url https://download.pytorch.org/whl/test/cu124
pip install -r requirements.txt
```
For detailed installation instructions, see [INSTALLATION.md](INSTALLATION.md).
## First Launch
### Basic Launch
```bash
python wgp.py
```
This launches the text-to-video generator with default settings.
### Alternative Modes
```bash
python wgp.py --i2v # Image-to-video mode
python wgp.py --t2v-1-3B # Smaller, faster model
```
## Understanding the Interface
When you launch WanGP, you'll see a web interface with several sections:
### Main Generation Panel
- **Model Selection**: Dropdown to choose between different models
- **Prompt**: Text description of what you want to generate
- **Generate Button**: Start the video generation process
### Advanced Settings (click checkbox to enable)
- **Generation Settings**: Steps, guidance, seeds
- **Loras**: Additional style customizations
- **Sliding Window**: For longer videos
## Your First Video
Let's generate a simple text-to-video:
1. **Launch WanGP**: `python wgp.py`
2. **Open Browser**: Navigate to `http://localhost:7860`
3. **Enter Prompt**: "A cat walking in a garden"
4. **Click Generate**: Wait for the video to be created
5. **View Result**: The video will appear in the output section
### Recommended First Settings
- **Model**: Wan 2.1 text2video 1.3B (faster, lower VRAM)
- **Frames**: 49 (about 2 seconds)
- **Steps**: 20 (good balance of speed/quality)
## Model Selection
### Text-to-Video Models
- **Wan 2.1 T2V 1.3B**: Fastest, lowest VRAM (6GB), good quality
- **Wan 2.1 T2V 14B**: Best quality, requires more VRAM (12GB+)
- **Hunyuan Video**: Excellent quality, slower generation
- **LTX Video**: Good for longer videos
### Image-to-Video Models
- **Wan Fun InP 1.3B**: Fast image animation
- **Wan Fun InP 14B**: Higher quality image animation
- **VACE**: Advanced control over video generation
### Choosing the Right Model
- **Low VRAM (6-8GB)**: Use 1.3B models
- **Medium VRAM (10-12GB)**: Use 14B models or Hunyuan
- **High VRAM (16GB+)**: Any model, longer videos
## Basic Settings Explained
### Generation Settings
- **Frames**: Number of frames (more = longer video)
- 25 frames ≈ 1 second
- 49 frames ≈ 2 seconds
- 73 frames ≈ 3 seconds
- **Steps**: Quality vs Speed tradeoff
- 15 steps: Fast, lower quality
- 20 steps: Good balance
- 30+ steps: High quality, slower
- **Guidance Scale**: How closely to follow the prompt
- 3-5: More creative interpretation
- 7-10: Closer to prompt description
- 12+: Very literal interpretation
### Seeds
- **Random Seed**: Different result each time
- **Fixed Seed**: Reproducible results
- **Use same seed + prompt**: Generate variations
## Common Beginner Issues
### "Out of Memory" Errors
1. Use smaller models (1.3B instead of 14B)
2. Reduce frame count
3. Lower resolution in advanced settings
4. Enable quantization (usually on by default)
### Slow Generation
1. Use 1.3B models for speed
2. Reduce number of steps
3. Install Sage attention (see [INSTALLATION.md](INSTALLATION.md))
4. Enable TeaCache: `python wgp.py --teacache 2.0`
### Poor Quality Results
1. Increase number of steps (25-30)
2. Improve prompt description
3. Use 14B models if you have enough VRAM
4. Enable Skip Layer Guidance in advanced settings
## Writing Good Prompts
### Basic Structure
```
[Subject] [Action] [Setting] [Style/Quality modifiers]
```
### Examples
```
A red sports car driving through a mountain road at sunset, cinematic, high quality
A woman with long hair walking on a beach, waves in the background, realistic, detailed
A cat sitting on a windowsill watching rain, cozy atmosphere, soft lighting
```
### Tips
- Be specific about what you want
- Include style descriptions (cinematic, realistic, etc.)
- Mention lighting and atmosphere
- Describe the setting in detail
- Use quality modifiers (high quality, detailed, etc.)
## Next Steps
Once you're comfortable with basic generation:
1. **Explore Advanced Features**:
- [Loras Guide](LORAS.md) - Customize styles and characters
- [VACE ControlNet](VACE.md) - Advanced video control
- [Command Line Options](CLI.md) - Optimize performance
2. **Improve Performance**:
- Install better attention mechanisms
- Optimize memory settings
- Use compilation for speed
3. **Join the Community**:
- [Discord Server](https://discord.gg/g7efUW9jGV) - Get help and share videos
- Share your best results
- Learn from other users
## Troubleshooting First Steps
### Installation Issues
- Ensure Python 3.10.9 is used
- Check CUDA version compatibility
- See [INSTALLATION.md](INSTALLATION.md) for detailed steps
### Generation Issues
- Check GPU compatibility
- Verify sufficient VRAM
- Try basic settings first
- See [TROUBLESHOOTING.md](TROUBLESHOOTING.md) for specific issues
### Performance Issues
- Use appropriate model for your hardware
- Enable performance optimizations
- Check [CLI.md](CLI.md) for optimization flags
Remember: Start simple and gradually explore more advanced features as you become comfortable with the basics!

170
docs/INSTALLATION.md Normal file
View File

@@ -0,0 +1,170 @@
# Installation Guide
This guide covers installation for different GPU generations and operating systems.
## Requirements
- Python 3.10.9
- Conda or Python venv
- Compatible GPU (RTX 10XX or newer recommended)
## Installation for RTX 10XX to RTX 40XX (Stable)
This installation uses PyTorch 2.6.0 which is well-tested and stable.
### Step 1: Download and Setup Environment
```shell
# Clone the repository
git clone https://github.com/deepbeepmeep/Wan2GP.git
cd Wan2GP
# Create Python 3.10.9 environment using conda
conda create -n wan2gp python=3.10.9
conda activate wan2gp
```
### Step 2: Install PyTorch
```shell
# Install PyTorch 2.6.0 with CUDA 12.4
pip install torch==2.6.0 torchvision torchaudio --index-url https://download.pytorch.org/whl/test/cu124
```
### Step 3: Install Dependencies
```shell
# Install core dependencies
pip install -r requirements.txt
```
### Step 4: Optional Performance Optimizations
#### Sage Attention (30% faster)
```shell
# Windows only: Install Triton
pip install triton-windows
# For both Windows and Linux
pip install sageattention==1.0.6
```
#### Sage 2 Attention (40% faster)
```shell
# Windows
pip install triton-windows
pip install https://github.com/woct0rdho/SageAttention/releases/download/v2.1.1-windows/sageattention-2.1.1+cu126torch2.6.0-cp310-cp310-win_amd64.whl
# Linux (manual compilation required)
git clone https://github.com/thu-ml/SageAttention
cd SageAttention
pip install -e .
```
#### Flash Attention
```shell
# May require CUDA kernel compilation on Windows
pip install flash-attn==2.7.2.post1
```
## Installation for RTX 50XX (Beta)
RTX 50XX GPUs require PyTorch 2.7.0 (beta). This version may be less stable.
⚠️ **Important:** Use Python 3.10 for compatibility with pip wheels.
### Step 1: Setup Environment
```shell
# Clone and setup (same as above)
git clone https://github.com/deepbeepmeep/Wan2GP.git
cd Wan2GP
conda create -n wan2gp python=3.10.9
conda activate wan2gp
```
### Step 2: Install PyTorch Beta
```shell
# Install PyTorch 2.7.0 with CUDA 12.8
pip install torch==2.7.0 torchvision torchaudio --index-url https://download.pytorch.org/whl/test/cu128
```
### Step 3: Install Dependencies
```shell
pip install -r requirements.txt
```
### Step 4: Optional Optimizations for RTX 50XX
#### Sage Attention
```shell
# Windows
pip install triton-windows
pip install sageattention==1.0.6
# Linux
pip install sageattention==1.0.6
```
#### Sage 2 Attention
```shell
# Windows
pip install triton-windows
pip install https://github.com/woct0rdho/SageAttention/releases/download/v2.1.1-windows/sageattention-2.1.1+cu128torch2.7.0-cp310-cp310-win_amd64.whl
# Linux (manual compilation)
git clone https://github.com/thu-ml/SageAttention
cd SageAttention
pip install -e .
```
## Attention Modes
WanGP supports several attention implementations:
- **SDPA** (default): Available by default with PyTorch
- **Sage**: 30% speed boost with small quality cost
- **Sage2**: 40% speed boost
- **Flash**: Good performance, may be complex to install on Windows
## Performance Profiles
Choose a profile based on your hardware:
- **Profile 3 (LowRAM_HighVRAM)**: Loads entire model in VRAM, requires 24GB VRAM for 8-bit quantized 14B model
- **Profile 4 (LowRAM_LowVRAM)**: Default, loads model parts as needed, slower but lower VRAM requirement
## Troubleshooting
### Sage Attention Issues
If Sage attention doesn't work:
1. Check if Triton is properly installed
2. Clear Triton cache
3. Fallback to SDPA attention:
```bash
python wgp.py --attention sdpa
```
### Memory Issues
- Use lower resolution or shorter videos
- Enable quantization (default)
- Use Profile 4 for lower VRAM usage
- Consider using 1.3B models instead of 14B models
### GPU Compatibility
- RTX 10XX, 20XX: Supported with SDPA attention
- RTX 30XX, 40XX: Full feature support
- RTX 50XX: Beta support with PyTorch 2.7.0
For more troubleshooting, see [TROUBLESHOOTING.md](TROUBLESHOOTING.md)

197
docs/LORAS.md Normal file
View File

@@ -0,0 +1,197 @@
# Loras Guide
Loras (Low-Rank Adaptations) allow you to customize video generation models by adding specific styles, characters, or effects to your videos.
## Directory Structure
Loras are organized in different folders based on the model they're designed for:
### Text-to-Video Models
- `loras/` - General t2v loras
- `loras/1.3B/` - Loras specifically for 1.3B models
- `loras/14B/` - Loras specifically for 14B models
### Image-to-Video Models
- `loras_i2v/` - Image-to-video loras
### Other Models
- `loras_hunyuan/` - Hunyuan Video t2v loras
- `loras_hunyuan_i2v/` - Hunyuan Video i2v loras
- `loras_ltxv/` - LTX Video loras
## Custom Lora Directory
You can specify custom lora directories when launching the app:
```bash
# Use shared lora directory for both t2v and i2v
python wgp.py --lora-dir /path/to/shared/loras --lora-dir-i2v /path/to/shared/loras
# Specify different directories for different models
python wgp.py --lora-dir-hunyuan /path/to/hunyuan/loras --lora-dir-ltxv /path/to/ltx/loras
```
## Using Loras
### Basic Usage
1. Place your lora files in the appropriate directory
2. Launch WanGP
3. In the Advanced Tab, select the "Loras" section
4. Check the loras you want to activate
5. Set multipliers for each lora (default is 1.0)
### Lora Multipliers
Multipliers control the strength of each lora's effect:
#### Simple Multipliers
```
1.2 0.8
```
- First lora: 1.2 strength
- Second lora: 0.8 strength
#### Time-based Multipliers
For dynamic effects over generation steps, use comma-separated values:
```
0.9,0.8,0.7
1.2,1.1,1.0
```
- For 30 steps: steps 0-9 use first value, 10-19 use second, 20-29 use third
- First lora: 0.9 → 0.8 → 0.7
- Second lora: 1.2 → 1.1 → 1.0
## Lora Presets
Presets are combinations of loras with predefined multipliers and prompts.
### Creating Presets
1. Configure your loras and multipliers
2. Write a prompt with comments (lines starting with #)
3. Save as a preset with `.lset` extension
### Example Preset
```
# Use the keyword "ohnvx" to trigger the lora
A ohnvx character is driving a car through the city
```
### Using Presets
```bash
# Load preset on startup
python wgp.py --lora-preset mypreset.lset
```
### Managing Presets
- Edit, save, or delete presets directly from the web interface
- Presets include comments with usage instructions
- Share `.lset` files with other users
## CausVid Lora (Special)
CausVid is a distilled Wan model that generates videos in 4-12 steps with 2x speed improvement.
### Setup Instructions
1. Download the CausVid Lora:
```
https://huggingface.co/Kijai/WanVideo_comfy/blob/main/Wan21_CausVid_14B_T2V_lora_rank32.safetensors
```
2. Place in your `loras/` directory
### Usage
1. Select a Wan t2v model (e.g., Wan 2.1 text2video 13B or Vace 13B)
2. Enable Advanced Mode
3. In Advanced Generation Tab:
- Set Guidance Scale = 1
- Set Shift Scale = 7
4. In Advanced Lora Tab:
- Select CausVid Lora
- Set multiplier to 0.3
5. Set generation steps to 12
6. Generate!
### CausVid Step/Multiplier Relationship
- **12 steps**: 0.3 multiplier (recommended)
- **8 steps**: 0.5-0.7 multiplier
- **4 steps**: 0.8-1.0 multiplier
*Note: Lower steps = lower quality (especially motion)*
## Supported Formats
WanGP supports multiple lora formats:
- **Safetensors** (.safetensors)
- **Replicate** format
- **Standard PyTorch** (.pt, .pth)
## Performance Tips
### Fast Loading/Unloading
- Loras can be added/removed without restarting the app
- Use the "Refresh" button to detect new loras
- Enable `--check-loras` to filter incompatible loras (slower startup)
### Memory Management
- Loras are loaded on-demand to save VRAM
- Multiple loras can be used simultaneously
- Time-based multipliers don't use extra memory
## Finding Loras
### Sources
- **[Civitai](https://civitai.com/)** - Large community collection
- **HuggingFace** - Official and community loras
- **Discord Server** - Community recommendations
### Creating Loras
- **Kohya** - Popular training tool
- **OneTrainer** - Alternative training solution
- **Custom datasets** - Train on your own content
## Macro System (Advanced)
Create multiple prompts from templates using macros:
```
! {Subject}="cat","woman","man", {Location}="forest","lake","city", {Possessive}="its","her","his"
In the video, a {Subject} is presented. The {Subject} is in a {Location} and looks at {Possessive} watch.
```
This generates:
1. "In the video, a cat is presented. The cat is in a forest and looks at its watch."
2. "In the video, a woman is presented. The woman is in a lake and looks at her watch."
3. "In the video, a man is presented. The man is in a city and looks at his watch."
## Troubleshooting
### Lora Not Working
1. Check if lora is compatible with your model size (1.3B vs 14B)
2. Verify lora format is supported
3. Try different multiplier values
4. Check the lora was trained for your model type (t2v vs i2v)
### Performance Issues
1. Reduce number of active loras
2. Lower multiplier values
3. Use `--check-loras` to filter incompatible files
4. Clear lora cache if issues persist
### Memory Errors
1. Use fewer loras simultaneously
2. Reduce model size (use 1.3B instead of 14B)
3. Lower video resolution or frame count
4. Enable quantization if not already active
## Command Line Options
```bash
# Lora-related command line options
--lora-dir path # Path to t2v loras directory
--lora-dir-i2v path # Path to i2v loras directory
--lora-dir-hunyuan path # Path to Hunyuan t2v loras
--lora-dir-hunyuan-i2v path # Path to Hunyuan i2v loras
--lora-dir-ltxv path # Path to LTX Video loras
--lora-preset preset # Load preset on startup
--check-loras # Filter incompatible loras
```

232
docs/MODELS.md Normal file
View File

@@ -0,0 +1,232 @@
# Models Overview
WanGP supports multiple video generation models, each optimized for different use cases and hardware configurations.
## Text-to-Video Models
### Wan 2.1 Models
#### Wan 2.1 Text2Video 1.3B
- **Size**: 1.3 billion parameters
- **VRAM**: 6GB minimum
- **Speed**: Fast generation
- **Quality**: Good quality for the size
- **Best for**: Quick iterations, lower-end hardware
- **Command**: `python wgp.py --t2v-1-3B`
#### Wan 2.1 Text2Video 14B
- **Size**: 14 billion parameters
- **VRAM**: 12GB+ recommended
- **Speed**: Slower but higher quality
- **Quality**: Excellent detail and coherence
- **Best for**: Final production videos
- **Command**: `python wgp.py --t2v-14B`
#### Wan Vace 1.3B
- **Type**: ControlNet for advanced video control
- **VRAM**: 6GB minimum
- **Features**: Motion transfer, object injection, inpainting
- **Best for**: Advanced video manipulation
- **Command**: `python wgp.py --vace`
#### Wan Vace 14B
- **Type**: Large ControlNet model
- **VRAM**: 12GB+ recommended
- **Features**: All Vace features with higher quality
- **Best for**: Professional video editing workflows
### Hunyuan Video Models
#### Hunyuan Video Text2Video
- **Quality**: Among the best open source t2v models
- **VRAM**: 12GB+ recommended
- **Speed**: Slower generation but excellent results
- **Features**: Superior text adherence and video quality
- **Best for**: High-quality text-to-video generation
#### Hunyuan Video Custom
- **Specialty**: Identity preservation
- **Use case**: Injecting specific people into videos
- **Quality**: Excellent for character consistency
- **Best for**: Character-focused video generation
### LTX Video Models
#### LTX Video 13B
- **Specialty**: Long video generation
- **Resolution**: Fast 720p generation
- **VRAM**: Optimized by WanGP (4x reduction in requirements)
- **Best for**: Longer duration videos
#### LTX Video 13B Distilled
- **Speed**: Generate in less than one minute
- **Quality**: Very high quality despite speed
- **Best for**: Rapid prototyping and quick results
### Other Models
#### Sky Reels v2
- **Type**: Diffusion Forcing model
- **Specialty**: "Infinite length" videos
- **Features**: High quality continuous generation
- **Note**: Uses causal attention (SDPA only)
#### MoviiGen (Experimental)
- **Resolution**: Claims 1080p capability
- **VRAM**: 20GB+ required
- **Speed**: Very slow generation
- **Status**: Experimental, feedback welcome
#### CausVid (Via Lora)
- **Type**: Distilled model (Lora implementation)
- **Speed**: 4-12 steps generation, 2x faster
- **Compatible**: Works with Wan 14B models
- **Setup**: Requires CausVid Lora (see [LORAS.md](LORAS.md))
## Image-to-Video Models
### Wan Fun InP Models
#### Wan Fun InP 1.3B
- **Size**: 1.3 billion parameters
- **VRAM**: 6GB minimum
- **Quality**: Good for the size, accessible to lower hardware
- **Best for**: Entry-level image animation
- **Command**: `python wgp.py --i2v-1-3B`
#### Wan Fun InP 14B
- **Size**: 14 billion parameters
- **VRAM**: 12GB+ recommended
- **Quality**: Better end image support
- **Limitation**: Existing loras don't work as well
- **Command**: `python wgp.py --i2v-14B`
### Specialized Models
#### FantasySpeaking
- **Type**: Talking head animation
- **Input**: Voice track + image
- **Works on**: People and objects
- **Use case**: Lip-sync and voice-driven animation
#### Phantom
- **Type**: Person/object transfer
- **Resolution**: Works well at 720p
- **Requirements**: 30+ steps for good results
- **Best for**: Transferring subjects between videos
#### Recam Master
- **Type**: Viewpoint change
- **Requirements**: 81+ frame input videos, 15+ denoising steps
- **Use case**: View same scene from different angles
#### FLF2V
- **Type**: Start/end frame specialist
- **Resolution**: Optimized for 720p
- **Official**: Wan team supported
- **Use case**: Image-to-video with specific endpoints
## Model Selection Guide
### By Hardware (VRAM)
#### 6-8GB VRAM
- Wan 2.1 T2V 1.3B
- Wan Fun InP 1.3B
- Wan Vace 1.3B
#### 10-12GB VRAM
- Wan 2.1 T2V 14B
- Wan Fun InP 14B
- Hunyuan Video (with optimizations)
- LTX Video 13B
#### 16GB+ VRAM
- All models supported
- Longer videos possible
- Higher resolutions
- Multiple simultaneous Loras
#### 20GB+ VRAM
- MoviiGen (experimental 1080p)
- Very long videos
- Maximum quality settings
### By Use Case
#### Quick Prototyping
1. **LTX Video 13B Distilled** - Fastest, high quality
2. **Wan 2.1 T2V 1.3B** - Fast, good quality
3. **CausVid Lora** - 4-12 steps, very fast
#### Best Quality
1. **Hunyuan Video** - Overall best t2v quality
2. **Wan 2.1 T2V 14B** - Excellent Wan quality
3. **Wan Vace 14B** - Best for controlled generation
#### Advanced Control
1. **Wan Vace 14B/1.3B** - Motion transfer, object injection
2. **Phantom** - Person/object transfer
3. **FantasySpeaking** - Voice-driven animation
#### Long Videos
1. **LTX Video 13B** - Specialized for length
2. **Sky Reels v2** - Infinite length videos
3. **Wan Vace + Sliding Windows** - Up to 1 minute
#### Lower Hardware
1. **Wan Fun InP 1.3B** - Image-to-video
2. **Wan 2.1 T2V 1.3B** - Text-to-video
3. **Wan Vace 1.3B** - Advanced control
## Performance Comparison
### Speed (Relative)
1. **CausVid Lora** (4-12 steps) - Fastest
2. **LTX Video Distilled** - Very fast
3. **Wan 1.3B models** - Fast
4. **Wan 14B models** - Medium
5. **Hunyuan Video** - Slower
6. **MoviiGen** - Slowest
### Quality (Subjective)
1. **Hunyuan Video** - Highest overall
2. **Wan 14B models** - Excellent
3. **LTX Video models** - Very good
4. **Wan 1.3B models** - Good
5. **CausVid** - Good (varies with steps)
### VRAM Efficiency
1. **Wan 1.3B models** - Most efficient
2. **LTX Video** (with WanGP optimizations)
3. **Wan 14B models**
4. **Hunyuan Video**
5. **MoviiGen** - Least efficient
## Model Switching
WanGP allows switching between models without restarting:
1. Use the dropdown menu in the web interface
2. Models are loaded on-demand
3. Previous model is unloaded to save VRAM
4. Settings are preserved when possible
## Tips for Model Selection
### First Time Users
Start with **Wan 2.1 T2V 1.3B** to learn the interface and test your hardware.
### Production Work
Use **Hunyuan Video** or **Wan 14B** models for final output quality.
### Experimentation
**CausVid Lora** or **LTX Distilled** for rapid iteration and testing.
### Specialized Tasks
- **VACE** for advanced control
- **FantasySpeaking** for talking heads
- **LTX Video** for long sequences
### Hardware Optimization
Always start with the largest model your VRAM can handle, then optimize settings for speed vs quality based on your needs.

338
docs/TROUBLESHOOTING.md Normal file
View File

@@ -0,0 +1,338 @@
# Troubleshooting Guide
This guide covers common issues and their solutions when using WanGP.
## Installation Issues
### PyTorch Installation Problems
#### CUDA Version Mismatch
**Problem**: PyTorch can't detect GPU or CUDA errors
**Solution**:
```bash
# Check your CUDA version
nvidia-smi
# Install matching PyTorch version
# For CUDA 12.4 (RTX 10XX-40XX)
pip install torch==2.6.0 torchvision torchaudio --index-url https://download.pytorch.org/whl/test/cu124
# For CUDA 12.8 (RTX 50XX)
pip install torch==2.7.0 torchvision torchaudio --index-url https://download.pytorch.org/whl/test/cu128
```
#### Python Version Issues
**Problem**: Package compatibility errors
**Solution**: Ensure you're using Python 3.10.9
```bash
python --version # Should show 3.10.9
conda create -n wan2gp python=3.10.9
```
### Dependency Installation Failures
#### Triton Installation (Windows)
**Problem**: `pip install triton-windows` fails
**Solution**:
1. Update pip: `pip install --upgrade pip`
2. Try pre-compiled wheel
3. Fallback to SDPA attention: `python wgp.py --attention sdpa`
#### SageAttention Compilation Issues
**Problem**: SageAttention installation fails
**Solution**:
1. Install Visual Studio Build Tools (Windows)
2. Use pre-compiled wheels when available
3. Fallback to basic attention modes
## Memory Issues
### CUDA Out of Memory
#### During Model Loading
**Problem**: "CUDA out of memory" when loading model
**Solutions**:
```bash
# Use smaller model
python wgp.py --t2v-1-3B
# Enable quantization (usually default)
python wgp.py --quantize-transformer True
# Use memory-efficient profile
python wgp.py --profile 4
# Reduce preloaded model size
python wgp.py --preload 0
```
#### During Video Generation
**Problem**: Memory error during generation
**Solutions**:
1. Reduce frame count (shorter videos)
2. Lower resolution in advanced settings
3. Use lower batch size
4. Clear GPU cache between generations
### System RAM Issues
#### High RAM Usage
**Problem**: System runs out of RAM
**Solutions**:
```bash
# Limit reserved memory
python wgp.py --perc-reserved-mem-max 0.3
# Use minimal RAM profile
python wgp.py --profile 5
# Enable swap file (OS level)
```
## Performance Issues
### Slow Generation Speed
#### General Optimization
```bash
# Enable compilation (requires Triton)
python wgp.py --compile
# Use faster attention
python wgp.py --attention sage2
# Enable TeaCache
python wgp.py --teacache 2.0
# Use high-performance profile
python wgp.py --profile 3
```
#### GPU-Specific Optimizations
**RTX 10XX/20XX Series**:
```bash
python wgp.py --attention sdpa --profile 4 --teacache 1.5
```
**RTX 30XX/40XX Series**:
```bash
python wgp.py --compile --attention sage --profile 3 --teacache 2.0
```
**RTX 50XX Series**:
```bash
python wgp.py --attention sage --profile 4 --fp16
```
### Attention Mechanism Issues
#### Sage Attention Not Working
**Problem**: Sage attention fails to compile or work
**Diagnostic Steps**:
1. Check Triton installation:
```python
import triton
print(triton.__version__)
```
2. Clear Triton cache:
```bash
# Windows
rmdir /s %USERPROFILE%\.triton
# Linux
rm -rf ~/.triton
```
3. Fallback solution:
```bash
python wgp.py --attention sdpa
```
#### Flash Attention Issues
**Problem**: Flash attention compilation fails
**Solution**:
- Windows: Often requires manual CUDA kernel compilation
- Linux: Usually works with `pip install flash-attn`
- Fallback: Use Sage or SDPA attention
## Model-Specific Issues
### Lora Problems
#### Loras Not Loading
**Problem**: Loras don't appear in the interface
**Solutions**:
1. Check file format (should be .safetensors, .pt, or .pth)
2. Verify correct directory:
```
loras/ # For t2v models
loras_i2v/ # For i2v models
loras_hunyuan/ # For Hunyuan models
```
3. Click "Refresh" button in interface
4. Use `--check-loras` to filter incompatible files
#### Lora Compatibility Issues
**Problem**: Lora causes errors or poor results
**Solutions**:
1. Check model size compatibility (1.3B vs 14B)
2. Verify lora was trained for your model type
3. Try different multiplier values
4. Use `--check-loras` flag to auto-filter
### VACE-Specific Issues
#### Poor VACE Results
**Problem**: VACE generates poor quality or unexpected results
**Solutions**:
1. Enable Skip Layer Guidance
2. Use detailed prompts describing all elements
3. Ensure proper mask creation with Matanyone
4. Check reference image quality
5. Use at least 15 steps, preferably 30+
#### Matanyone Tool Issues
**Problem**: Mask creation difficulties
**Solutions**:
1. Use negative point prompts to refine selection
2. Create multiple sub-masks and combine them
3. Try different background removal options
4. Ensure sufficient contrast in source video
## Network and Server Issues
### Gradio Interface Problems
#### Port Already in Use
**Problem**: "Port 7860 is already in use"
**Solution**:
```bash
# Use different port
python wgp.py --server-port 7861
# Or kill existing process
# Windows
netstat -ano | findstr :7860
taskkill /PID <PID> /F
# Linux
lsof -i :7860
kill <PID>
```
#### Interface Not Loading
**Problem**: Browser shows "connection refused"
**Solutions**:
1. Check if server started successfully
2. Try `http://127.0.0.1:7860` instead of `localhost:7860`
3. Disable firewall temporarily
4. Use `--listen` flag for network access
### Remote Access Issues
#### Sharing Not Working
**Problem**: `--share` flag doesn't create public URL
**Solutions**:
1. Check internet connection
2. Try different network
3. Use `--listen` with port forwarding
4. Check firewall settings
## Quality Issues
### Poor Video Quality
#### General Quality Improvements
1. Increase number of steps (25-30+)
2. Use larger models (14B instead of 1.3B)
3. Enable Skip Layer Guidance
4. Improve prompt descriptions
5. Use higher resolution settings
#### Specific Quality Issues
**Blurry Videos**:
- Increase steps
- Check source image quality (i2v)
- Reduce TeaCache multiplier
- Use higher guidance scale
**Inconsistent Motion**:
- Use longer overlap in sliding windows
- Reduce window size
- Improve prompt consistency
- Check control video quality (VACE)
**Color Issues**:
- Check model compatibility
- Adjust guidance scale
- Verify input image color space
- Try different VAE settings
## Advanced Debugging
### Enable Verbose Output
```bash
# Maximum verbosity
python wgp.py --verbose 2
# Check lora compatibility
python wgp.py --check-loras --verbose 2
```
### Memory Debugging
```bash
# Monitor GPU memory
nvidia-smi -l 1
# Reduce memory usage
python wgp.py --profile 4 --perc-reserved-mem-max 0.2
```
### Performance Profiling
```bash
# Test different configurations
python wgp.py --attention sdpa --profile 4 # Baseline
python wgp.py --attention sage --profile 3 # Performance
python wgp.py --compile --teacache 2.0 # Maximum speed
```
## Getting Help
### Before Asking for Help
1. Check this troubleshooting guide
2. Read the relevant documentation:
- [Installation Guide](INSTALLATION.md)
- [Getting Started](GETTING_STARTED.md)
- [Command Line Reference](CLI.md)
3. Try basic fallback configuration:
```bash
python wgp.py --attention sdpa --profile 4
```
### Community Support
- **Discord Server**: https://discord.gg/g7efUW9jGV
- Provide relevant information:
- GPU model and VRAM amount
- Python and PyTorch versions
- Complete error messages
- Command used to launch WanGP
- Operating system
### Reporting Bugs
When reporting issues:
1. Include system specifications
2. Provide complete error logs
3. List the exact steps to reproduce
4. Mention any modifications to default settings
5. Include command line arguments used
## Emergency Fallback
If nothing works, try this minimal configuration:
```bash
# Absolute minimum setup
python wgp.py --t2v-1-3B --attention sdpa --profile 4 --teacache 0 --fp16
# If that fails, check basic PyTorch installation
python -c "import torch; print(torch.cuda.is_available())"
```

190
docs/VACE.md Normal file
View File

@@ -0,0 +1,190 @@
# VACE ControlNet Guide
VACE is a powerful ControlNet that enables Video-to-Video and Reference-to-Video generation. It allows you to inject your own images into output videos, animate characters, perform inpainting/outpainting, and continue videos.
## Overview
VACE is probably one of the most powerful Wan models available. With it, you can:
- Inject people or objects into scenes
- Animate characters
- Perform video inpainting and outpainting
- Continue existing videos
- Transfer motion from one video to another
- Change the style of scenes while preserving depth
## Getting Started
### Model Selection
1. Select either "Vace 1.3B" or "Vace 13B" from the dropdown menu
2. Note: VACE works best with videos up to 7 seconds with the Riflex option enabled
### Input Types
VACE accepts three types of visual hints (which can be combined):
#### 1. Control Video
- Transfer motion or depth to a new video
- Use only the first n frames and extrapolate the rest
- Perform inpainting with grey color (127) as mask areas
- Grey areas will be filled based on text prompt and reference images
#### 2. Reference Images
- Use as background/setting for the video
- Inject people or objects of your choice
- Select multiple reference images
- **Tip**: Replace complex backgrounds with white for better object integration
- Always describe injected objects/people explicitly in your text prompt
#### 3. Video Mask
- Stronger control over which parts to keep (black) or replace (white)
- Perfect for inpainting/outpainting
- Example: White mask except at beginning/end (black) keeps first/last frames while generating middle content
## Common Use Cases
### Motion Transfer
**Goal**: Animate a character of your choice using motion from another video
**Setup**:
- Reference Images: Your character
- Control Video: Person performing desired motion
- Text Prompt: Describe your character and the action
### Object/Person Injection
**Goal**: Insert people or objects into a scene
**Setup**:
- Reference Images: The people/objects to inject
- Text Prompt: Describe the scene and explicitly mention the injected elements
### Character Animation
**Goal**: Animate a character based on text description
**Setup**:
- Control Video: Video of person moving
- Text Prompt: Detailed description of your character
### Style Transfer with Depth
**Goal**: Change scene style while preserving spatial relationships
**Setup**:
- Control Video: Original video (for depth information)
- Text Prompt: New style description
## Integrated Matanyone Tool
WanGP includes the Matanyone tool, specifically tuned for VACE workflows. This helps create control videos and masks simultaneously.
### Creating Face Replacement Masks
1. Load your video in Matanyone
2. Click on the face in the first frame
3. Create a mask for the face
4. Generate both control video and mask video with "Generate Video Matting"
5. Export to VACE with "Export to current Video Input and Video Mask"
6. Load replacement face image in Reference Images field
### Advanced Matanyone Tips
- **Negative Point Prompts**: Remove parts from current selection
- **Sub Masks**: Create multiple independent masks, then combine them
- **Background Masks**: Select everything except the character (useful for background replacement)
- Enable/disable sub masks in Matanyone settings
## Recommended Settings
### Quality Settings
- **Skip Layer Guidance**: Turn ON with default configuration for better results
- **Long Prompts**: Use detailed descriptions, especially for background elements not in reference images
- **Steps**: Use at least 15 steps for good quality, 30+ for best results
### Sliding Window Settings
For very long videos, configure sliding windows properly:
- **Window Size**: Set appropriate duration for your content
- **Overlap Frames**: Long enough for motion continuity, short enough to avoid blur propagation
- **Discard Last Frames**: Remove at least 4 frames from each window (VACE 1.3B tends to blur final frames)
### Background Removal
VACE includes automatic background removal options:
- Use for reference images containing people/objects
- **Don't use** for landscape/setting reference images (first reference image)
- Multiple background removal types available
## Window Sliding for Long Videos
Generate videos up to 1 minute by merging multiple windows:
### How It Works
- Each window uses corresponding time segment from control video
- Example: 0-4s control video → first window, 4-8s → second window, etc.
- Automatic overlap management ensures smooth transitions
### Settings
- **Window Size**: Duration of each generation window
- **Overlap Frames**: Frames shared between windows for continuity
- **Discard Last Frames**: Remove poor-quality ending frames
- **Add Overlapped Noise**: Reduce quality degradation over time
### Formula
```
Generated Frames = [Windows - 1] × [Window Size - Overlap - Discard] + Window Size
```
### Multi-Line Prompts (Experimental)
- Each line of prompt used for different window
- If more windows than prompt lines, last line repeats
- Separate lines with carriage return
## Advanced Features
### Extend Video
Click "Extend the Video Sample, Please!" during generation to add more windows dynamically.
### Noise Addition
Add noise to overlapped frames to hide accumulated errors and quality degradation.
### Frame Truncation
Automatically remove lower-quality final frames from each window (recommended: 4 frames for VACE 1.3B).
## External Resources
### Official VACE Resources
- **GitHub**: https://github.com/ali-vilab/VACE/tree/main/vace/gradios
- **User Guide**: https://github.com/ali-vilab/VACE/blob/main/UserGuide.md
- **Preprocessors**: Gradio tools for preparing materials
### Recommended External Tools
- **Annotation Tools**: For creating precise masks
- **Video Editors**: For preparing control videos
- **Background Removal**: For cleaning reference images
## Troubleshooting
### Poor Quality Results
1. Use longer, more detailed prompts
2. Enable Skip Layer Guidance
3. Increase number of steps (30+)
4. Check reference image quality
5. Ensure proper mask creation
### Inconsistent Windows
1. Increase overlap frames
2. Use consistent prompting across windows
3. Add noise to overlapped frames
4. Reduce discard frames if losing too much content
### Memory Issues
1. Use VACE 1.3B instead of 13B
2. Reduce video length or resolution
3. Decrease window size
4. Enable quantization
### Blurry Results
1. Reduce overlap frames
2. Increase discard last frames
3. Use higher resolution reference images
4. Check control video quality
## Tips for Best Results
1. **Detailed Prompts**: Describe everything in the scene, especially elements not in reference images
2. **Quality Reference Images**: Use high-resolution, well-lit reference images
3. **Proper Masking**: Take time to create precise masks with Matanyone
4. **Iterative Approach**: Start with short videos, then extend successful results
5. **Background Preparation**: Remove complex backgrounds from object/person reference images
6. **Consistent Lighting**: Match lighting between reference images and intended scene