I’d been generating AI images locally with Stable Diffusion for a while, but I had given up on AI video. My GPU is an RTX 2080 SUPER with only 8GB of VRAM. I once tried Stable Video Diffusion and couldn’t get it to run properly. I knew about ComfyUI, but node-based workflows looked hard, and I assumed I’d hit the VRAM wall anyway.
So I handed the whole setup to Claude Code: not just wiring the nodes, but also figuring out what can and can’t run on 8GB, and how far the speed can be pushed. (This is a hobby project, not work.)
Short answer: with only a few short instructions from me, Claude Code built a working local video generation setup, and the biggest win was changing how the “VAE decode” step runs, which cut generation from 20 min 13 s to about 78 s (about 16× faster). The final setup makes a 1280x704, ~5-second video in about 226 s.
The model is the small 5B one, so text-to-video quality is limited, but image-to-video can be fun if you work on the prompt. Below is an image made in Stable Diffusion turned into a video with ComfyUI i2v (at a higher resolution than in my tests, so it took longer).
Note: numbers corrected in November 2026. The original version reported VAE-decode-only times (52 s, 63 s, 134 s) as total generation times; they are now measured end to end.
If you only want the steps: the quickest way is the launch .bat with the workflow I share in the follow-up, Wan 2.2 image-to-video on 8GB VRAM: best ComfyUI settings. To install everything yourself, jump to Manual setup without Claude Code. The story below can be skipped.
What I asked Claude Code
My instructions were mostly one-liners like these (translated):
“I have Stable Diffusion WebUI installed at D:\Program Files\Stable\stable-diffusion-webui. I want to add extensions so I can make videos from images, from text too.”
“This quality is unwatchable.”
“I can tell it flickers. Try raising the overlap.”
I only gave impressions plus requests. Finding causes, deciding fixes: all Claude.
How it went
Investigating my setup: AnimateDiff’s limits
Claude Code first surveyed my environment by itself: A1111 version, Python plus torch versions, GPU VRAM, installed extensions and models. It reported:
- A1111 development has stalled; the latest tag is from July 2024
- All my models are SDXL-based, but the video extension AnimateDiff works poorly with SDXL; its author calls SDXL results very disappointing in the docs
- Measured VRAM: SDXL wanted 8.66GB for 512px with 8 frames, while SD1.5 fits in 4.2-5.6GB at 512px with 16 frames
- In 2026 the main choice for local video is ComfyUI with Wan 2.2; A1111 is a lap behind
An honest “your current models won’t work well, and A1111 isn’t the tool for this anymore”. It offered three options (add SD1.5 models, install ComfyUI, add Deforum) and I had it do all three.
Installing ComfyUI, verifying it really runs
Installing ComfyUI is just extracting an archive. The important part came next: my RTX 2080 SUPER is a Turing GPU, close to being dropped by the latest CUDA. Instead of assuming the bundled torch supported it, Claude Code ran matrix operations on the GPU to check. fp16 on the GPU worked.
It also configured ComfyUI to share A1111’s models (22 checkpoints, VAEs, LoRAs).
Deciding what fits in 8GB from file sizes
For video in ComfyUI, the main option is the Wan 2.2 family, which comes in several sizes and formats. Claude Code pulled actual file sizes from the Hugging Face API to judge what fits in 8GB:
| Model | Smallest size | On 8GB VRAM |
|---|---|---|
| Wan 2.2 TI2V-5B (standard) | 10.00 GB | Tough |
| Wan 2.2 TI2V-5B (GGUF quantized) | 3.81 GB | Works |
| Wan 2.2 14B (high quality) | 14.29 GB × 2 files | No |
| Other large models (MiniMax-H3, LTX-2.3, etc.) | 29-42+ GB | No |
Only the GGUF-quantized Wan 2.2 TI2V-5B is realistic on 8GB, and that one model covers both text-to-video and image-to-video. A 704x480, 49-frame video generated fine.
Broken dependencies: A1111 stopped starting
This was the scariest moment. Installing ControlNet (needed for AnimateDiff) pulled in a newer numpy, and A1111 no longer started. Fixing that broke mediapipe, fixing that broke something else, a chain reaction. In the end Claude pinned specific versions of numpy, protobuf, mediapipe, imported about ten dependencies one by one to verify them, and everything started without errors again. On my own I would have given up here.
Finding why generation was so slow (the key part)
The first videos looked bad. The cause was resolution: Wan 2.2 TI2V-5B is trained at 1280x704, but the initial setting was 704x480 to be safe on 8GB. At the native 1280x704 the quality improved, but one video took 22 min 56 s.
The hypothesis “fewer steps means faster” led to trying a distilled 4-step Turbo model: 20 min 13 s, almost no change. Steps weren’t the problem.
Looking at the progress log, the sampling itself took only 23 seconds. The time went into VAE decode, the step that converts the result into actual video frames. The 1280x704, 49-frame data didn’t fit in VRAM, so it shuffled data between GPU and CPU for over 19 minutes.
Switching to tiled VAE decode (decode small pieces, then stitch them) brought the decode down to 63 seconds. But then the video flickered.
Claude’s first guess was that more tile overlap would fix it. Wrong: doubling the overlap didn’t change image quality (measured with PSNR) and made the decode slower (63 s → 170 s). Comparing four patterns showed the real cause was the temporal tile size (temporal_size). Tuning it brought flicker (difference between adjacent frames) almost to the level of untiled decoding, and the decode became the fastest yet, 52 s. End to end, about 78 s versus 20 min 13 s before: about 16× faster.
What impressed me was that it judged the results by measuring the videos numerically instead of eyeballing them.
| Stage | Settings | Total time |
|---|---|---|
| Initial | 704x480 / 49 frames / 20 steps | About 3 min (low quality) |
| Native resolution | 1280x704 / 49 frames / 20 steps | 22 min 56 s |
| Turbo model | 1280x704 / 49 frames / 4 steps | 20 min 13 s |
| Tiled VAE decode | Same + tiling (tile, overlap, temporal size tuned) | About 78 s (about 16×) |
| Extended to 5 seconds | 121 frames | About 226 s |
Finally I asked how to make videos longer. Claude read ComfyUI’s source to confirm how frame count is set, then measured that 121 frames (about 5 s) still leaves VRAM headroom; that’s also the length Wan 2.2 TI2V-5B was trained on.
Downloading from a login-only site via the browser
Wan 2.2 is trained mostly on realistic footage and is weak at anime styles, so I wanted a LoRA. The one I wanted was only on Civitai, which requires login. Using the Claude in Chrome extension, Claude operated my already-logged-in browser and picked the right file version (the default version targeted a different model size). It stalled at the save-location dialog, so I chose the folder myself. After downloading, it checked the tensor dimensions inside the LoRA to confirm it matched the model size, instead of trusting the file name.
Pitfalls
- With the normal VAE decode, an ~78 s generation turns into 20 min 13 s. Wan’s VAE doesn’t fit in 8GB as is
- More tile overlap doesn’t always fix flicker; the cause may be the temporal tile size
- Distilled (Turbo) models need their own steps/CFG. At CFG 1 the negative prompt is ignored entirely
- Don’t use SDXL models with AnimateDiff; get SD1.5 models
- Installing ControlNet can break an existing Python environment; you may need to pin numpy, protobuf, mediapipe
- On 8GB, don’t run A1111 next to ComfyUI at the same time; they fight over VRAM
- On Civitai, check the version you download; the default may target a larger model
Manual setup without Claude Code
These steps should reproduce the same setup if done in order. I haven’t run them by hand myself, so consider checking with Claude Code or Codex as you go. To copy my exact settings, the .bat plus workflow from the follow-up post are the safer route.
ComfyUI plus custom nodes
1. ComfyUIの公式リリースページから "ComfyUI_windows_portable_nvidia.7z" をダウンロードして展開する https://github.com/comfyanonymous/ComfyUI/releases 2. カスタムノードを導入する cd ComfyUI_windows_portable/ComfyUI/custom_nodes git clone --depth 1 https://github.com/Comfy-Org/ComfyUI-Manager.git git clone --depth 1 https://github.com/city96/ComfyUI-GGUF.git cd ../.. ./python_embeded/python.exe -m pip install "gguf>=0.13.0" sentencepiece protobuf ./python_embeded/python.exe -m pip install -r ComfyUI/custom_nodes/ComfyUI-Manager/requirements.txt
Model placement
| Folder | Source |
|---|---|
| ComfyUI/models/diffusion_models/ | huggingface.co/hum-ma/Wan2.2-TI2V-5B-Turbo-GGUF (Q6_K) |
| ComfyUI/models/text_encoders/ | huggingface.co/city96/umt5-xxl-encoder-gguf (Q5_K_M) |
| ComfyUI/models/vae/ | huggingface.co/QuantStack/Wan2.2-TI2V-5B-GGUF (VAE folder) |
Workflow settings
Start ComfyUI with run_nvidia_gpu.bat, load the “Wan 2.2 5B TI2V” template, then replace these nodes:
| Original node | Replace with |
|---|---|
| UNETLoader | Unet Loader (GGUF) |
| CLIPLoader | CLIPLoader (GGUF) (type “wan”) |
| VAEDecode | VAE Decode (Tiled) (tile 512 / overlap 128 / temporal_size 64 / temporal_overlap 8) |
| KSampler | steps 4 / cfg 1.0 |
| Wan22ImageToVideoLatent | 1280 x 704 / length 121 |
Note: in the current ComfyUI UI, UNETLoader and CLIPLoader are displayed as “Load Diffusion Model” and “Load CLIP”. Search for those names if the ones above don’t show up.
These settings include the VAE decode tuning described above. Copying just this part avoids the unusably slow state.
(Reference) Using A1111 + AnimateDiff
If you’d rather try video in A1111, here are the steps. As noted, it works poorly with SDXL, so you’ll need SD1.5 models.
cd stable-diffusion-webui/extensions git clone --depth 1 https://github.com/Mikubill/sd-webui-controlnet.git git clone --depth 1 https://github.com/continue-revolution/sd-webui-animatediff.git # モーションモジュールを extensions/sd-webui-animatediff/model/ に配置する # huggingface.co/conrevo/AnimateDiff-A1111 の motion_module/mm_sd15_v3.safetensors など # SD1.5系のモデルを models/Stable-diffusion/ に配置する # ControlNet導入後は依存関係が壊れやすいため、バージョンを固定しておく cd .. ./venv/Scripts/python.exe -m pip install "numpy==1.26.2" "protobuf==3.20.0" "mediapipe==0.10.14"
After starting, turn ON “Pad prompt/negative prompt to be same length” under Settings → Optimizations and press Apply settings. Otherwise it generates two unrelated videos at once.
Summary
Even an 8GB card, close to the minimum for video generation, can reach usable speeds with the right settings. And the trial and error to find them (checking the environment, repairing dependencies, isolating the cause, measuring) took almost no work on my part.
What stuck with me most is that the slowness came from neither the step count nor the model size, but from VAE decode, an easy step to overlook. Reading logs and measuring, instead of tweaking settings at random, is what got to 16×.
If you want to try local AI video but think your GPU is too weak, or ComfyUI’s nodes look intimidating, try asking Claude Code to set up the whole thing.
In the follow-up I looked for the best quality/time balance on this setup, with a downloadable .bat plus workflow:


コメント