Free VRAM Used by Ollama and AivisSpeech for Image Generation (8GB GPU)

スポンサーリンク
Free VRAM Used by Ollama and AivisSpeech for Image Generation (8GB GPU) Generative AI
スポンサーリンク

I run my own AI chat that talks with Ollama and speaks through AivisSpeech, on the same PC (RTX 2080 SUPER, 8GB VRAM) where I generate images and videos with ComfyUI. VRAM runs out fast.

When I checked, even while idle Ollama’s model was holding about 3.1GB of VRAM and AivisSpeech running on the GPU about 1.1GB. That’s more than half of 8GB.

Short answer: unload the Ollama model with ollama stop before generating images or videos, and start AivisSpeech without the GPU (on the CPU). Ollama frees about 3.1GB in roughly 0.2 seconds, and reloading on your next message takes only about 3 seconds. Running AivisSpeech on the CPU barely changed speech speed for short sentences.

Results: how much VRAM was freed

Total VRAM use measured with nvidia-smi (November 2026, RTX 2080 SUPER 8GB, Windows 11):

State VRAM used
No Ollama model loaded 792MiB
After loading qwen3:4b (4-bit) in Ollama 3949MiB (+ about 3.1GB)
After ollama stop 792MiB (back in 0.23 s)

AivisSpeech used about 1.1GB more VRAM when started on the GPU than on the CPU.

Unloading the Ollama model

1. Check which models are loaded

Run this in PowerShell or Command Prompt:

ollama ps

If a model is loaded you’ll see NAME, SIZE, PROCESSOR, UNTIL, etc. Mine showed “3.2 GB”, “100% GPU”, “4 minutes from now”. 100% GPU means the whole model sits in VRAM; UNTIL is the time left before it unloads automatically.

2. Unload with ollama stop (recommended)

Run it with the NAME shown by ollama ps:

ollama stop qwen3:4b-instruct-2507-q4_K_M

Nothing is printed on success. Run ollama ps again; an empty list means it worked. In my test the command returned in 0.13 s, with VRAM back at 792MiB 0.23 s later.

The model isn’t deleted. It reloads automatically the next time you talk to it. Timings:

State Until the reply finished
Talking after unloading 3.34 s (3.00 s of it loading)
Talking with the model already loaded 0.19 s

It’s only a 3-second wait, so unloading every time before image/video generation is no problem.

3. It unloads by itself after 5 minutes

Ollama unloads a model 5 minutes after its last use (the default, per the official FAQ). Left alone, my VRAM went back to 792MiB after 300.8 seconds. If you can wait 5 minutes, you don’t need to do anything.

Unloading from your own app (keep_alive)

If your own script or app calls Ollama, sending keep_alive set to 0 to the API unloads it immediately. In PowerShell (11434 is Ollama’s default port):

Invoke-RestMethod http://localhost:11434/api/generate -Method Post -Body '{"model":"qwen3:4b-instruct-2507-q4_K_M","keep_alive":0}'

It worked in Windows PowerShell 5.1 even without -ContentType. It unloaded in 0.10 s, with done_reason set to unload in the response. With curl:

curl.exe http://localhost:11434/api/generate -d "{\"model\":\"qwen3:4b-instruct-2507-q4_K_M\",\"keep_alive\":0}"

If you send keep_alive with every chat request, the model unloads after each reply. But then every message has a ~3 s load, so it’s nicer to leave it off during a conversation. For a one-off prompt from the command line that unloads right after, add --keepalive 0:

ollama run qwen3:4b-instruct-2507-q4_K_M --keepalive 0 "こんにちは"

To change the 5-minute default itself, the official FAQ says to use the OLLAMA_KEEP_ALIVE environment variable (on Windows, set it as a user environment variable). I haven’t tried it for this post. A keep_alive sent through the API takes priority over the environment variable.

Running AivisSpeech on the CPU

AivisSpeech Engine (the speech engine itself, run.exe) uses the GPU only when started with --use_gpu; on Windows it uses the GPU through DirectML (per the official README). So start it without --use_gpu and it runs on the CPU, using no VRAM.

Open PowerShell in the AivisSpeech Engine folder (where run.exe is) and start it like this. 10101 is AivisSpeech Engine’s default port.

.\run.exe --host 127.0.0.1 --port 10101

To go back to the GPU, just add --use_gpu at the end:

.\run.exe --host 127.0.0.1 --port 10101 --use_gpu

If an engine is already running on the GPU, close it first (two engines can’t share a port).

When the engine is started from the AivisSpeech app (editor), on my PC the GPU setting was stored as useGpu in %APPDATA%\AivisSpeech\config.json (false means CPU). I didn’t check how to switch it in the app’s settings screen.

Is speech slower on the CPU?

Same PC (Core i9-9900K), speech generation time with GPU versus CPU (Japanese text):

Text GPU CPU
Short (7 characters) about 0.9 s about 0.6 s
Medium (28 characters) about 1.0 s about 1.4 s
Long (43 characters) about 1.0 s about 2.2 s
VRAM CPU uses about 1.1GB less

Short text was faster on the CPU; long text took about twice as long. My chat speaks one sentence at a time, so the delay before speech starts hardly changed. ComfyUI image generation speed didn’t change either with AivisSpeech on the CPU (about 6.4 s → 6.3 s per image).

If you read long passages in one go, keep the GPU. For conversation-style short sentences, the CPU is the better deal because it frees VRAM.

Command to check free VRAM

On NVIDIA GPUs, this shows total VRAM use. Compare before versus after unloading.

nvidia-smi --query-gpu=memory.used,memory.total --format=csv

Summary

  • Before generating images or video, run ollama stop MODEL_NAME. About 3.1GB (4B 4-bit model) is freed in 0.2 s; it reloads in about 3 s
  • Even if you do nothing, Ollama unloads after 5 minutes
  • Start AivisSpeech Engine without --use_gpu to run it on the CPU and free about 1.1GB. Short sentences are about as fast

The settings I use to make videos with the freed VRAM are here:

To see at a glance what is using VRAM, I also built a local AI dashboard.

References

Test environment: RTX 2080 SUPER (8GB VRAM), Core i9-9900K, Windows 11, Ollama 0.34.0, model qwen3:4b-instruct-2507-q4_K_M. Measured on November 11, 2026.

コメント

Copied title and URL