I run my own AI chat that talks with Ollama and speaks through AivisSpeech, on the same PC (RTX 2080 SUPER, 8GB VRAM) where I generate images and videos with ComfyUI. VRAM runs out fast.
When I checked, even while idle Ollama’s model was holding about 3.1GB of VRAM and AivisSpeech running on the GPU about 1.1GB. That’s more than half of 8GB.
Short answer: unload the Ollama model with ollama stop before generating images or videos, and start AivisSpeech without the GPU (on the CPU). Ollama frees about 3.1GB in roughly 0.2 seconds, and reloading on your next message takes only about 3 seconds. Running AivisSpeech on the CPU barely changed speech speed for short sentences.
Results: how much VRAM was freed
Total VRAM use measured with nvidia-smi (November 2026, RTX 2080 SUPER 8GB, Windows 11):
| State | VRAM used |
|---|---|
| No Ollama model loaded | 792MiB |
| After loading qwen3:4b (4-bit) in Ollama | 3949MiB (+ about 3.1GB) |
After ollama stop |
792MiB (back in 0.23 s) |
AivisSpeech used about 1.1GB more VRAM when started on the GPU than on the CPU.
Unloading the Ollama model
1. Check which models are loaded
Run this in PowerShell or Command Prompt:
ollama ps
If a model is loaded you’ll see NAME, SIZE, PROCESSOR, UNTIL, etc. Mine showed “3.2 GB”, “100% GPU”, “4 minutes from now”. 100% GPU means the whole model sits in VRAM; UNTIL is the time left before it unloads automatically.
2. Unload with ollama stop (recommended)
Run it with the NAME shown by ollama ps:
ollama stop qwen3:4b-instruct-2507-q4_K_M
Nothing is printed on success. Run ollama ps again; an empty list means it worked. In my test the command returned in 0.13 s, with VRAM back at 792MiB 0.23 s later.
The model isn’t deleted. It reloads automatically the next time you talk to it. Timings:
| State | Until the reply finished |
|---|---|
| Talking after unloading | 3.34 s (3.00 s of it loading) |
| Talking with the model already loaded | 0.19 s |
It’s only a 3-second wait, so unloading every time before image/video generation is no problem.
3. It unloads by itself after 5 minutes
Ollama unloads a model 5 minutes after its last use (the default, per the official FAQ). Left alone, my VRAM went back to 792MiB after 300.8 seconds. If you can wait 5 minutes, you don’t need to do anything.
Unloading from your own app (keep_alive)
If your own script or app calls Ollama, sending keep_alive set to 0 to the API unloads it immediately. In PowerShell (11434 is Ollama’s default port):
Invoke-RestMethod http://localhost:11434/api/generate -Method Post -Body '{"model":"qwen3:4b-instruct-2507-q4_K_M","keep_alive":0}'
It worked in Windows PowerShell 5.1 even without -ContentType. It unloaded in 0.10 s, with done_reason set to unload in the response. With curl:
curl.exe http://localhost:11434/api/generate -d "{\"model\":\"qwen3:4b-instruct-2507-q4_K_M\",\"keep_alive\":0}"
If you send keep_alive with every chat request, the model unloads after each reply. But then every message has a ~3 s load, so it’s nicer to leave it off during a conversation. For a one-off prompt from the command line that unloads right after, add --keepalive 0:
ollama run qwen3:4b-instruct-2507-q4_K_M --keepalive 0 "こんにちは"
To change the 5-minute default itself, the official FAQ says to use the OLLAMA_KEEP_ALIVE environment variable (on Windows, set it as a user environment variable). I haven’t tried it for this post. A keep_alive sent through the API takes priority over the environment variable.
Running AivisSpeech on the CPU
AivisSpeech Engine (the speech engine itself, run.exe) uses the GPU only when started with --use_gpu; on Windows it uses the GPU through DirectML (per the official README). So start it without --use_gpu and it runs on the CPU, using no VRAM.
Open PowerShell in the AivisSpeech Engine folder (where run.exe is) and start it like this. 10101 is AivisSpeech Engine’s default port.
.\run.exe --host 127.0.0.1 --port 10101
To go back to the GPU, just add --use_gpu at the end:
.\run.exe --host 127.0.0.1 --port 10101 --use_gpu
If an engine is already running on the GPU, close it first (two engines can’t share a port).
When the engine is started from the AivisSpeech app (editor), on my PC the GPU setting was stored as useGpu in %APPDATA%\AivisSpeech\config.json (false means CPU). I didn’t check how to switch it in the app’s settings screen.
Is speech slower on the CPU?
Same PC (Core i9-9900K), speech generation time with GPU versus CPU (Japanese text):
| Text | GPU | CPU |
|---|---|---|
| Short (7 characters) | about 0.9 s | about 0.6 s |
| Medium (28 characters) | about 1.0 s | about 1.4 s |
| Long (43 characters) | about 1.0 s | about 2.2 s |
| VRAM | CPU uses about 1.1GB less | |
Short text was faster on the CPU; long text took about twice as long. My chat speaks one sentence at a time, so the delay before speech starts hardly changed. ComfyUI image generation speed didn’t change either with AivisSpeech on the CPU (about 6.4 s → 6.3 s per image).
If you read long passages in one go, keep the GPU. For conversation-style short sentences, the CPU is the better deal because it frees VRAM.
Command to check free VRAM
On NVIDIA GPUs, this shows total VRAM use. Compare before versus after unloading.
nvidia-smi --query-gpu=memory.used,memory.total --format=csv
Summary
- Before generating images or video, run
ollama stop MODEL_NAME. About 3.1GB (4B 4-bit model) is freed in 0.2 s; it reloads in about 3 s - Even if you do nothing, Ollama unloads after 5 minutes
- Start AivisSpeech Engine without
--use_gputo run it on the CPU and free about 1.1GB. Short sentences are about as fast
The settings I use to make videos with the freed VRAM are here:
To see at a glance what is using VRAM, I also built a local AI dashboard.
References
- FAQ - Ollama (How do I keep a model loaded in memory or make it unload immediately? / How can I tell if my model was loaded onto the GPU?)
- Aivis-Project/AivisSpeech-Engine - GitHub (--use_gpu, port 10101)
Test environment: RTX 2080 SUPER (8GB VRAM), Core i9-9900K, Windows 11, Ollama 0.34.0, model qwen3:4b-instruct-2507-q4_K_M. Measured on November 11, 2026.


コメント