SDXL on 8GB VRAM: --medvram Cut Generation from 18 Min to Under 1

スポンサーリンク
SDXL on 8GB VRAM: --medvram Cut Generation from 18 Min to Under 1 Generative AI
スポンサーリンク

I generate SDXL images with Stable Diffusion web UI (AUTOMATIC1111, “A1111” below) on an 8GB VRAM card (RTX 2080 SUPER). My launch options include --xformers --medvram --no-half-vae, but I had never checked how much each one actually helps.

So I removed them one at a time and generated one image under identical conditions each time. Short answer: keep all three. Without --medvram in particular, generation took 18 minutes with no error at all.

Launch options Generation time What happened
All three (my usual setup) 40.1 s (36.1 s on the second run) Generated normally
Without --no-half-vae 1 min 0.1 s NaN in the VAE; it retried in 32-bit, then finished
Without --medvram 18 min 28.2 s No error, just extremely slow
Without --xformers 47.9 s Generated normally, slightly slower

I couldn’t see any difference between the resulting images. Only the time differed.

If you use SDXL models with 8GB of VRAM, put at least these three on the COMMANDLINE_ARGS line of webui-user.bat:

set COMMANDLINE_ARGS=--xformers --medvram --no-half-vae

I also use --listen --port 7861 --enable-insecure-extension-access (to use it from another PC on my home network and to install extensions). They’re unrelated to this comparison, so I won’t cover them.

Test environment, conditions

  • GPU: NVIDIA GeForce RTX 2080 SUPER (8192MiB VRAM), driver 610.88
  • A1111: v1.10.1 (Python 3.10.6)
  • Model: songmix_v22 (SDXL-based, 6.62GB; the startup log shows sd_xl_base.yaml)
  • Size: 768 x 1344
  • Sampler: DPM++ 2M, Schedule type: Simple, 25 steps, CFG scale 6
  • Seed: 12345 (fixed so every run produces the same image)
  • Hires. fix off, batch count / batch size 1
  • Prompt: identical every time

I restarted the WebUI after every change of launch options. Before starting I copied webui-user.bat so I could restore it:

cd "D:\Program Files\Stable\stable-diffusion-webui"
Copy-Item webui-user.bat webui-user.bat.bak

Afterwards, Copy-Item webui-user.bat.bak webui-user.bat -Force restores the original.

Baseline: all three options

First, the baseline with my usual setup. “Time taken” below the image is the generation time.

Result with the usual settings: Time taken 40.1 sec.

The first run took 40.1 s. The first run can be slow due to warm-up, so I ran it again under the same conditions: 36.1 s.

Second run: Time taken 36.1 sec.

Without --no-half-vae: NaN, then slower because of the retry

--no-half-vae keeps the VAE (the final stage that turns the result into an image) from running in 16-bit. Without it, the image still came out, but the console showed:

Console message: A tensor with all NaNs was produced in VAE
A tensor with all NaNs was produced in VAE.
Web UI will now convert VAE into 32-bit float and retry.
To disable this behavior, disable the 'Automatically revert VAE to 32-bit floats' setting.
To always start with 32-bit VAE, use --no-half-vae commandline flag.

A1111 retried automatically, so the image was fine, but it took 1 min 0.1 s, about 20 seconds more than the baseline.

Without --no-half-vae: Time taken 1 min. 0.1 sec.

If the “Automatically revert VAE to 32-bit floats” setting is off, there is no automatic retry. I didn’t test what happens then.

Without --medvram: no error, 18 minutes

--medvram trades a little speed for lower VRAM use. Without it, generation took 18 min 28.2 s.

Without --medvram: Time taken 18 min. 28.2 sec.

The console showed about 40 seconds per step (39.80s/it). With --medvram (the run without only --no-half-vae) it was 1.70 steps per second (1.70it/s), so each step took more than 60 times longer.

Console without --medvram: 25/25 16:35, 39.80s/it

I expected a “CUDA out of memory” error if VRAM ran short, but none appeared. Because it just gets slow without any error, a missing --medvram is easy to overlook. If generation suddenly takes many minutes, check your launch options first.

“A: 9.68 GB” below the image (the peak memory actually used by tensors, explained below) exceeds 8GB on an 8GB card. NVIDIA drivers can use system RAM when VRAM runs out (“CUDA - Sysmem Fallback Policy” in the NVIDIA Control Panel). I believe that’s why it slowed down instead of failing, though I haven’t verified that setting.

Without --xformers: 20-30% slower

--xformers runs part of the computation (cross attention) with the xformers library. Without it, the console said “No module ‘xformers’. Proceeding without it.” and the Doggettx method was used instead.

No module 'xformers'. Proceeding without it. / Applying attention optimization: Doggettx... done.

It took 47.9 s, roughly 20-30% slower than the baseline’s second run (36.1 s).

Without --xformers: Time taken 47.9 sec.

About VRAM usage numbers

Next to the time, below the image, A1111 shows “A: x GB, R: x GB, Sys: x/8 GB”. Hovering over the text shows what they mean (from the A1111 source):

  • A (Active): peak VRAM actually used by tensors during generation, excluding cache
  • R (Reserved): peak VRAM reserved by PyTorch (A1111’s compute library), including cache kept for reuse
  • Sys (System): peak VRAM used by the whole PC, including other apps, versus the card’s total

A and R peaks are reset for every generation. My values:

Launch options A R Sys
All three 5.21GB 10.57GB 8.0/8GB (100%)
Without --no-half-vae 5.21GB 11.45GB 8.0/8GB (100%)
Without --medvram 9.68GB 12.48GB 8.0/8GB (100%)
Without --xformers 5.67GB 11.31GB 8.0/8GB (100%)

Sys was 100% in every case, so the card’s VRAM was always full and Sys can’t distinguish the setups.

R exceeded 8GB every time: PyTorch reserved more than the card’s 8GB, presumably with the driver making up the difference from system RAM (not verified).

A exceeded 8GB only without --medvram (9.68GB). With --medvram it stayed at 5.2-5.7GB, within 8GB. If the tensors themselves didn’t fit in VRAM, data had to move to system RAM every step, which would explain the 18 minutes. That’s also my inference.

Summary

For SDXL models on 8GB of VRAM, keep --xformers --medvram --no-half-vae. Removing any of them still produced an image, but none made anything faster. Removing --medvram in particular silently stretches generation to 18 minutes, so suspect it first when generation suddenly gets slow.

Related posts

Video generation with ComfyUI on the same 8GB card:

References

コメント

Copied title and URL