Samprix

What AI can the GeForce RTX 4090 run?

The GeForce RTX 4090 can give a local model 24 GB of memory. At a 4K context, 39 of the 42 models we track fit entirely on it; the largest is Qwen3 32B.

NVIDIA Specs last checked: how
Your computer
Context length
System RAM
Memory a model can use 24 GB 24 GB of VRAM · source

Models on this machine

  • Model file
  • Context
  • Allowance
  • Your GPU memory

Fits

39
  • Qwen3 32B
    Alibaba · Q4_K_M
    Needs 20 GB 3.9 GB spare
    Context up to
    19,962
  • Qwen2.5 32B
    Alibaba · Q4_K_M
    Needs 20 GB 3.9 GB spare
    Context up to
    19,962
  • DeepSeek-R1 Distill 32B
    DeepSeek · Q4_K_M
    Needs 20 GB 3.9 GB spare
    Context up to
    19,962
  • Qwen2.5 Coder 32B
    Alibaba · Q4_K_M
    Needs 20 GB 3.9 GB spare
    Context up to
    19,962
  • Qwen3 14B
    Alibaba · Q8_0
    Needs 16 GB 8 GB spare
    Context up to
    40,960
  • Mistral Small 24B
    Mistral AI · Q4_K_M
    Needs 14 GB 9.8 GB spare
    Context up to
    32,768
  • Mistral Small 22B
    Mistral AI · Q4_K_M
    Needs 13 GB 11 GB spare
    Context up to
    32,768
  • Qwen3 14B
    Alibaba · Q4_K_M
    Needs 9.8 GB 14 GB spare
    Context up to
    40,960
  • Phi-4 14B
    Microsoft · Q4_K_M
    Needs 9.8 GB 14 GB spare
    Context up to
    16,384
  • Qwen2.5 14B
    Alibaba · Q4_K_M
    Needs 9.6 GB 14 GB spare
    Context up to
    32,768
  • DeepSeek-R1 Distill 14B
    DeepSeek · Q4_K_M
    Needs 9.6 GB 14 GB spare
    Context up to
    82,564
  • Qwen2.5 Coder 14B
    Alibaba · Q4_K_M
    Needs 9.6 GB 14 GB spare
    Context up to
    32,768
  • Qwen3 8B
    Alibaba · Q8_0
    Needs 9.4 GB 15 GB spare
    Context up to
    40,960
  • Phi-3 medium 14B
    Microsoft · Q4_K_M
    Needs 9.3 GB 15 GB spare
    Context up to
    81,215
  • OLMo 2 13B
    Allen Institute for AI · Q4_K_M
    Needs 11 GB 13 GB spare
    Context up to
    4,096
  • Mistral Nemo 12B
    Mistral AI · Q4_K_M
    Needs 8.1 GB 16 GB spare
    Context up to
    108,233
  • Qwen3 8B
    Alibaba · Q4_K_M
    Needs 5.9 GB 18 GB spare
    Context up to
    40,960
  • DeepSeek-R1 0528 8B
    DeepSeek · Q4_K_M
    Needs 5.9 GB 18 GB spare
    Context up to
    131,072
  • DeepSeek-R1 Distill 8B
    DeepSeek · Q4_K_M
    Needs 5.6 GB 18 GB spare
    Context up to
    131,072
  • Dolphin 3.0 8B
    Cognitive Computations · Q4_K_M
    Needs 5.6 GB 18 GB spare
    Context up to
    131,072
  • Qwen2.5 7B
    Alibaba · Q4_K_M
    Needs 5.1 GB 19 GB spare
    Context up to
    32,768
  • Qwen2.5 Coder 7B
    Alibaba · Q4_K_M
    Needs 5.1 GB 19 GB spare
    Context up to
    32,768
  • DeepSeek-R1 Distill 7B
    DeepSeek · Q4_K_M
    Needs 5.1 GB 19 GB spare
    Context up to
    131,072
  • OLMo 2 7B
    Allen Institute for AI · Q4_K_M
    Needs 6.7 GB 17 GB spare
    Context up to
    4,096
  • Mistral 7B
    Mistral AI · Q4_K_M
    Needs 5.1 GB 19 GB spare
    Context up to
    32,768
  • Qwen3 4B
    Alibaba · Q4_K_M
    Needs 3.5 GB 21 GB spare
    Context up to
    40,960
  • Phi-4 mini 3.8B
    Microsoft · Q4_K_M
    Needs 3.3 GB 21 GB spare
    Context up to
    131,072
  • Phi-3 mini 3.8B
    Microsoft · Q4_K_M
    Needs 4.2 GB 20 GB spare
    Context up to
    4,096
  • Qwen2.5 3B
    Alibaba · Q4_K_M
    Needs 2.4 GB 22 GB spare
    Context up to
    32,768
  • Qwen2.5 Coder 3B
    Alibaba · Q4_K_M
    Needs 2.4 GB 22 GB spare
    Context up to
    32,768
  • Qwen3 1.7B
    Alibaba · Q4_K_M
    Needs 2.2 GB 22 GB spare
    Context up to
    40,960
  • DeepSeek-R1 Distill 1.5B
    DeepSeek · Q4_K_M
    Needs 1.6 GB 22 GB spare
    Context up to
    131,072
  • SmolLM2 1.7B
    Hugging Face · Q4_K_M
    Needs 2.3 GB 22 GB spare
    Context up to
    8,192
  • Qwen2.5 1.5B
    Alibaba · Q4_K_M
    Needs 1.5 GB 22 GB spare
    Context up to
    32,768
  • Qwen2.5 Coder 1.5B
    Alibaba · Q4_K_M
    Needs 1.5 GB 22 GB spare
    Context up to
    32,768
  • TinyLlama 1.1B
    TinyLlama · Q4_K_M
    Needs 1.2 GB 23 GB spare
    Context up to
    2,048
  • Qwen3 0.6B
    Alibaba · Q4_K_M
    Needs 1.4 GB 23 GB spare
    Context up to
    40,960
  • Qwen2.5 0.5B
    Alibaba · Q4_K_M
    Needs 0.9 GB 23 GB spare
    Context up to
    32,768
  • SmolLM2 360M
    Hugging Face · Q4_K_M
    Needs 0.9 GB 23 GB spare
    Context up to
    8,192

Won’t fit

3
Memory needed = model file + context + a 0.5 GB allowance (our estimate). See methodology.

Frequently asked questions

What’s the biggest AI model the GeForce RTX 4090 can run?

Qwen3 32B (Q4_K_M, a 20 GB download) is the largest of the 42 models we track that fits entirely in the 24 GB the GeForce RTX 4090 can give a model at a 4K context, with 3.9 GB to spare and room for up to 19,962 tokens of context.

Can the GeForce RTX 4090 run DeepSeek-R1 Distill 70B?

Not entirely in its own memory. DeepSeek-R1 Distill 70B (Q4_K_M) needs 42 GB at a 4K context, and the GeForce RTX 4090 can give a model 24 GB. With enough system RAM the rest can spill over into it, which works but is much slower; pick your RAM above to check.

How much memory can the GeForce RTX 4090 give an AI model?

All 24 GB of its VRAM, per NVIDIA GeForce graphics card comparison (checked Oct 2, 2026). The model file, its context and the program running it all have to fit in that.

How fast will local AI run on the GeForce RTX 4090?

Hardware makers and model publishers don’t publish tokens-per-second figures, so there is no source we could cite. We only show whether a model fits and how much context it leaves room for.

Other graphics cards and Macs

Did this tool do what you needed?

Related tools

Ollama & LM Studio Beginner’s Guide

Set up local AI on Windows or Mac, step by step.

Local AI Electricity Cost Calculator

What running a model on your own hardware costs, compared with the API.

AI Subscription vs API Calculator

See how many messages a day it takes before a ChatGPT, Claude or Gemini plan beats paying per token.

© 2026 Samprix. Not affiliated with OpenAI, Anthropic or Google.

PrivacyTerms