GPU VRAM Calculator
Estimate the GPU VRAM needed to run or train a transformer / large language model. Enter parameter count, precision (fp16, int8, int4…), context length and batch size to break down weights, KV cache, activations, gradients and optimizer state, and check which GPUs it fits on.
Authorizations
Bearer authentication header of the form Bearer <token>, where <token> is your auth token.
Body
Model Preset
custom, gpt2-124m, llama-3-2-1b, llama-3-2-3b, mistral-7b, llama-3-8b, llama-2-13b, mixtral-8x7b, llama-3-70b, llama-3-1-405b Parameters (B)
0.001 <= x <= 2000Hidden Dim
64 <= x <= 65536Layers
1 <= x <= 256Workload
inference, training Precision
fp32, fp16, int8, int4 Batch Size
1 <= x <= 4096Sequence Length
1 <= x <= 1048576Optimizer
adam, sgd-momentum, sgd Mixed precision (fp32 master weights + grads)
Gradient checkpointing (recompute activations)
Response
Tool output
The tool's slug, echoing the {slug} in the request path.
Output-contract version for this tool.
Credits this call consumed, after any settlement refund. 0 when metering is disabled.
Credits left in the current monthly allowance, or null when metering is disabled.
Correlation id, also sent as x-request-id.