Start from your workload and we'll recommend a GPU type & count, then carry it straight into the cost and returns model. Every number stays editable at every step.
Not sure which fits? Either sizing path only takes a couple of minutes and you can switch or override at any time. New here — read the plain-English guide first.
Size for Inference STEP 2 OF 3
Pick a model + use case → recommended GPU type & count, with the full math shown.
Sizing is a planning heuristic based on published model architectures — not a capacity guarantee. Override anything.
1. Model
This model routes each token to a few experts (Mixture-of-Experts). "Routed experts" models what a decode step really reads: the shared layers plus every expert at least one token in the batch was routed to — near the active-param count at batch 1, approaching the full expert set at large batches. "All experts every step" assumes every expert's weights are read and computed every token (an upper bound). Weight memory always holds the full expert set either way.
FP4 formats run on FP4 tensor cores on Blackwell and MI355X. On Hopper/MI300 they still save memory but compute at BF16 rate (weight-only quantization).
Serving stacks (vLLM, SGLang, TensorRT-LLM) quantize the KV cache separately from the weights — it is never stored at 4-bit here.
2. Use Case (Interactive Inference)
Visible answer tokens only — see "Reasoning tokens" below for thinking-mode models.
For thinking/reasoning models (e.g. DeepSeek-R1, o1-style) that generate a hidden chain-of-thought before the visible answer. These tokens are generated autoregressively exactly like any other output token — same compute, bandwidth, and KV cache cost — so leaving this at 0 for a reasoning workload will undersize the cluster. Typical reasoning traces run 1,000–8,000+ tokens depending on task difficulty. Leave at 0 for non-reasoning models.
Time to deliver a complete response (prompt processing + all output tokens). It sets the per-user speed every stream must get: (output + reasoning tokens) ÷ (response time − time-to-first-token). Bigger batches raise total throughput but slow each stream, so this caps how many sessions one replica can serve.
3. GPU
4. Advanced / Contested Assumptions
Fraction of nameplate VRAM actually usable after CUDA context, framework runtime, etc.
Discount for NVLink all-reduce traffic when one replica spans several GPUs.
Only used when a model is too big for one NVLink domain and is split into pipeline stages across nodes (bubbles + inter-node hops).
The most contested number here. What fraction of the hardware roofline (memory bandwidth or tensor-core FLOPs, whichever binds) your serving stack actually achieves — kernel efficiency, scheduling gaps, sampling, CPU overhead. 50% reproduces published MLPerf Llama-70B H100 results within ~20%; tuned TensorRT-LLM can exceed it, an untuned stack falls well below. Calibrate it from your own deployment in section 5.
5. Calibrate from Real Deployment
From your own vLLM/SGLang/TensorRT-LLM deployment: sustained output tokens/sec on ONE replica with the topology shown above (same model, precision, GPU and prompt/response lengths), and how many requests were in flight. This back-solves serving efficiency from real data and remembers it in this browser.
Memory Constraint — GPUs per Replica
Throughput Constraint — Replicas Needed
Estimated Cost per Million Tokens
Override
Recommended: — GPUs. Carry this into the cost & returns model.
Size for Training / Fine-tuning STEP 2 OF 3
Pick a model + training job → recommended GPU type, count, and expected training time.
Sizing is a planning heuristic based on published model architectures — not a capacity guarantee. Override anything.
1. Model
This model routes each token to a subset of experts (Mixture-of-Experts). "Active experts" computes training FLOPs off only the experts actually touched per token — the standard MoE training-compute convention (matches how DeepSeek-V3/Mixtral report their own training compute). "Full / all experts" instead assumes every expert is touched every token, a conservative upper bound. Memory sizing always accounts for the full expert set either way — all experts' weights/optimizer states must stay resident regardless of this toggle.
Mixed-precision training keeps FP32 master weights and Adam state either way (16 bytes/param of training state); this sets the compute rate and the frozen-weight size under LoRA.
Off = the standard 6 × params × tokens. On adds the attention work (12 × heads × head-dim × attended tokens per layer: scores and value mixing, forward + backward) — material at long sequence lengths; sliding-window layers only count their window.
ZeRO-3/FSDP shards weights, gradients, and optimizer state evenly across every GPU in the job — how virtually all production training actually runs today. Legacy assumes a fixed-size group sized to fit one full copy, then cloned whole for more compute (can overstate GPU count when your compute need lands just past a clone boundary).
2. Training Job
3. GPU
4. Advanced / Contested Assumptions
Applied on top of Korthikanti et al. 2022's activation-memory formula. 1.0x = paper baseline; flash-attention-style kernels typically land under 1.0x, older/naive implementations can exceed it.
The most contested number here. Same role as inference's serving efficiency.
Discount for cross-GPU communication as you add GPUs. Under ZeRO-3/FSDP this covers real added traffic (parameter all-gather on top of the gradient reduce, ~1.5x baseline comm volume per Rajbhandari et al. 2019) — real large runs report 55-60% sustained MFU with good overlap, down to ~7% MFU regression scaling 8→512 GPUs (Zhao et al. 2023).
Memory Constraint — GPU count for model+data parallelism
Compute Constraint — GPUs to hit target training time
Override
Recommended: — GPUs. Carry this into the cost & returns model.
Cost & Returns STEP 3 OF 3
GPU total cost of ownership, unit economics, and a simplified 3-statement return model for a GPU-as-a-Service operator
Runs entirely client-side in your browser. All figures are illustrative planning assumptions, not vendor quotes.
1. GPU & Cluster
Fully-loaded system cost per GPU: compute tray, NVLink/switch fabric, networking, local storage, integration. GB200 figure is rack price ÷ 72.
Includes host CPU/DRAM/NIC/storage share, not just GPU TDP.
Used to size the platform licensing fee below.
2. Power & Facility Cost
Bundled power + space + cooling per kW of critical IT load.
Annual cost = IT load (kW) × 8,760 hrs × PUE × electricity rate.
3. Financing
Default 50% at 8% assumes take-or-pay customer contracts backing the debt. Speculative (uncontracted) capacity typically raises ~35% at 10–11%.
4. Depreciation & Useful Life
Straight-line depreciation to salvage value.
5. Revenue & Pricing
Set from the GPU preset when you pick a GPU (Oct-2026 market list price, 1-yr committed). Overwrite with your own price book.
GPU rental $/hr tends to fall over time as newer accelerators arrive and supply catches up with demand.
6. Opex & Tax
Default 37% = Pakistan FY2026-27 (Finance Act 2026): 29% corporate rate + 8% super tax on income above PKR 500m. Use 29% below that threshold, ~0.25% final tax on qualifying IT-export revenue (s.154A, to TY2029), or ~25% for a US federal + state basis. Minimum tax (1.25% of turnover) and provincial sales tax on services are not modelled.
7. Multi-Tenancy Platform
Toggle sets which scenario drives the Financial Statements & Returns tabs. Both are always compared side-by-side in the "Multi-Tenancy ROI" tab.
Multi-tenant pooling lets you sell more vGPU-hours than physical GPU-hours exist.