Guides
Cloud GPU Pricing Comparison: A Practical Guide
If you rent GPUs every week, you know the hourly sticker price is only the start. I’ve spent the last year moving training jobs between AWS, Azure, GCP, Lambda, and Vast, and the real cost differences are rarely where people expect. This cloud GPU pricing comparison is about what you actually pay per hour once you factor in storage, network, idle time, and spot interruptions. Not the marketing page — the invoice.
Why the List Price Doesn’t Tell the Whole Story
Every provider publishes a per-hour rate for each GPU. But when you compare cloud GPU prices per hour, you’re comparing more than silicon. You’re comparing the entire operational context. A $2.50/hour H100 on a bare-metal provider with no data egress fee can be cheaper than a $2.10/hour H100 on a hyperscaler once you add the cost of moving checkpoints out.
For training, the hidden costs are: storage (often $0.10–$0.20 per GB-month), inter-node bandwidth, and the time you waste waiting for spot instances to come up. For inference, the hidden cost is cold start latency and whether you’re paying for idle replicas during traffic troughs. My rule: always compute the effective hourly cost — total bill divided by actual compute hours, not wall-clock hours.
AWS vs Azure vs GCP GPU Pricing: The Hyperscaler Reality
When I run an aws vs azure vs gcp gpu pricing check for the same generation, the raw numbers are close. A100 40GB on-demand runs roughly $3–$4/hour on all three. H100s are $6–$8/hour. But the moment you look at commitment discounts, the gap widens. Azure’s reserved instances can drop H100 to $4.50/hour if you commit for a year. GCP’s committed use discounts are similar. AWS’s Savings Plans work, but the pricing is less transparent.
What actually matters is the ecosystem. If you’re already in GCP with data in Cloud Storage, the egress is free. If you’re on AWS, moving 5TB of training data out to another provider can cost $90. That’s one hour of H100 time wasted on a transfer fee. So when you do a cloud GPU cost comparison for ai, include your data gravity.
Spot and Preemptible: The Real Price Cut
Spot prices are where the cheapest cloud GPU per hour usually lives. On GCP, preemptible A100s can drop to $1.20/hour. Azure’s spot is similar. AWS spot for A100 is often $1.50–$2.00. But you get interrupted. For fine-tuning a small model with checkpointing every 5 minutes, spot is a no-brainer. For a long, monolithic training run without resumption, you’ll pay more in restart time than you save.
Comparing the Smaller Providers: Where the Value Actually Is
Beyond the big three, the cost landscape changes. Lambda Labs, RunPod, Vast.ai, and CoreWeave offer H100s at $2–$3/hour on demand. That’s half the hyperscaler price. The trade-off: less mature networking, fewer regions, and support that’s a ticket queue, not a phone call. I’ve run 8-GPU training jobs on Lambda with NVLink and it worked fine. But I’ve also seen Vast instances disappear mid-run with no refund.
For a cloud GPU rental price comparison, you need to value your own time. If a cheap provider saves $20/hour but costs you 2 hours of debugging a flaky network, the cheap option isn’t cheap.
A Concrete Example: Fine-Tuning a 7B LLM
Last month, I fine-tuned a 7B model on 8×A100s. On AWS with on-demand, that’s about $32/hour. The run took 6 hours — $192. On GCP with preemptible, it would have been $8/hour but I’d have hit 2 interruptions, adding maybe 1.5 hours of restart time. Total: $60. That’s a 70% savings. But if my code wasn’t checkpoint-safe, I’d have lost progress and paid more. The cheapest cloud GPU per hour is only cheap if your workload tolerates interruption.
Cloud GPU Cost Comparison for AI: Inference vs Training
For inference, the game changes. You’re not renting a GPU for 6 hours; you’re renting it 24/7. At $3/hour, an A100 costs $2,160/month. That’s the price of a dedicated box from some providers. But if you only need 10% utilization, on-demand is wasteful. Serverless inference or spot for low-latency workloads? Not a thing. You need steady capacity.
When I do a cloud GPU cost comparison for ai inference, I look at price per token, not per hour. A provider with a $2.50/hour H100 but poor batching might serve fewer tokens per dollar than a $4/hour H100 with better software. Benchmarks matter. Don’t assume the hourly rate maps to cost per request.
A Quick Comparison Table (Generic, Not Exact)
| Provider Type | Typical H100 On-Demand | Typical A100 On-Demand | Best For |
|---|---|---|---|
| Hyperscaler (AWS/Azure/GCP) | $6–$8/hr | $3–$4/hr | Reliability, integrated services, SLAs |
| Specialized (Lambda, CoreWeave) | $2–$3/hr | $1.50–$2.50/hr | Bare-metal, large multi-GPU, no egress fees |
| Marketplace (Vast, RunPod) | $1.50–$3/hr | $0.80–$1.50/hr | Cheapest cloud GPU per hour, spot-like flexibility |
Those numbers are ranges I’ve seen in the last quarter; they’ll drift. The point is the order of magnitude.
How to Do Your Own Cloud GPU Pricing Comparison
Don’t trust a single blog post, including this one. Prices change weekly. Here’s the process I use:
- List your exact GPU and VRAM requirement. A 70B model needs 80GB+; a 7B fits in 24GB.
- Check three providers: one hyperscaler, one specialized, one marketplace. Use their pricing APIs or dashboards.
- Add data egress costs. If you move data in and out daily, that’s a monthly line item.
- Estimate your interruption tolerance. Spot is only for resumable workloads.
- Compute effective hourly cost: total bill / actual compute hours.
That last step is the one most people skip. You end up comparing cloud GPU prices per hour from a list, but the real cost is hidden in your workflow’s friction.
Final Thoughts: The Cheapest Hour Isn’t Always the Best Deal
I’ve seen engineers burn $500 on a “cheap” provider because the network couldn’t handle multi-node training. I’ve also seen teams save 60% by moving to preemptible and adding checkpointing. The cloud GPU pricing comparison isn’t about finding the lowest number on a website — it’s about matching the price to your workload’s reality. Start with your data, your tolerance for interruptions, and your actual utilization. Then pick a provider. That’s how you’ll keep your per-hour costs sane.