Guides
Cloud GPU Rental: What Your Hourly Rate Actually Covers
Which workloads actually belong on rented hardware
Cloud GPU rental is a bad deal for anything that runs continuously at high utilisation, and a very good deal for anything bursty. Before you compare prices, sort your jobs into three buckets: batch work that finishes (fine-tuning, embeddings, offline evaluation), steady inference that runs all day, and spiky inference that peaks on weekday mornings.
Cloud GPU rental for AI work is mostly a matching exercise between three numbers: how much memory the model needs, how many tokens or samples per second you need, and how long the job runs. If any of those is fuzzy, you will over-provision and pay for capacity you never use. The batch bucket is where renting wins outright — you start a job, you stop paying when it ends, and you never carry depreciation on hardware that is obsolete in eighteen months.
What the hourly rate does and doesn't include
A quoted cloud GPU rental price is dollars per GPU-hour and almost nothing else. The line items that decide your actual bill sit outside it:
- Persistent storage. Volumes are billed per GB-month whether or not the GPU is running. A 500GB checkpoint directory left in place between runs can cost more than the training job itself.
- Egress. Pulling a dataset in is usually free; pushing results, logs or model weights out often is not.
- Idle time. Per-second billing is only cheap if you actually stop the instance. Plenty of teams leave a box running "just in case" and pay for 700 hours out of 720.
- Cold starts. Pulling a 20GB container image and loading weights can take longer than the inference batch you were about to run.
- Interruptions. Spot and interruptible tiers are cheaper per hour but can vanish mid-run, which costs you the whole job if you did not checkpoint.
None of these are hidden on purpose, but they are easy to miss when you compare two headline numbers side by side. Model them explicitly before you commit — a rough estimate beats a surprise invoice.
Comparing cloud GPU rental platforms on the things that matter
Cloud GPU rental platforms fall into three rough groups: marketplaces that aggregate independent hosts, managed clouds that hand you a container or a pod, and the large hyperscalers. Cloud GPU rental services differ far less in the silicon than in the wrapper around it — networking, storage semantics, orchestration, support, and how quickly capacity actually appears when you need it.
| What to check | Why it changes your bill | How to verify it |
|---|---|---|
| Billing granularity | Per-minute rounding on a 90-second job can double the effective rate | Read the billing docs, then run one tiny job and check the invoice |
| Storage persistence | Cheap GPU hours with expensive always-on volumes is a common trap | Price a volume separately from the instance |
| Interconnect | Multi-GPU training slows badly without NVLink or a fast fabric | Ask what the GPUs are connected with, not just how many there are |
| Availability | A cheap tier you cannot get when you need it is worth nothing | Check stock at the hours you actually run |
| Driver and image control | Locked-down images break custom CUDA builds | Test your own image before committing |
If you are choosing between the two best-known marketplaces, a direct head-to-head is more useful than reading feature lists: vastai vs runpod covers where they diverge on pricing model and host quality.
Cloud GPU rental in India: latency, payments and residency
Most cloud GPU rental India searches come from one of two situations. The first is a team serving users in Mumbai, Delhi or Bengaluru that wants to keep round-trip latency low. The second is a team that simply wants to pay in rupees without a cross-border card and the FX fees that come with it.
Both are legitimate, and they pull in opposite directions. Domestic regions cut network latency and simplify compliance, but GPU inventory in Indian regions is thinner than in US or European zones, so availability is spottier and per-hour pricing is often higher. If your users are in India but your training runs are offline, there is no latency argument for hosting the training job locally — only a data residency one. Check whether the rules you are actually subject to require local storage before you pay a premium for it.
Cross-border shopping and what changes when you buy abroad
Teams searching for alquiler gpu nube or gpu mieten cloud are usually looking for the same hardware with friendlier invoicing. The GPUs do not change. What changes is currency, VAT handling, support hours, and who is legally on the other end of the contract.
For European teams, a provider that can issue a reverse-charge VAT invoice saves real accounting time, and that alone can outweigh a small difference in hourly rate. For teams paying in pesos or rupees, watch the FX spread your card issuer applies — over a month of heavy usage it can exceed the gap between two providers' rates. Ask for a sample invoice before you sign up for anything long-running.
Sizing for LLMs: memory first, compute second
Cloud GPU rental for LLM work is dominated by memory, not raw throughput. Weights, KV cache and activations all have to fit, and the KV cache grows with context length and batch size. A model that fits comfortably at 2K context can fall over at 32K.
That is why 80GB cards such as the A100 and H100 are the default for anything above roughly 13B parameters in half precision, while 24GB cards handle smaller models, embedding pipelines and quantised inference well. Compare current H100 SXM rental prices against older parts before assuming you need the newest silicon — for memory-bound decoding, the throughput gap is often smaller than the price gap suggests. If you are sharding across multiple GPUs, check the interconnect: PCIe-only nodes can lose a large fraction of scaling efficiency on tensor-parallel workloads.
Build your own cost model before you commit
Write down five numbers: GPU count, expected hours per week, storage in GB, egress in GB, and the fraction of rented time you expect to actually be computing. Multiply the first two by the hourly rate, add storage and egress, then divide by your utilisation fraction. That last step is the one people skip, and it is the one that explains why two teams with identical workloads report very different monthly bills.
An hourly and monthly cost estimator will do the arithmetic, but the inputs are yours to get wrong. Run one real job on a small instance first, measure actual runtime and egress, and feed those numbers back in instead of guesses.
Once the model holds up, the last step is the boring one: pick a provider, run a small job, and check the invoice against your estimate. Marketplace rates move around, so look up today's rates on Vast.ai and see whether the market still agrees with your spreadsheet.