Guides
L40S vs A100 price: which card costs less per job
Comparing L40S vs A100 price on a rental marketplace is rarely as simple as picking the lower hourly number. The two cards were designed for different jobs. The L40S is a 48 GB Ada Lovelace part that leans toward inference, mixed-precision work and single-GPU tasks. The A100 is an Ampere data centre card with HBM memory and, on the SXM variant, NVLink for tightly coupled multi-GPU runs. Which one is cheaper depends almost entirely on what you ask it to do.
Start with the workload, not the price tag
Before opening two price tabs, write down three things: the model size in parameters, the sequence length you actually serve or train at, and whether the job needs more than one GPU to fit. Those three answers rule out one of the two cards more often than any price comparison will. A 7B model served at 4k context fits comfortably on either. A 70B model at long context, or a training run that needs tensor parallelism across four GPUs, pushes you toward A100 SXM and its NVLink fabric. Once the card is settled, price shopping becomes a much smaller problem.
How the two cards differ in ways that affect cost
Rental rates follow the hardware underneath them, so it helps to separate the differences that cost money from the ones that only look good on a spec sheet.
| Attribute | L40S | A100 |
|---|---|---|
| Architecture | Ada Lovelace | Ampere |
| Memory | 48 GB GDDR6 with ECC | 40 GB or 80 GB HBM, depending on variant |
| Interconnect | PCIe only, no NVLink | PCIe on the PCIe card, NVLink on SXM |
| Low-precision formats | FP8 supported natively | No FP8; FP16 and INT8 tensor core paths |
| Where it tends to fit | Inference, quantised models, single-GPU fine-tuning | Training, multi-GPU jobs, bandwidth-heavy serving |
Two rows in that table drive most of the price gap. HBM costs more to produce than GDDR6, and NVLink adds board complexity and power draw. The third factor is demand: A100 capacity has been rented hard for years, and marketplaces price against the utilisation they can realistically achieve.
Where the L40S vs A100 price gap actually comes from
- Memory technology. HBM on the A100 is faster and more expensive than the GDDR6 in the L40S, and that shows up directly in the hourly rate.
- Interconnect. NVLink makes multi-GPU jobs practical, and providers charge for that capability whether or not your job uses it.
- Power and cooling. A higher board power draw means more expensive hosting, which is passed on per rented hour.
- Supply and age. The A100 has been in production longer and is still in demand, so its pricing is less likely to fall than newer mid-range cards.
- Marketplace competition. The same model can differ noticeably between hosts on any given day, which is why comparing listings matters more than comparing spec sheets.
If you want a concrete baseline, the current A100 PCIe rental prices are a useful anchor, because that variant sits closest to the L40S in both capability and typical rate.
Where the L40S usually wins on value
Inference at moderate batch sizes
Serving a model that fits in 48 GB, at batch sizes below the point where memory bandwidth saturates, is the L40S's home turf. You get one card, no interconnect to pay for, and enough VRAM for a 13B model in FP16 or a much larger one quantised. When the L40S undercuts the A100 per hour, this is the workload where that saving turns into real money.
Quantised and FP8 workloads
Ada Lovelace supports FP8 natively, and most modern serving stacks have a path for it. If you have already quantised to INT8 or FP8, the L40S will often deliver comparable throughput to an A100 at a lower hourly rate. The catch is that FP8 support is not automatic: you need a runtime that actually uses it, and you should measure tokens per second rather than trusting the format name.
Single-GPU fine-tuning
LoRA and other parameter-efficient methods keep memory use low enough that a 48 GB card handles a lot of fine-tuning work. If your job never spans two GPUs, the A100's NVLink advantage is dead weight you are paying for. A 24 GB card such as an RTX 4090 rental can be cheaper still for small models, though it lacks ECC memory and the data centre drivers production teams usually want.
Where the A100 justifies its rate
Multi-GPU training
Once a job needs tensor or pipeline parallelism, NVLink changes the economics. Two A100 SXM cards connected properly can behave like one larger device; two L40S cards have to exchange gradients over PCIe, and the communication overhead grows with model size. You pay more per hour and finish sooner, which is often the better trade.
Bandwidth-bound inference
Decode-heavy, long-context serving is limited by memory bandwidth more than by compute. HBM gives the A100 an edge there, and the 80 GB variant holds longer contexts without sharding. If your p95 latency target is tight and your context is long, the cheaper card may not meet the SLA at all.
A stable software baseline
The A100 has been the default training card for years, so most frameworks, kernels and published benchmarks assume it. Reproducing someone else's result is easier on the same hardware. That is not a performance argument, but it is a real cost when your team's time is the expensive part.
Comparing quotes without being misled
Hourly rates on marketplaces are quoted in ways that are not directly comparable. Before you declare one card cheaper, normalise the quote.
- Billing granularity. Per-second billing with a short minimum is different from a one-hour minimum, especially for bursty inference.
- Interruptible versus on-demand. Spot capacity can be far cheaper, but only if your job checkpoints and restarts cleanly.
- Attached resources. vCPU, RAM and disk are sometimes bundled and sometimes billed separately.
- Storage and egress. Model weights are large, and repeated downloads add up quietly.
- Utilisation. A card that is 30 percent faster at the same rate is the cheaper card, even when the sticker number looks worse.
The honest metric is cost per unit of finished work: per million tokens served, per training step, or per fine-tune completed. Our hourly and monthly cost estimator lets you plug in the rate you were quoted alongside the runtime you measured, so you can compare two cards on the same footing instead of arguing about list prices.
Pick the card, then verify the rate
If your workload fits on one GPU and you are not bandwidth-starved, start with the L40S and check whether it meets your latency target. If it needs NVLink, long context, or an 80 GB memory pool, start with the A100 and treat the higher rate as the price of finishing the job. Either way, the rate you saw last month is not the rate you will pay today.
Rates move with supply, so the only figure worth trusting is the one on the listing page right now: check what each card is going for on Vast.ai.