NVIDIA H100
NVIDIA lists the H100 SXM with 80GB of memory at 3.35TB/s, the H100 NVL with 94GB at 3.9TB/s, and the H100 PCIe with 80 GB HBM2e at 2,000 GB/s. Specs below are from NVIDIA's own spec sheets; rental prices are from each cloud's own price list. 8 of the GPU clouds tracked here price the H100.
Specifications
Figures as NVIDIA prints them, one table per NVIDIA source, each with the date it was read. Hover a figure for the line it was read from.
H100 SXM and H100 NVL
Figures from the Specifications table on NVIDIA's H100 product page. Figures marked * are with sparsity, as NVIDIA's footnote states.
| Spec | H100 SXM | H100 NVL |
|---|---|---|
| GPU memory | 80GB | 94GB |
| Memory bandwidth | 3.35TB/s | 3.9TB/s |
| FP8 Tensor Core | 3,958 teraFLOPS* | 3,341 teraFLOPS* |
| INT8 Tensor Core | 3,958 TOPS* | 3,341 TOPS* |
| FP16 Tensor Core | 1,979 teraFLOPS* | 1,671 teraFLOPS* |
| BF16 Tensor Core | 1,979 teraFLOPS* | 1,671 teraFLOPS* |
| TF32 Tensor Core | 989 teraFLOPS* | 835 teraFLOPs* |
| FP32 | 67 teraFLOPS | 60 teraFLOPs |
| FP64 Tensor Core | 67 teraFLOPS | 60 teraFLOPs |
| FP64 | 34 teraFLOPS | 30 teraFLOPs |
| Max thermal design power (TDP) | Up to 700W (configurable) | 350-400W (configurable) |
| Interconnect | NVIDIA NVLink™: 900GB/s PCIe Gen5: 128GB/s | NVIDIA NVLink: 600GB/s PCIe Gen5: 128GB/s |
| Form factor | SXM | PCIe dual-slot air-cooled |
| Multi-Instance GPU (MIG) | Up to 7 MIGS @ 10GB each | Up to 7 MIGS @ 12GB each |
H100 PCIe
Figures for the 80GB H100 PCIe card from NVIDIA's product brief PB-11133-001_v02 (November 2022).
| Spec | H100 PCIe |
|---|---|
| GPU memory | 80 GB HBM2e |
| Memory bandwidth | 2,000 GB/s |
| Max thermal design power (TDP) | 350 W |
| Form factor | Dual-slot 10.5-inch PCI Express Gen5 card, full-height full-length (FHFL) |
| Multi-Instance GPU (MIG) | Supported (seven instances) |
No row here for FP8 Tensor Core, INT8 Tensor Core, FP16 Tensor Core, BF16 Tensor Core, TF32 Tensor Core, FP32, FP64 Tensor Core, FP64, Interconnect: none is recorded from this source for the H100 PCIe, and none is carried over from the H100 SXM or H100 NVL, which are different cards.
What renting an H100 costs
12 hourly prices from 8 clouds, each from the cloud's own price list. Alphabetical by cloud and not ranked: some price per GPU and others a whole multi-GPU node, and the unit is on every row, with the date the list was read.
| Cloud | As listed | VRAM | Per hour | Read |
|---|---|---|---|---|
| CoreWeave | NVIDIA HGX H100 | 80 GB | $49.24 for 8 GPUs | 17 Oct |
| Crusoe Cloud | NVIDIA H100 80GB HGX | — | $4.50 per GPU | 17 Oct |
| Hyperstack | NVIDIA H100 | 80 GB | $2.50 per GPU | 17 Oct |
| Hyperstack | NVIDIA H100 NVLink | 80 GB | $2.60 per GPU | 17 Oct |
| Hyperstack | NVIDIA H100 SXM | 80 GB | $3.20 per GPU | 17 Oct |
| Lambda | NVIDIA H100 PCIe | 80 GB | $3.29 per GPU | 17 Oct |
| Lambda | NVIDIA H100 SXM | 80 GB | $4.29 per GPU | 17 Oct |
| Nebius AI Cloud | NVIDIA HGX H100 | — | $4.50 per GPU | 17 Oct |
| Paperspace | H100 | — | $5.95 per GPU | 17 Oct |
| RunPod | H100 SXM | 80 GB | $3.99 per GPU | 17 Oct |
| Vast.ai | H100 SXM | — | $1.73 per GPU | 17 Oct |
| Vast.ai | H100 PCIE | — | $2.59 per GPU | 17 Oct |