Which Cloud Platforms Support Azure H100 and NC H100 GPUs? 

1. What Are Azure H100 and NC H100 GPUs? 

The NVIDIA H100 Tensor Core GPU (based on the Hopper architecture) is the most powerful AI accelerator available in the cloud today. It delivers transformational performance for generative AI training, large language model (LLM) inference, and high-performance computing (HPC).

Microsoft Azure offers the H100 NVL variant – a pair of PCIe-based H100 GPUs connected via NVIDIA NVLink, as the backbone of its NC H100 v5 VM series. This is the world’s first cloud instance to feature NVIDIA H100 NVL GPUs, announced by Microsoft at Ignite 2023 and now generally available.

Key reasons enterprises are moving to H100 on Azure:

• Up to 12x higher performance on GPT-3 175B compared to previous-generation GPUs
• 188 GB of HBM3 memory (dual-GPU config), enabling large model inference without memory bottlenecks
• Native integration with Azure’s enterprise security, compliance, and networking ecosystem
• Support for confidential computing – unique to Azure among H100 cloud providers

2. Azure H100 VM Series: Complete Breakdown 

Azure organizes its H100 GPU offerings into three distinct VM families, each targeting different workload profiles:

3. NCads H100 v5 – Specs & Use Cases 

The NCads H100 v5 series is Azure’s most accessible H100 VM, purpose-built for applied AI training and batch inference workloads.

Hardware Specifications

Processor: 4th-generation AMD EPYC™ Genoa (up to 96 non-multithreaded cores)
Network: Up to 80,000 Mbps (4 NICs)
Storage: Premium SSD supported; up to 240,000 IOPS / 7,000 MBps
Interconnect: NVIDIA NVLink (for dual-GPU config); InfiniBand not supported

Ideal Workloads

• GPU-accelerated analytics and databases
• Batch inferencing with heavy pre/post-processing pipelines
• Autonomy model training
• Oil & gas reservoir simulation
• Machine learning model development and fine-tuning
• Video processing at scale
• AI/ML web services and API backends

Key Limitation to Know

Live Migration and Memory Preserving Updates are not supported on NCads H100 v5 VMs. Plan your workload scheduling and maintenance windows accordingly.

4. ND H100 v5 – Specs & Use Cases 

The ND H100 v5 is Azure’s flagship GPU VM for organizations running large-scale deep learning training and tightly coupled HPC workloads.

What Makes It Different

This is not just a bigger version of the NC series. The ND H100 v5 is architecturally distinct:

• 8× NVIDIA H100 SXM Tensor Core GPUs per VM (vs. 1–2 NVL GPUs in NC series)
• 3.2 Tbps of interconnect bandwidth per VM
• Each GPU has its own dedicated 400 Gb/s NVIDIA Quantum-2 CX7 InfiniBand connection
• NVLink 4.0 for ultra-fast intra-VM GPU communication
• Supports GPU Direct RDMA — enabling direct memory access across GPUs in a scale-out cluster
• 96 physical 4th Gen Intel Xeon Scalable processor cores

Scale-Out Capability

ND H100 v5 deployments can scale to thousands of GPUs through Virtual Machine Scale Sets, all interconnected with the same 3.2 Tbps fabric. This is the architecture you need for:

• Training foundation models (LLMs, diffusion models, multimodal models)
• Tightly coupled HPC simulations
• Distributed deep learning with frameworks like PyTorch, TensorFlow, RAPIDS, Caffe
• Large-scale generative AI infrastructure

5. NCC H100 v5 – Confidential AI Computing 

The NCC H100 v5 is Azure’s answer to a critical question: how do you use H100 GPU acceleration while keeping sensitive data protected in memory?

These are Azure’s confidential VMs with NVIDIA H100 Tensor Core GPUs, now generally available. They allow customers to protect the confidentiality and integrity of data and applications while in use – meaning even the cloud provider cannot access your data during processing.

Who Needs This?

• Healthcare organizations processing protected health information (PHI) in AI pipelines
• Financial institutions running proprietary models on sensitive transaction data
• Government and defense agencies with strict data sovereignty requirements
• Enterprises subject to GDPR, HIPAA, or other data residency regulations who still want GPU acceleration

This combination of Trusted Execution Environments (TEE) and H100 GPU performance is currently unique to Azure among major cloud providers – a significant competitive differentiator for regulated industries.

6. Other Cloud Platforms Supporting NVIDIA H100 

While Azure-specific NC H100 v5 VMs are exclusive to Microsoft Azure, the underlying NVIDIA H100 GPU (in different configurations) is available on several other cloud platforms:

Amazon Web Services (AWS)

AWS offers H100-based instances through its P5 instance family, using H100 SXM GPUs. These are designed for large-scale ML training and tightly coupled HPC. AWS does not offer H100 NVL instances specifically.

Google Cloud Platform (GCP)

GCP offers NVIDIA H100 SXM GPUs through its A3 VM family, optimized for large-scale generative AI and HPC workloads. GCP’s strength is in integration with its TPU ecosystem and BigQuery ML.

CoreWeave

CoreWeave is a GPU-specialized cloud that offers H100 SXM and PCIe configurations with highly competitive pricing and fast provisioning. It’s particularly popular with AI startups and research organizations that need raw GPU access without the overhead of a full-stack cloud.

Lambda Cloud

Lambda offers on-demand H100 instances at accessible pricing, popular with ML researchers and smaller teams.

DigitalOcean, Northflank, and Others

Several mid-tier cloud providers are beginning to offer H100 GPU instances, mainly targeting developers and startups who need simpler GPU access without enterprise-level complexity.

7. Azure vs. Other Clouds: H100 Comparison 

Bottom line: If your workload involves enterprise compliance, confidential computing, or you’re already in the Azure ecosystem, Azure H100 VMs are the clear choice. If you’re purely optimizing for raw H100 SXM cost per hour, CoreWeave and Lambda are worth evaluating.

8. Who Should Use Azure H100 GPUs? 

Best Fit for Azure H100

Enterprises in regulated industries – Healthcare, finance, and government organizations that need GPU performance with data confidentiality guarantees only Azure currently provides.

Organizations in the Azure ecosystem – If you’re already using Azure Active Directory, Azure DevOps, Azure Blob Storage, or Azure Kubernetes Service (AKS), adding H100 GPU VMs is a natural extension with no additional vendor complexity.

Teams running large-scale LLM training or fine-tuning– The ND H100 v5’s InfiniBand interconnect and NVLink 4.0 topology is optimized specifically for the kind of distributed, gradient-synchronization-heavy training that foundation models require.

Applied AI and inference teams – The NCads H100 v5 hits a sweet spot for batch inference pipelines that need more power than A100-class VMs but don’t need the full 8-GPU ND series.

Consider Alternatives If…

• You’re a small team or startup with minimal compliance requirements and need the lowest cost per H100 hour (CoreWeave or Lambda may be cheaper)
• You’re deeply embedded in the GCP or AWS ecosystem and a cross-cloud migration isn’t worth the operational overhead
• You need real-time, ultra-low-latency inference at consumer scale (consider dedicated inference platforms)

9. How Cantech Can Help You Deploy on Azure H100 

At Cantech, we specialize in helping organizations design, deploy, and optimize GPU-accelerated AI infrastructure on Microsoft Azure. Whether you’re just evaluating H100 VMs or ready to migrate production workloads, we can accelerate your journey.
Checkout: NVIDIA H100 GPUs at the Best Price.

Frequently Asked Questions 

1. What is the difference between Azure NC H100 v5 and ND H100 v5?

The NCads H100 v5 uses NVIDIA H100 NVL GPUs (PCIe-based, 1–2 GPUs per VM) and is designed for applied AI inference and batch training. The ND H100 v5 uses 8× H100 SXM GPUs per VM with InfiniBand interconnect and NVLink 4.0, designed for large-scale distributed deep learning and HPC. If you’re training large models or need multi-node GPU clusters, ND H100 v5 is the right choice.

2. Is Azure the only cloud with H100 NVL GPUs?

Yes. The H100 NVL configuration (dual PCIe H100s linked via NVLink) is exclusive to Microsoft Azure’s NCads H100 v5 VMs. Other clouds offer H100 SXM or H100 PCIe in single configurations, but not the NVL dual-linked variant.

3. What regions are Azure H100 VMs available in?

Azure H100 VM series availability varies by region and is expanding. As of 2025, NCads H100 v5 and ND H100 v5 are available in select Azure regions in the US, Europe, and Asia Pacific. Check the Azure Products by Region page for current availability, or contact Cantech for current regional capacity guidance.

4. How much does an Azure H100 VM cost?

Azure H100 VM pricing varies by region and VM size. At standard on-demand rates, NC40ads H100 v5 (1 GPU) starts at approximately $3.50–$6/hour depending on region, while the NC80adis H100 v5 (2 GPUs) approximately doubles that. The ND H100 v5 (8 GPUs) is significantly more expensive. Spot pricing can reduce costs by 60–80%. Contact Cantech for a cost estimate tailored to your specific workload.

5. Can I run PyTorch or TensorFlow on Azure NC H100 v5?

Yes. Azure H100 VMs support all major ML frameworks including PyTorch, TensorFlow, JAX, Caffe, and RAPIDS. Microsoft provides Azure HPC images pre-configured with the necessary GPU drivers, CUDA toolkit, and networking software to get started quickly. Cantech can help you set up optimized Docker containers or Azure Machine Learning environments for your specific framework.

6. What is Azure NCC H100 v5 confidential computing?

Azure NCC H100 v5 VMs use hardware-based Trusted Execution Environments (TEEs) combined with NVIDIA H100 Tensor Core GPUs. This means your data and model weights are encrypted in memory even while being processed — the hypervisor and cloud provider cannot access your data. This is designed for highly regulated industries that need GPU acceleration without compromising data confidentiality.

7. Can I scale across multiple H100 VMs on Azure?

Yes. The ND H100 v5 series is built for multi-node scale-out via Virtual Machine Scale Sets. Deployments can span thousands of GPUs, all connected via 400 Gb/s InfiniBand per GPU. This architecture supports GPU Direct RDMA for efficient gradient sharing in distributed training. Cantech specializes in designing and managing these multi-node clusters.