News

Azure AI Foundry Gets NVIDIA Tech

• By David Ramel
• 03/21/2025

AI powerhouse NVIDIA flexed its muscle at its GTC 2025 conference this week where several partnerships with Microsoft were announced, mostly concerning Microsoft's Azure AI Foundry offering.

The Azure AI Foundry platform, formerly called Azure AI Studio, helps cloud developers build, deploy, and manage advanced AI solutions using cutting-edge tools, models, and seamless integration with Azure's ecosystem.

In turn, that ecosystem is now further integrated with NVIDIA technology, announced in multiple posts published throughout the week. This integration introduces NVIDIA NIM microservices, which are optimized tools that simplify the deployment of AI applications, making the process faster and more efficient. Additionally, the inclusion of NVIDIA's AgentIQ toolkit allows for real-time monitoring and optimization of AI agents, improving their performance while reducing operational costs. These and other advancements aim to streamline the development and management of AI projects, enabling businesses to implement AI solutions more effectively.

Here's a summary of the news:

Integration of NVIDIA NIM into Azure AI Foundry
NVIDIA NIM (Inference Microservices) has been incorporated into Azure AI Foundry. These pre-packaged containers are optimized for deploying generative AI applications and agents, supporting over two dozen foundation models. This enhancement accelerates inferencing workloads and simplifies the deployment process for developers.

Microsoft said this provides:

• Zero-configuration deployment: Get up and running quickly with out-of-the-box optimization.
• Seamless Azure integration: Works effortlessly with Azure AI Agent Service and Semantic Kernel.
• Enterprise-grade reliability: Benefit from NVIDIA AI Enterprise support for continuous performance and security.
• Scalable inference: Tap into Azure's NVIDIA accelerated infrastructure for demanding workloads.
• Optimized workflows: Accelerate applications ranging from large language models to advanced analytics.

NVIDIA AgentIQ Added to Azure AI Stack
AgentIQ, an open-source toolkit from NVIDIA, is now part of Azure AI Foundry. It helps manage, monitor, and optimize AI agent workflows, using real-time telemetry to improve performance and lower compute costs. NVIDIA AgentIQ integration within the Azure AI stack is about making it easier and more efficient to build and run complex AI applications that rely on the coordinated efforts of multiple AI agents. It provides the tools and infrastructure needed to optimize these multi-agent systems for better performance and lower costs.

Microsoft said this delivers:

• Profiling and optimization: AgentIQ leverages real-time telemetry to fine-tune AI agent placement, reducing latency and compute overhead.
• Dynamic inference enhancements: The toolkit continuously collects and analyzes metadata, such as predicted output tokens per call, estimated time to next inference, and expected token lengths, to dynamically improve agent performance.
• Integration with Semantic Kernel: AgentIQ directly integrates with Azure AI Foundry Agent Service, empowering agents with enhanced semantic reasoning and task execution capabilities.

Semantic Kernel Integration
As noted, Microsoft's open-source Semantic Kernel framework now directly integrates with both AgentIQ and NIM, making it easier to build complex, multi-agent systems with enhanced reasoning and task execution capabilities. Semantic Kernel helps developers build AI-powered apps by combining natural language AI (like large language models) with traditional programming logic and memory. Semantic Kernel also now supports NVIDIA embedding models through a new connector, enabling developers to use NVIDIA's NIM API for text embeddings in their applications, facilitating the use of NVIDIA's embedding models within the Semantic Kernel SDK. More information is here.

And More
There were plenty more announcements, summarized here:

• Azure ND GB200 V6 Virtual Machines: Microsoft launched the Azure ND GB200 V6 virtual machine series powered by NVIDIA GB200 Grace Blackwell Superchips and NVIDIA Quantum InfiniBand networking for high-performance AI workloads.
• Preview of NVIDIA Llama Nemotron Reason Integration: Microsoft and NVIDIA revealed upcoming support for the Nemotron Llama Reasoning model, optimized for scientific reasoning, coding, and retrieval-augmented generation (RAG). "NVIDIA Llama Nemotron Reason is a powerful AI model family designed for advanced reasoning," Microsoft said. "According to NVIDIA, Nemotron excels at coding, complex math, and scientific reasoning while understanding user intent and seamlessly calling tools like search and translations to accomplish tasks."
• Future Blackwell Ultra GPU VMs: Microsoft plans to introduce virtual machines based on NVIDIA Blackwell Ultra GPUs on Azure later in 2025.
• Future RTX PRO 6000 Server Edition on Azure: Microsoft will also bring the performance of the NVIDIA RTX PRO 6000 Blackwell Server Edition to its Azure platform.
• Optimization of Meta Llama Models: Microsoft is working with NVIDIA to optimize the performance of Meta Llama models on Azure AI Foundry using TensorRT-LLM.
• NVIDIA Omniverse on Azure Marketplace: Preconfigured NVIDIA Omniverse virtual desktop workstations are now available on the Azure marketplace.
• NVIDIA Isaac Sim on Azure Marketplace: Preconfigured NVIDIA Isaac Sim virtual desktop workstations are also available on the Azure marketplace.