Hugging Face Models on Foundry Managed Compute
Weights are pre-staged in Azure, runtimes are built and scanned by Microsoft, and every model in the Collection ships with the same enterprise security, governance, observability, and billing that applies to every other model on Foundry. The Platform: Microsoft Foundry and Managed Compute Microsoft Foundry is a platform for building and operating agentic AI applications. Foundry starts with the widest model selection on any cloud - models from Microsoft, OpenAI, Anthropic, Meta, Mistral, DeepSeek, Hugging Face, and others, spanning frontier, open-source, and custom weights - all accessible through a single endpoint and a single set of SDKs in Python, C#, JavaScript, and Java.
Key Takeaways
- On top of those models sits the Foundry Agent Service : multi-agent orchestration with built-in memory, knowledge grounding through Foundry IQ, and a catalog of connectable tools via agentic protocols, so agents can work with enterprise data.
Once agents are running, Foundry provides end-to-end tracing, real-time monitoring, continuous evaluations, and a prompt optimizer that improves agent behavior based on eval results - observability and quality loops that are part of the platform.
- You deploy a model instance described by the things that matter to your workload - parameter count, context length, and whether you want to optimize for latency or throughput - and Foundry handles the GPU topology underneath, whether the instance lands on one accelerator or several, so you think and plan in model terms.
Microsoft takes care of the machine: container updates, runtime upgrades, and security patches happen automatically on the supported runtimes - vLLM, SGLang, TensorRT-LLM, NIM, TEI, llama.
- cpp - without redeploying your model, while model configuration, deployment behavior, and routing stay with you.
- Managed Compute offers: Global deployments - broadest capacity and best pricing Data Zone deployments - residency and sovereignty Same code, same workflow.
Quota is aligned to accelerator families, so a plan built on the H100 family today carries forward as new hardware generations come online.
- It's the GitHub of open models, where the community publishes weights, writes model cards, compares evaluations, and pulls models for experimentation.
Stats & Key Facts
- #Why Hugging Face Hugging Face is the public square of open AI: 15 million builders, 400,000 organizations, and over 3 million open models published , with new frontier capabilities - agentic coding, video segmentation, speech, embeddings - landing weekly.
On top of those models sits the Foundry Agent Service : multi-agent orchestration with built-in memory, knowledge grounding through Foundry IQ, and a catalog of connectable tools via agentic protocols, so agents can work with enterprise data. Once agents are running, Foundry provides end-to-end tracing, real-time monitoring, continuous evaluations, and a prompt optimizer that improves agent behavior based on eval results - observability and quality loops that are part of the platform. You deploy a model instance described by the things that matter to your workload - parameter count, context length, and whether you want to optimize for latency or throughput - and Foundry handles the GPU topology underneath, whether the instance lands on one accelerator or several, so you think and plan in model terms.
Microsoft takes care of the machine: container updates, runtime upgrades, and security patches happen automatically on the supported runtimes - vLLM, SGLang, TensorRT-LLM, NIM, TEI, llama. cpp - without redeploying your model, while model configuration, deployment behavior, and routing stay with you. That consistency carries through the developer surface - pay-per-token, provisioned throughput, and Managed Compute share: A single endpoint The same SDKs The same authentication The same observability A single bill Open-source models integrate with Foundry Agents the same way frontier models do, so you can mix model types in a single agent without a separate integration path.
Managed Compute offers: Global deployments - broadest capacity and best pricing Data Zone deployments - residency and sovereignty Same code, same workflow. Quota is aligned to accelerator families, so a plan built on the H100 family today carries forward as new hardware generations come online. Why Hugging Face Hugging Face is the public square of open AI: 15 million builders, 400,000 organizations, and over 3 million open models published , with new frontier capabilities - agentic coding, video segmentation, speech, embeddings - landing weekly.
For more details please read the original article at Hugging Face.
Continue Learning
Comments
Sign in to join the conversation