AI's easy on-ramp has become a costly exit problem for enterprises, says Red Hat
As enterprises push AI beyond the pilot stage, the cost and complexity of running inference at scale are forcing a fundamental rethink of how infrastructure is designed, governed and sourced, putting horizontal cloud — one shared foundation for running workloads across the enterprise — at the center of AI strategy. The open hybrid cloud model […] The post AI’s easy on-ramp has become a costly exit problem for enterprises, says Red Hat appeared first on SiliconANGLE. Learn how a horizontal cloud architecture is eliminating silos and reducing delivery times for global telecommunications and enterprise leaders.
Key Takeaways
- The open hybrid cloud model is emerging as a practical answer to a market that has become dangerously dependent on a small number of frontier model providers, including Anthropic PBC and OpenAI Group PBC.
The journey from frontier model convenience to self-managed, cost-efficient inference now sits at the center of enterprise AI strategy, according to Stephen Watt (pictured), vice president and distinguished engineer, Office of the CTO, at Red Hat Inc.
- ) Horizontal cloud as the AI inference escape hatch The pressure to move off expensive frontier models is fueling demand for a new class of shared, governed inference infrastructure.
4 extends model-as-a-service and distributed inferencing capabilities, the industry is converging on the idea that a horizontal cloud platform - one shared layer spanning storage, compute and management - can unlock both efficiency and control.
- "Every department's doing their own experimentation, their own pilots, but everybody's going about it a different way," Watt said.
"Everybody will emerge from the pilot phase, and there'll be some shared observations.
- Rather than relying on a single large model monolith, the router directs inference requests to purpose-trained open-weight models - one tuned for physics, another for history, for example - based on the nature of each query, improving accuracy while lowering cost, Watt explained.
"You can basically ensure that the inference requests are always going to the highest-performing models," he said.
- ) Photo: SiliconANGLE A message from John Furrier, co-founder of SiliconANGLE: Support our mission to keep content open and free by engaging with theCUBE community.
Stats & Key Facts
- #The operator cut delivery time by 40% and reduced operational costs by 30-45%, collapsing processes that once took weeks or months into days.
Learn how a horizontal cloud architecture is eliminating silos and reducing delivery times for global telecommunications and enterprise leaders. The open hybrid cloud model is emerging as a practical answer to a market that has become dangerously dependent on a small number of frontier model providers, including Anthropic PBC and OpenAI Group PBC. The journey from frontier model convenience to self-managed, cost-efficient inference now sits at the center of enterprise AI strategy, according to Stephen Watt (pictured), vice president and distinguished engineer, Office of the CTO, at Red Hat Inc.
"You'd be crazy today not to start on a frontier model provider, like OpenAI or Anthropic, but then after a while, when you hit a certain scale - like in token economics - you'd be crazy to stay on that," Watt said. "That's the dilemma: When you want to leave, what are your options, and how do you navigate [that]?
They discussed how inference routing, agentic AI governance and horizontal cloud architecture are reshaping enterprise AI deployments. ) Horizontal cloud as the AI inference escape hatch The pressure to move off expensive frontier models is fueling demand for a new class of shared, governed inference infrastructure. 4 extends model-as-a-service and distributed inferencing capabilities, the industry is converging on the idea that a horizontal cloud platform - one shared layer spanning storage, compute and management - can unlock both efficiency and control.
For more details please read the original article at SiliconANGLE AI.
Continue Learning
Comments
Sign in to join the conversation