Arm and Red Hat Unveil Agentic AI Infrastructure for Data Centers

What is the Shift Toward Agentic AI Infrastructure?

The landscape of artificial intelligence is undergoing a significant transformation. As the industry moves beyond the initial phase of model training, the focus is increasingly shifting toward inference. This transition is being driven largely by the emergence of agentic AI systems. Unlike traditional models that operate in discrete batches, agentic systems are designed to function continuously. They coordinate complex tasks, interact dynamically with data pipelines, and make autonomous decisions in real time. This continuous operation places immense pressure on underlying data center infrastructure, demanding a new level of efficiency, compute density, and orchestration capability.

Data centers are no longer just storage and processing hubs; they are becoming the nervous system for always-on distributed AI workloads. The challenge for modern enterprises is not merely about achieving raw performance metrics. Instead, the critical issue is how to scale these systems effectively across hybrid environments. Organizations need infrastructure that can handle the seamless movement of workloads between cloud instances and on-premises servers while maintaining consistent performance and security standards. This shift requires a holistic approach that integrates hardware, software, and networking capabilities into a unified ecosystem.

The rise of agentic AI means that data centers must support systems that are perpetually active. These systems require low-latency access to data and the ability to process vast amounts of information without interruption. Traditional infrastructure, often optimized for batch processing or periodic training cycles, is struggling to meet these demands. The industry is now looking for solutions that can provide the necessary bandwidth, processing power, and energy efficiency to sustain these continuous operations. This has led to a renewed interest in alternative architectures that can offer better performance-per-watt ratios and higher core densities.

How Does the Arm and Red Hat Collaboration Address These Challenges?

To tackle the complexities of scaling agentic AI, Arm has entered into a strategic collaboration with Red Hat. This partnership aims to deliver a fully integrated, production-ready AI stack. By combining the capabilities of the Arm AGI CPU with Red Hat’s enterprise-grade open-source platforms, the collaboration provides a consistent foundation for running AI agents, cloud-native workloads, and enterprise applications. This approach ensures that organizations can deploy and manage their AI infrastructure with greater confidence and efficiency, regardless of whether their systems are located in the cloud or on-premises.

At the software layer, Red Hat plays a crucial role in this ecosystem. Red Hat Enterprise Linux on Arm delivers a stable and secure operating system that is fully compatible with existing enterprise tools, frameworks, and workflows. For organizations adopting hybrid strategies, this compatibility is essential. It ensures a consistent runtime environment, enabling workloads to move seamlessly between Arm-based cloud instances and on-premises infrastructure. This consistency reduces the complexity of managing hybrid environments and allows enterprises to leverage the benefits of Arm architecture without sacrificing operational stability.

Red Hat OpenShift further extends this capability by enabling enterprises to deploy and manage Kubernetes clusters at scale on Arm architecture. AI agents, microservices, and data pipelines can be orchestrated within a unified cloud-native platform, simplifying operations across diverse environments. With the introduction of Red Hat OpenShift Virtualization support on the AGI CPU, organizations can run virtual machines and containers side-by-side. This feature allows for gradual modernization and migration to Arm without disrupting existing workloads, providing a flexible path for enterprises to adopt new technologies at their own pace.

The synergy between Arm and Red Hat is rooted in a long-standing collaboration in the data center space. Steven Huels, Vice President of AI Engineering at Red Hat, emphasized that Red Hat delivers enterprise-grade, open-source platforms that enable innovation across hybrid cloud environments. He noted that the expansion of the Arm ecosystem has helped deliver a broader set of customer solutions. The Arm AGI CPU is poised to continue this trend, powering a next generation of AI-ready infrastructure on Red Hat’s trusted, open foundation. This partnership underscores the importance of having a robust software layer that can fully exploit the capabilities of modern hardware.

Why is the Arm AGI CPU Critical for Agentic AI?

Delivering the level of consistency and scalability required for agentic AI necessitates a new foundation at the silicon level. At the core of this solution is the Arm AGI CPU, Arm’s first system-on-chip designed specifically for data center infrastructure. Introduced on March 24, 2026, the AGI CPU is purpose-built to handle a wide range of AI-driven workloads. It is designed to manage everything from scalable inference and orchestration to databases, video processing, and enterprise services. This versatility makes it an ideal candidate for the diverse demands of agentic AI systems.

The technical specifications of the AGI CPU are designed to meet the rigorous requirements of next-generation workloads. The chip features 136 Arm Neoverse V3 cores, providing substantial processing power for parallel tasks. It includes 96 lanes of PCIe Gen6, ensuring high-speed data transfer capabilities. Additionally, it supports 12 channels of DDR5 memory running at up to 8800 MT/s, delivering the bandwidth necessary for data-intensive applications. These specifications enable the AGI CPU to handle the complex computations and data movements required by agentic AI systems, which often involve real-time decision-making and continuous data processing.

Arm-based cloud platforms, such as AWS Graviton, Microsoft Azure Cobalt, and Google Axion, have already demonstrated strong performance-per-watt and total cost of ownership advantages. The AGI CPU extends these benefits to on-premises environments. This allows organizations to bring cloud-optimized Arm workloads into their own data centers while maintaining consistency across their hybrid infrastructure. By leveraging the same architecture used in leading cloud providers, enterprises can achieve similar efficiency and performance gains in their private data centers, reducing the need for costly hardware upgrades and migrations.

The AGI CPU is particularly well-suited for the demands of agentic AI. Unlike training workloads that may occur periodically, agentic systems require continuous inference and real-time orchestration. The AGI CPU is designed to meet these demands with high efficiency. At a thermal design power of 300 watts, it delivers significantly higher efficiency than traditional 500-watt-class x86 processors. This power efficiency is crucial for data centers looking to reduce energy consumption and operational costs while maintaining high performance levels.

What Are the Implications for Data Center Density and Efficiency?

The efficiency of the AGI CPU has profound implications for data center density and scalability. In a standard 36-kilowatt air-cooled Open Compute Project rack, the AGI CPU enables approximately 8,160 cores per rack. This is nearly double the compute density of traditional x86 systems, which typically support around 4,352 cores within the same power envelope. This increased density allows organizations to pack more processing power into their existing infrastructure, reducing the need for additional physical space and cooling resources.

In liquid-cooled environments, such as OCP Open Rack v3, the scalability potential of the AGI CPU increases even further. Deployments can support up to 336 AGI CPUs, resulting in as many as 45,696 cores per rack. This represents a fivefold increase in compute density compared to traditional systems. Such density improvements are critical for organizations looking to scale their AI capabilities without expanding their physical footprint. It also aligns with sustainability goals by maximizing the utility of existing resources and reducing energy consumption per core.

While GPU-accelerated platforms remain the leaders in AI training, the AGI CPU addresses critical bottlenecks in pre-processing, orchestration, and large-scale inference. These are key requirements for agentic AI systems, which rely on efficient data movement and real-time decision-making. By handling these tasks effectively, the AGI CPU complements GPU workloads, creating a more balanced and efficient AI infrastructure. This division of labor allows data centers to optimize their resources, using GPUs for heavy training tasks and AGI CPUs for continuous inference and orchestration.

The validated ecosystem surrounding the AGI CPU further enhances its appeal. The platform is supported by a broad range of OEM and ODM partners, including Supermicro, Lenovo, and ASRock Rack. This diversity provides flexibility in how organizations deploy and scale their infrastructure. Enterprises can choose from a variety of server configurations and solutions that best fit their specific needs, ensuring that they can implement the Arm AGI CPU in a way that integrates seamlessly with their existing operations. Solutions based on this integrated stack are expected to be available in calendar Q4 2026, giving organizations a clear timeline for deployment and planning.

The collaboration between Arm and Red Hat represents a significant step forward in the evolution of data center infrastructure. By combining purpose-built silicon with enterprise-grade software, they are addressing the unique challenges posed by agentic AI. This integrated approach offers a path toward more efficient, scalable, and flexible AI deployments. As the industry continues to embrace agentic systems, solutions like the Arm AGI CPU and Red Hat platforms will play a crucial role in enabling the next generation of AI-driven innovation.