Posts Tagged 'server connectivity'
-
August 04, 2026
Marvell Structera™ X, A and S: A Comprehensive CXL Portfolio Powering AI Memory Innovation
By Khurram Malik, AVP, Data Center Memory and Storage Solutions, Marvell
CXL has become one of the most important technologies shaping AI infrastructure. As hyperscalers race to deploy larger AI models, longer context windows and increasingly memory-intensive inference workloads, memory capacity and bandwidth have emerged as critical constraints on performance, efficiency and scaling. At the same time, CXL adoption is reaching an inflection point, moving from evaluation into real-world deployment across hyperscale environments.
Marvell is leading this transition with Structera™ X memory expansion solutions developed alongside the world’s leading hyperscalers. The story begins with the shipping of Structera X 2404 and 2504 platforms, which have enabled hyperscalers to expand memory resources more efficiently, including extending the useful life of existing DDR4 investments while powering demanding AI workloads.
Structera X is not a series of disconnected product eras—it is a single, continuous architectural evolution. Today’s generation is already delivering real hyperscaler deployments, ecosystem maturity and a compelling TCO advantage. From that foundation, Marvell is extending the architecture toward the next phase of AI infrastructure innovation, adding capabilities enabled by the evolving CXL 3.2 ecosystem, PCIe Gen 6 connectivity and more advanced multi-host memory sharing architectures. These advancements will create larger, more flexible memory pools, enabling more efficient sharing of resources across servers and improving infrastructure utilization at hyperscale. As the architecture advances, Marvell is driving it toward higher bandwidth, deeper data optimization and increasingly disaggregated memory environments built to meet the growing demands of AI workloads.
-
August 04, 2026
Photonic Fabric™ Technology: How Optical Connectivity Enables the Next Generation of AI Infrastructure
By Ravi Mahatme, Senior Director, Product Management, Photonic Fabric Business Unit, Marvell
AI infrastructure is entering a new architectural era. As AI inference scales, performance increasingly depends not simply on adding more compute, but on how efficiently compute, memory and connectivity operate together as one unified AI infrastructure system. Modern AI inference workloads are insatiable consumers of memory. Large language models, reasoning models and agentic AI applications require rapid access to model parameters, embeddings and rapidly growing key-value (KV) caches that preserve conversational context. These working data sets are growing into the hundreds of gigabytes, and increasingly terabytes, making memory capacity, bandwidth and latency just as important as accelerator performance. As AI infrastructure scales, overall system performance increasingly depends on how efficiently accelerators can access, move and utilize memory resources rather than simply adding more compute.
Why AI Needs a New Memory Tier
Today's AI memory hierarchy was never designed for inference at the scale modern workloads demand. High bandwidth memory (HBM) attached directly to GPUs delivers exceptional performance but remains expensive and capacity constrained. System DRAM provides larger memory pools but cannot economically scale alongside every accelerator. NVMe SSDs offer abundant capacity, yet their latency makes them unsuitable for serving active inference workloads.
This challenge is especially visible in large language models, where growing KV caches must remain readily accessible to avoid repeatedly recomputing previous tokens. Keeping these caches entirely in HBM is prohibitively expensive, while moving them to storage introduces latency that reduces token generation performance. The result is that GPUs increasingly spend valuable cycles waiting for data rather than performing inference.
-
May 28, 2026
Open CPX Sets the Stage for More Flexible, Scalable Connectivity
By George Hervey, Associate Vice President, Cloud Switch Marketing, Marvell
Co-packaged connectivity is coming. The Open CPX MSA (Co-packaging Multisource Agreement) is working to simplify adoption.
The consortium, which includes Marvell and other leaders in connectivity, is developing specifications and standards for solutions for integrating near-packaged optical (NPO) and/or co-packaged optical (CPO) technology into switches and servers in scalable, repeatable ways. Members are also working to support interoperability with co-packaged copper (CPC).
The idea is to give data center service providers, equipment manufacturers and others a unified framework for next-generation connectivity to accelerate innovation and meet the surging demand for these technologies. Fewer than one million near- and co-packaged ports shipped in 2025, according to LightCounting; by 2030, shipments are projected to surpass 100 million ports per year.1 Standards that can ensure predictability and flexibility will be critical in enabling this expected growth.
“The initial target of the MSA will be to develop an optimized optical engine with a defined pluggable socket and electrical connector system supporting high speed and high-density connectivity between a switch or processor and co-packaged and near-package interconnects,” the Open CPX MSA website states. “The specifications will define connector mechanicals, thermals, electrical pinout, mechanical form factors, electrical, optical, and management interface specifications to ensure interoperability between multiple vendors of Open CPX.”
-
January 15, 2025
Next Up for Custom AI Accelerators: Co-Packaged Optics
By Michael Kanellos, Head of Influencer Relations, Marvell
Computer architects have touted the performance and efficiency gains that can be achieved by replacing copper interconnects with optical technology in servers and processors for decades1.
With AI, it’s finally happening.
Marvell earlier this month announced that it will integrate co-packaged optics (CPO) technology into custom AI accelerators to improve the bandwidth, performance and efficiency of the chips powering AI training clusters and inference servers and opening the door to higher-performing scale-up servers.
The foundation of the offering is the Marvell 6.4Tbps 3D SiPho Engine announced in December 2023 and first demonstrated at OFC in March 2024. The 3D SiPho Engine effectively combines hundreds of components—drivers, transimpedance amplifiers, modulators, etc.—into a chiplet that itself becomes part of the XPU.
With CPO, XPUs will connect directly into an optical scale-up network, transmitting data further, faster, and with less energy per bit. LightCounting estimates that shipments of CPO-enabled ports in servers and other equipment will rise from a nominal number of shipments per year today to over 18 million by 20292.
Additionally, the bandwidth provided by CPO lets system architects think big. Instead of populating data centers with conventional servers containing four or eight XPUs, clouds can shift to systems sporting hundreds or even thousands of CPO-enhanced XPUs spread over multiple racks based around novel architectures—innovative meshes, torus networks—that can slash cost, latency and power. If supercomputers became clusters of standard servers in the 2000s, AI is shifting the pendulum back and turning servers into supercomputers again.
“It enables a huge diversity of parallelism schemes that were not possible with a smaller scale-up network domain,” wrote Dylan Patel of SemiAnalysis in a December article.
-
December 19, 2024
Custom, Copper and Cross-Country Connectivity: Eight Big Trends for Marvell in 2024
By Michael Kanellos, Head of Influencer Relations, Marvell
What happened in semis and accelerated infrastructure in 2024? Here is the recap:
1. Custom Controls the Future
Until relatively recently, computing performance was achieved by increasing transistor density à la Moore’s Law. In the future, it will be achieved through innovative design, and many of those innovative design ideas will come to market first—and mostly— through custom processors tailored to use cases, software environments and performance goals thanks to a convergence of unusual and unstoppable forces1 that quietly began years ago.
- At Marvell AI Day, we announced that custom processors would represent 25% of the AI accelerator market by 2028 and exceed $42 billion. We also announced we had secured four designs with three of the major hyperscalers.
- In July, we unveiled StructeraTM A and Structera X,2 CXL controllers for near memory compute and increasing memory capacity, respectively. Merchant and custom versions of the chips were released at the same time, a reverse of the traditional pattern where custom chips might follow merchant ones by a lengthy lag.
- At OCP, Meta showed off FBNIC, a custom network interface developed in collaboration with Marvell to reduce the time needed to resolve network issues.3 Make that four out of four hyperscalers. See the image below.
FB NIC on display at OFC- In December, we announced a five-year collaboration that includes supplying custom AI products and other devices for AWS.4 The collaboration also encompasses performing semiconductor design tasks in the cloud, which promises to reduce the time and cost of creating new chips.
- Finally, we ended the year with an alliance for custom high bandwidth memory with Micron, Samsung Electronics, and SK hynix.5 Custom HBM can reduce chip real estate by up to 25% and increase memory capacity by up to 33%.6 Custom HBM runs against industry orthodoxy,7 writes analyst Joseph Byrne, but it could pay dividends.
Recent Posts
- The Road Back to Marvell: Finding Impact and Scale in AI Networking
- Marvell® Photonic Fabric™ Wins AI Infrastructure Award at FMS 2026
- Accelerating AI Infrastructure with Marvell Structera A and SK hynix CXL Memory: Enabling Efficient Near-Memory Processing
- Marvell Structera™ X, A and S: A Comprehensive CXL Portfolio Powering AI Memory Innovation
- Photonic Fabric™ Technology: How Optical Connectivity Enables the Next Generation of AI Infrastructure
