For most of the last decade, scaling AI meant scaling compute. If you built faster accelerators and wired enough of them together, the models would follow.

That is no longer the whole picture. Models now run to hundreds of billions of parameters with extensive context windows, and whether an expensive accelerator is working or just waiting comes down to memory: how much you have, how fast you can reach it, and how much time you lose moving data around.

Once memory leaves the server and rides a switched fabric, is it still memory, or has it become a network? This was the central question of MemNetAI, the first workshop on Memory-Semantic Networking for AI-Scale Systems, launched by Marvell with researchers from IIT Hyderabad and IIIT Delhi. Held at ACM SIGCOMM 2026 in Denver and guided by a program committee spanning academia and industry, it brought speakers from Cornell, alongside industry experts and researchers presenting their work, to debate these questions on the bleeding edge of AI infrastructure.

Why Memory Is Becoming the Bottleneck

In most high-performance systems, memory is the slowest thing, and making it fast requires assembling a lot of devices together and connecting them. Previously, memory sat right next to the processor and caches soaked up the frequent accesses. AI changed that. The biggest models today are far too large to sit beside any single accelerator, so their memory ends up spread across the system: pooled across a rack, expanded over CXL, reached over optical links.

Once memory is no longer local—once it must cross a switched fabric—it starts running into the same problem as networks: congestion. Keeping memory fed is as much a networking problem as a memory one, and that overlap is what the workshop was about.

A Photonic Vision for Scale-up Networks

Rachee Singh delivered a keynote presentation.

In a talk entitled “Intelligent Scale-up Domains for Machine Learning,” Cornell professional Rachee Singh, who leads a SysPhotonics group at her institution, set the scene with impactful figures. Accelerator compute has been roughly tripling every two years, while the bandwidth between accelerators has grown far more slowly, around 1.4x. When two things grow at differing rates, the gap between them increases. In practice, that gap is an accelerator sitting idle, waiting on data that has not arrived.

Singh’s pitch is to rebuild the scale-up domain, the tightly connected group of accelerators inside a rack, so it is bigger, faster, and programmable, using a wafer-scale silicon photonic interconnect instead of copper. Through two demos, she brought the vision to life. In the first, her team physically pulled a GPU out of a running training job and let the optical network reconfigure itself and move the work to a healthy GPU, without shipping any model weights across the fabric. In the second, they ran an AllReduce, one of the workhorses of distributed training, entirely in light, skipping the usual round trip out to electrical signals and back. Optics is a real way around the bandwidth wall.

Treating Memory as a First-class Resource

Muhammad Shahbaz (University of Michigan) chaired the research track.

Memory should be a first-class resource: scheduled and measured rather than assumed. Talks from the paper sessions covered high-bandwidth CXL memory expanders that replace slower staging in LLM inference, fine-grained GPU memory tracing to make hardware-software co-design possible, memory disaggregation for more flexible AI systems, and memory as a first-class citizen in AI-factory simulation. Different problems, same thread: once memory is attached to a fabric, it needs the same care in modeling, scheduling, and monitoring that networks have always been given.

Is Memory the New Network?

The panel, from the left: Abed Md Kamaluddin (Marvell, moderator), Rachit Agarwal (Cornell) and Shrijeet Mukherjee, with Rip Sohan joining remotely.

The closing panel debated this essential question. Industry experts Shrijeet Mukherjee and Rip Sohan alongside Rachit Agarwal from Cornell presented different viewpoints on stage, however they each agreed on the common point that the difficulty is now the boundary between memory and the network. At line rate, a small stumble in the memory subsystem turns into a big performance hit, and that is pushing ideas from networking, like congestion control, down into memory itself.

The sharpest back-and-forth was over whether “scale-up” and “scale-out” still mean different things as fabrics get bigger and features cross over from one to the other. One panelist drew the line at latency, another at what you are optimizing for, a third at whether I/O management happens in hardware.

Looking to 2035, ideas included co-designing compute, memory, and interconnect together; building systems that expect failure and degrade gracefully instead of falling over; treating energy as the real limit; and taking observability—a current afterthought— seriously. Optics and new memory technologies will also have a grand impact.

A Problem No Single Layer Owns

The MemNetAI workshop comes to a close for the day.

None of these problems belong to one layer. Turning memory into a shared, fabric-attached tier touches the accelerator, the memory devices, the interconnect, the network, and the software scheduling across all of them, which is why so much of the panel kept coming back to co-design. Two directions ran through most of the talks: CXL, for expanding, pooling, and disaggregating memory across a coherent fabric, and optical interconnect, for the bandwidth and reach that copper is running out of.

Marvell offers compelling solutions across both categories. For CXL, the Structera® product line brings Compute Express Link to memory bandwidth and capacity challenges, with the Structera A series of near-memory accelerators, the Structera X series of memory-expansion controllers, and the Structera S series of CXL switches that enable access to disaggregated memory resources outside the server. For optical interconnect, Marvell PAM4, coherent, and coherent-lite DSPs, along with silicon photonics, power optical modules that deliver high-bandwidth connectivity inside data centers, across data center campuses, and between geographically dispersed data centers as bandwidth needs scale toward 1.6 Tbps and beyond.

Much is still unsettled. Neither Marvell nor the ecosystem has converged on the abstractions, benchmarks, and failure models this new memory tier needs—that groundwork gets done in the open, with everyone in the same room. Who manages memory across the whole stack, whether scale-up survives as its own category, and what belongs in this new layer between the backplane and the network are questions worth coming back to in future MemNetAI workshops.

# # # 

This blog contains forward-looking statements within the meaning of the federal securities laws that involve risks and uncertainties. Forward-looking statements include, without limitation, any statement that may predict, forecast, indicate or imply future events or achievements. Actual events or results may differ materially from those contemplated in this blog. Forward-looking statements are only predictions and are subject to risks, uncertainties and assumptions that are difficult to predict, including those described in the “Risk Factors” section of our Annual Reports on Form 10-K, Quarterly Reports on Form 10-Q and other documents filed by us from time to time with the SEC. Forward-looking statements speak only as of the date they are made. Readers are cautioned not to put undue reliance on forward-looking statements, and no person assumes any obligation to update or revise any such forward-looking statements, whether as a result of new information, future events or otherwise.