5 Times More Queries per Second: What CXL Compute Accelerators Can Do for AI

Near-memory compute technologies have always been compelling. They can offload tasks from CPUs to boost utilization and revenue opportunities for cloud providers. They can reduce data movement, one of the primary contributors to power consumption,¹  while also increasing memory bandwidth for better performance.  

They have also only been deployed sporadically; thermal problems, a lack of standards, cost and other issues have prevented many of these ideas giving developers that goldilocks combination of wanted features that will jumpstart commercial adoption.²

This picture is now changing with CXL compute accelerators, which leverage open standards, familiar technologies and a broad ecosystem. And, in a demonstration at OCP 2025, Samsung Electronics, software-defined composable solution provider Liqid, and Marvell showed how CXL accelerators can deliver outsized gains in performance.

The Liqid EX5410C is a demonstration of a CXL memory pooling and sharing appliance capable of scaling up to 20TB of additional memory. Five of the 4RU appliances can then be integrated into a pod for a whopping 100TB of memory and 5.1Tbps of additional memory bandwidth. The CXL fabric is managed by Liqid’s Matrix software that enables real-time and precise memory deployment based on workload requirements: 

The system is based around a Marvell® Structera™ A board. An individual Marvell Structera A memory accelerator contains 16 Arm Neoverse cores and up to 4TB of additional DDR5 memory for adding up to 200Gbps of memory bandwidth. For the modules in the system pictured below, however, only 128GB of DDR5 are included on each board: 

The board sits in a module designed in engineering collaboration between Samsung and Marvell that can be swapped in and out. Like other CXL devices, the module plugs into the appliance and the broader network through PCIe. Up to ten modules can fit into a single EX5410C chassis.

In benchmark tests conducted as part of Samsung’s research project, the system containing Structera A was able to process 19.798 vector search queries per second versus 3.579 for a system containing a CXL memory pooling device. Put another way, that’s 5.3X queries. Vector searches focus on embedded data in documents and the context surrounding information. While that’s beneficial for natural language queries, it can consume significant bandwidth and computational resources.³

A CXL pooling device was used in the control system to make the memory in both systems equal and isolate the impact of the two attributes of memory accelerators—more computing core and memory bandwidth. Structera A can manage as much as 4TB of memory. The system in the test, however, only contains 128GB, showing that substantial gains can be made with far smaller amounts of memory.

Vasanthi Jagathi of Samsung explains more in this video.