Re: Even more confused @Flocke Kroes
"I cannot understand how you can have remote access to another system's memory via RDMA faster than access to local memory."
The main aspect of the session this related to was that when they teamed 4 network adapters and the transfer bottle-necked on the memory bus.
Given that RDMA is designed to be high-throughput and low latency, it is possible that it has fewer overheads than NUMA node memory access. It doesn't mean that a NUMA node on one computer can access memory on another computer faster than on the same device, it just means that the RDMA transfers can move data faster than the NUMA nodes can access it.
"It would be interesting to see whether the systems he has seen in the lab are the ones using PCIe4 as an interconnect."
According to Ben they were using 4x PCIe3 NICs. I didn't catch the specifics of the NICs, but he said they bottle-necked the PCI bus with 3 of them (well, 2 really - the third NIC made no difference to transfer rate). No doubt when they post the session recordings you can listen to what he said and figure it out.