Online GPUAnalysis using Adaptive DMA Controlled by Softcore for 2D Detectors

R Ponsard , N. Janvier (1) , D Houzet , V. Fristot (2) , W. Mansour (1)

1 ESRF - European Synchrotron Radiation Facility
2 CIME - Centre Interuniversitaire de Micro-Electronique

New generation X-ray detectors enables cutting-edge experiments that can produce very high throughput data streams that are challenging to manage and store. This paper presents an evaluation of a configurable data placement mechanism from an FPGA device collecting detector raw data to a burst-cache memory and concurrently to a GPU accelerator, bypassing hardware and software extraneous copies and bottlenecks via PCI-Express. It includes a DMA controller dynamically configured in real-time by a Microblaze soft-processor. A low-latency synchronization mechanism using GPUDirect technology is presented as well. Multi-GB, DMA-able memory buffer allocation, leveraging Linux contiguous memory allocator is investigated. As illustrative workloads, real-time raw-data correction as foreseen in Serial Synchrotron X-ray experiments were processed. Obtained results showed that if one could reach a data throughput of 12.7GB/s to CPU memory when using PCIe gen3 x16, a 12-cores OpenMP CPU application processes the raw data only up to 2.7GB/s and is outperformed by a GPU accelerator (NVIDIA RTX 6000).

Mots clés: FPGA; PCIe; DMA; CMA; GPUDirect; Microblaze

Domaines: Performance et fiabilité [cs.PF]; Traitement du signal et de l'image [eess.SP]

R Ponsard, N. Janvier, D Houzet, V. Fristot, W. Mansour. Online GPUAnalysis using Adaptive DMA Controlled by Softcore for 2D Detectors. Euromicro DSD 2020, Aug 2020, Portorož (Virtual), Slovenia. ⟨10.1109/DSD51259.2020.00075⟩. ⟨hal-02932093⟩