VINEYARD github on Spark acceleration in the Pynq device

Spark is one of the most widely used frameworks for data analytics that offers fast development of applications like machine learning and graph computations in distributed systems. VINEYARD has recently released the SPynq framework.

SPynq is a framework for the efficient mapping and acceleration of Spark applications on heterogeneous MPSoC FPGAs, such as Zynq. Spark has been mapped to the Pynq platform and the proposed framework allows the seamlessly utilization of the programmable logic for the hardware acceleration of computational intensive Spark kernels.

VINEYARD has also developed the required libraries in Spark that hides the accelerator's details to minimize the design effort to utilize the accelerators. On the reconfigurable logic part, the hardware accelerators for the specific application are hosted. The hardware accelerators are invoked by the python API of the Spark application. Therefore, the only modification that is required is the extension of the python library with the new function calls for the communication with the hardware accelerator.

SPynq has been evaluated in a typical machine learning application based on logistic regression. The logistic regression kernel has been developed as a Pynq overlay and incorporated to the Spark. The performance evaluation shows that the heterogeneous FPGA-based MPSoC can achieve up to 53x speedup compared with a x86_64 laptop system.

The proposed system can also offer reduced energy consumption and can also reduce significantly the development time of embedded and cyber-physical systems on Spark applications.

The SPynq framework is available to download and try in the following Github HERE

 

Dr. Christoforos Kachris

(ICCS - Institute of Communications and Computer Systems)