Most of the companies around the globe using Netezza data warehouse appliance to perform their near real-time, complex analytics and data warehouse needs. Netezza FPGA is important part of NPS system.
NPS performance is made of a result of framework of Netezza FPGA Accelerated streaming technology (FAST) engine. This engine is just like a turbo-charger to an already powerful engine. This FAST engine what makes the Netezza a powerful server.
Read:
Netezza has introduced a FAST Engine, Compress, which uses technology to deliver yet another 100-200% performance increase in streaming analytic performance. Netezza’s Compress Engine is designed primarily for performance improvement. Rather than the CPU-intensive compression efforts employed by other vendors to reduce disk usage that also result in reduced performance, the Compress Engine accelerates performance.
The Netezza FPGA – FAST Engines Framework
The Netezza FPGA is a the heart of Netezza’s patented streaming architecture. Netezza uses the FPGA to handle important functionalities and filter out unwanted data during early stage of the disk read.
The process of early unnecessary data elimination improves the performance as it frees up the downstream systems like I/O, CPU, memory and network. This enables NPS to process only the required data and client request as soon as possible. Those primary functions that are built into the FPGA in the NPS system are referred to as “engines” and compose the FAST Engines framework.
TheNetezza FPGA-Accelerated Streaming Technology (FAST) Engines Framework’s three main concepts:
- FAST Engines are basic analytic functions electronically programmed into the FPGA to accelerate query performance
- FAST Engines are dynamically reconfigurable
- FAST Engines are customized at run-time for each snippet executed in the SPU.
Dynamic re-configurability allows each of the engines embedded in the FPGA to be modified, disabled or extended by the NPS system at any point of time. Run-time customization enables the FPGA to incorporate parameters passed to each engine to optimize the behavior of the FPGA for a particular query snippet.
By enabling this snippet-by-snippet customization, one could think of the NPS platform as providing an ‘optimized hardware configuration for each snippet’.
The five current FAST Engines in the framework include Control, Parse, Visibility, Project and Restrict. These engines work in a combined serial and parallel manner.
Overall FAST framework is depicted in below diagram.
Control Engine
The Control Engine controls the hard disk and manages direct memory access (DMA) data flows from the disk to the SPU memory. It manages reads and writes to the disk drive and data flows to memory.
Storage management performance enhancement functions, such as Zone Maps are enforced through the Control Engine.
Parse Engine
The Parse Engine performs two key roles in the FAST Engines framework.
- It provides Error-Correcting Code (ECC) checking and correction as data streams flows out of disk drive.
- It parses the incoming data stream from the disk drive and passes the appropriate data to each of the remaining three downstream engines such as Visibility, Project and Restrict.
Visibility Engine
The Visibility Engine provides the ACID (Atomicity, Consistency, Isolation and Durability) isolation enforcement for the NPS appliance. As data are streamed from the disk drive, the Visibility Engine will removes rows of data that should not be visible to the particular query snippet being executed – either because the records had been marked deleted by an earlier query or because they had been added to the database after the start of the current query.
FPGA engine immediately eliminate data that should not be visible for processing.
Project Engine
The Project Engine enforces filtering of the column data read from the disk based on the “SELECT” clause of the SQL statement. Only columns included by the SELECT clause are allowed to continue for further processing and any columns not selected will be filtered and eliminated prior to any records being sent to the CPU and memory.
This step increases the processing performance as unwanted columns are not available for processing.
Restrict Engine
The Restrict Engine enforces filtering of records that are read from the disk based on the “WHERE” clauses of the SQL statement. Only records that satisfy the WHERE clauses will be allowed to continue for further processing and any records not selected are filtered and eliminated prior to being sent to the on-board CPU and memory
The Restrict Engine typically results in an exceptionally high degree of data record filtering, greatly reducing the follow-on snippet processing work required of the CPU and memory on each SPU.
Compress Engine
The Compress Engine will boost streaming query performance by 2-3x (100-200%) via software enhancement to the NPS appliance. Compress Engine will be able to compress the data to use disk much more effectively and increase query performance as data flows from the disk drives.
Netezza’s Compress Engine is designed primarily for performance improvement, and not just to reduce data sizes.
The Compress Engine, like all other FAST Engines, is dynamically configurable and optimized at run-time. As data is written to disk, (e.g., during data load, insert or update operations) it is compressed into a compiled format, column-by-column with the original data replaced by the Compress Engine instruction set for de-compilation. As this data is read from the disk, the Compress Engine reads its instruction set and reassembles the original data as it streams from the disk, effectively raising the streaming data rate by as much as 200% – this step increases effective scanning rate per SPU node from over 60 MB/s to approximately 200 MB/s.
Read :