Key Terms:
- AMPP (Asymmetric Massively Parallel Processing)
- Intelligent Query StreamingTM Technology
- FAST EngineTM - FPGA Accelerated Streaming Technology
- Uses AMPP (Asymmetric Massively Parallel Processing) – combination of MPP and SMP
- Netezza’s AMP architecture is a two tiered system-
o The first tier is a high performance Linux SMP host.
o The second tier consists of thousands of SPU (Snippet Processing Unit) operating in parallel.
- Each SPU consists of –
o Processor
o Dedicated memory
o A disk drive
o A FPGA
- FPGA is the heart of Netezza’s patented streaming architecture. By using FPGA, less computational stress is put on CPU and memory of each SPU.
- Sorts, joins, aggregates are executed primarily on the SPUs, but can also be done on the host, depending on the processing cost of that operation.
- Additional arrays of snippet processing units can be added to the NPS system without impacting performance.
Intelligent Query StreamingTM Technology:
o “Bringing the query to the data” approach is used.
o Greatly reduces data traffic among SPU nodes and between SPU nodes and SMP host.
o Intelligent Query Processing is performed on each SPU by a FPGA
FAST EngineTM - FPGA Accelerated Streaming Technology- provides performance acceleration
- For each snippet there are two elements that are distributed to the SPUs:
§ The compiled snippet file
§ A set of FPGA parameters
- FAST engines are customized at run-time (enables the FPGA to incorporate parameters passed to each engine to optimize the behavior of the FPGA for a particular query snippet) for each snippet executed in the SPU.
- Five current FAST engines in the framework include
o Control engine (do ZoneMap acceleration)
o Parse engine (do Error-Correcting Code (ECC) checking and correction)
o Visibility engine (Provide ACID (Atomicity, Consistency, Isolation and Durability) )
o Project engine (filters based on the “SELECT”)
o Restrict engine (filters based on the “WHERE”)
All engines work in combined serial and parallel manner.
SELECT count ( * ) , sex , age
FROM emp
WHERE state = ‘VA’ and age > 18
GROUP BY sex , age
ORDER BY age ;
- Integrates all major ETL tools from vendors including- Ab Initio, Ascential, Informatica, Sunopsis
- With a data transfer speed of up to 500 GB/hour.
- Customers have the flexibility to convert the portion of ETL process to ELT process.
- Fast Backups: The 500 GB/hour (2 TB/hour compressed) data transfer
- Streaming Query processing: Rather than “moving data to the query”, the NPS system takes a
different approach: “moving the query to the data”
different approach: “moving the query to the data”
- Each processing and storage modules includes a data mirror.
- If the system detects an error , or a processing/storage module stops responding, it deactivates the failed module
o The system regenerates data on to a hot spare module from mirrored data, a process known as failover.
o If a hot spare module is not available or fails, the system continues processing at a reduced performance by directly using the mirrored data. Note that, depending up on the system model, there are between four and sixteen hot spare modules available.
- Performance: 10x to 100x faster query performance than traditional data warehouse
- Effortless scalability
- Flexibility: designed to grow with the easy addition of modular processing units for scalability &
reconfiguration.
reconfiguration.
- Reliability: Fully ACID and RAID compliant to secure maximum structural reliability.
- No indexing required.
- No tablespace & redo log sizing and configuration
- No journaling/logical log sizing and configuration
- No page/block sizing and configuration for tables
- No extent sizing and configuration for tables
- No temp space allocation and monitoring
- No maintenance of OS recommended path levels
- No software to install
