Netezza

Key Terms:

-   AMPP (Asymmetric Massively Parallel Processing)

-   Intelligent Query Streaming^TM Technology

-   FAST Engine^TM - FPGA Accelerated Streaming Technology

  

Netezza Architecture:

-   Uses AMPP (Asymmetric Massively Parallel Processing) – combination of MPP and SMP

-   Netezza’s AMP architecture is a two tiered system-

o  The first tier is a high performance Linux SMP host.

o  The second tier consists of thousands of SPU (Snippet Processing Unit) operating in parallel.

-   Each SPU consists of –

o  Processor

o  Dedicated memory

o  A disk drive

o  A FPGA 

-   FPGA is the heart of Netezza’s patented streaming architecture. By using FPGA, less computational stress is put on CPU and memory of each SPU.

-   Sorts, joins, aggregates are executed primarily on the SPUs, but can also be done on the host, depending on the processing cost of that operation.

-   Additional arrays of snippet processing units can be added to the NPS system without impacting performance.

Intelligent Query Streaming^TM Technology:

o  “Bringing the query to the data” approach is used.

o  Greatly reduces data traffic among SPU nodes and between SPU nodes and SMP host.

o  Intelligent Query Processing is performed on each SPU by a FPGA

FAST Engine^TM - FPGA Accelerated Streaming Technology- provides performance acceleration

     -   For each snippet there are two elements that are distributed to the SPUs:

§  The compiled snippet file

§  A set of FPGA parameters

-  FAST engines are customized at run-time (enables the FPGA to incorporate parameters passed to each engine to optimize the behavior of the FPGA for a particular query snippet) for each snippet executed in the SPU.

-   Five current FAST engines in the framework include

o  Control engine (do ZoneMap acceleration)

o  Parse engine (do Error-Correcting Code (ECC) checking and correction)

o  Visibility engine (Provide ACID (Atomicity, Consistency, Isolation and Durability) )

o  Project engine (filters based on the “SELECT”)

o  Restrict engine (filters based on the “WHERE”)

    All engines work in combined serial and parallel manner.

Example Query

SELECT count ( * ) , sex , age

FROM emp

WHERE state = ‘VA’ and age > 18

GROUP BY sex , age

ORDER BY age ;

Latency barrier with NPS

-    Integrates all major ETL tools from vendors including- Ab Initio, Ascential, Informatica, Sunopsis

-    With a data transfer speed of up to 500 GB/hour.

-    Customers have the flexibility to convert the portion of ETL process to ELT process. 

-    Fast Backups: The 500 GB/hour (2 TB/hour compressed) data transfer

-    Streaming Query processing: Rather than “moving data to the query”, the NPS system takes a
     different approach: “moving the query to the data”

Redundancy

-   Each processing and storage modules includes a data mirror.

-   If the system detects an error , or a processing/storage module stops responding, it deactivates the failed module

o  The system regenerates data on to a hot spare module from mirrored data, a process known as failover.

o  If a hot spare module is not available or fails, the system continues processing at a reduced performance by directly using the mirrored data. Note that, depending up on the system model, there are between four and sixteen hot spare modules available.

Strength

-   Performance: 10x to 100x faster query performance than traditional data warehouse

-   Effortless scalability

-   Flexibility: designed to grow with the easy addition of modular processing units for scalability &
     reconfiguration.

-   Reliability: Fully ACID and RAID compliant to secure maximum structural reliability.

-   No indexing required.

-   No tablespace & redo log sizing and configuration

-   No journaling/logical log sizing and configuration

-   No page/block sizing and configuration for tables

-   No extent sizing and configuration for tables

-   No temp space allocation and monitoring

-   No maintenance of OS recommended path levels

-   No software to install