Is there a future for Analytics on IBM Z: BM Db2 Analytics Accelerator

This blog is part of a blog series around Analytics on IBM Z and has been created by 

Leif PedersenAymeric AffouardGuillaume Arnould and Khadija Souissi

IBM Db2 Analytics Accelerator for z/OS 
Tom is a data analytics expert working for an international retail company. He has been asked by his manager to deliver a real time dashboard providing insights about Warehouse Operating costs and cost distribution, total shipments by country, shipment delays in real time with a comparison with the same day of the previous year.
As new results come in, the dashboard users can see if they are performing well and where they need to make decisions to optimize at the right time. This kind of dashboard includes 6 complex queries which need to run against tables including both historical data and fresh data. Applying those queries against transactional data on the system of record to include real time information might have an impact on the transactional workload performance. On the other hand, using a traditional ETL process to load the data outside the system of record would prevent Tom from getting the required real time information in the dashboard. Furthermore, those complex queries might take long time (hours), but nowadays, no user would wait longer than one minute for a report to come with results. Even if he invests long time to tune the query performance, he would get only 30-50% performance optimization.
Tom needs indeed a mechanism which allows him to accelerate his long running queries to seconds, otherwise nobody would benefit from using his dashboard. The target solution has to be integrated with the system of record to ensure that he gets the freshest data in his dashboard. Therefore, it needs to address both transactional and Analytical processing requirements.
The retail company is using IBM Z for their transactions workload. The solution that Tom's team has adopted is based on Db2 Analytics Accelerator which is an analytics engine that is tightly integrated with Db2 for z/OS. With this solution, Tom can benefit from the Hybrid Transactional and Analytics Processing (HTAP) capability, to accelerate his complex queries from hours to seconds without any need to offload data to a separate system of insight and benefit from using real time data without having to care about data latency. Indeed, his dashboards can include the freshest operational data as soon as they are available to the Db2 for z/OS database. Tom does not even need to change his dashboard application or re-write his queries. The usage of the Accelerator is transparent for the Analytics applications.
In order to accelerate his queries, the database administrator simply needs to define the corresponding
Db2 for z/OS tables on the accelerator using Stored procedures or a user interface like Data Studio or the Data Server Manager.
When a table is stored on the accelerator, it is called a shadow table which can be automatically maintained by Db2 for z/OS. The administrator needs to load the data into the shadow table stored on the accelerator. After this initial load of the shadow table, Db2 for z/OS can automatically maintain the data in the shadow table when the data is changed in Db2 for z/OS table through inserts, updates or deletes.
Organizations which have acquired the Db2 Analytics Accelerator can leverage it for further use cases in addition to the acceleration of complex queries. Indeed, the accelerator offers additional capabilities e.g. it can be used as a High Performance Storage Saver (HPSS). This feature is important if the company has historical data on the mainframe which is not actively used but needs to be available for reporting or analytics workload. HPSS archives data of table partitions in Db2 for z/OS to the accelerator by moving data that is no longer actively used to a less expensive storage device. By taking advances of the HPSS feature,Tom’s team can free up costly storage space on the mainframe.

Jane, Tom’s colleague, creates so called multi-step reports consisting of reports which include several subsequent queries. She has e.g. a report with 3 queries Query1, Query2 and Query3 which are run subsequently. Query2 uses temporary results delivered by Query1. Jane can also leverage the accelerator to accelerate her report. The result set of Query1 can be stored on the accelerator as Accelerator only tables (AOT) and can then be used by Query2. An AOT table only exists on the accelerator, but its definition is still available and controlled by Db2. The AOT feature opens up additional usage like using data transformations from existing ETL processes and applying those transformations on the accelerator using the in-database transformation functionality.

Jane can also enrich her analytics reports by including non Db2 sources in the report. This is namely possible by loading data from e.g. MS SQL Server, Oracle, IMS, VSAM or other data sources into the accelerator using the Db2 Analytics Accelerator Loader.

The Db2 Analytics Accelerator provides a cost-effective solution and takes advantage of the superior IBM z quality of service, including availability, reliability and security.

 

The first version of the Db2 Analytics Accelerator was built on Netezza technology and was a dedicated hardware appliance. The latest version of the Db2 Analytics Accelerator is built on Db2 Warehouse on IBM Integrated Analytics System (a Power platform) as a dedicated hardware appliance or it is also delivered as a software appliance as a secure Docker container that can run on mainframe. The two deployment options of the Db2 Analytics Accelerator are functionally identical. The main difference is the capacity and resources available.

 

Both of these deployment options are based on the premier IBM analytical engine, Db2 Warehouse software. This guarantees a uniform experience that allows IBM z clients to invest once to enable their applications and workloads for the Accelerator and then to transition easily between deployment options.

Db2 Warehouse includes latest analytics technology innovations. It represents IBM’s premier analytics engine across many solutions and offers better SQL compatibility across all IBM products as well as a high degree of concurrent users and queries.

Outside of the container, the accelerator solution includes physical compute resources, a server that includes multi-core CPUs, large memory and a shared filesystem to persist the data

On top of the physical hardware there is a Docker supported Linux operating system that is just used to launch the Docker container and manage the Hardware resources.

The Accelerator based on IBM Integrated Analytics System is a pre-configured offering that includes hardware (compute, storage, and networking) and software for easy deployment, management, and high performance. It offers secure, flexible and elastic data storage that is both easy to deploy and easy to manage.

The new option consists of a “software appliance” directly on IBM z. This provides far deeper integration into an existing mainframe environment – no additional, external hardware is required, management of the whole Db2 Analytics Accelerator setup, configuration, resources and problem handling using IBM z system management, and so on.

Now it is really easy to get started with the Db2 Analytics Accelerator and take advantage of the query acceleration and analytics capabilities it provides for Db2 for z/OS. The Accelerator on IBM Z simplifies deployments dramatically to a true download & go experience.

The new deployment option presents a great opportunity for both clients with small scale analytical workloads requirements based on Db2 for z/OS data as well as clients who run massive Analytical workloads without needing any additional hardware for Accelerator.

Related Blogs:

Part 1: Is there a future for Analytics on IBM Z?
Part 2: IBM Db2 Analytics Accelerator
Part 3: Machine Learning on z/OS
Part 4: Operational Decison Manager
Part 5: Modernizing Applications by using API’s

Views: 271

Add a Comment

You need to be a member of The World of DB2 to add comments!

Join The World of DB2

Bringing Db2 enthusiasts together virtually. Expert or novice, distributed or mainframe, this is the place for everything DB2.

Forum

MSSQL linked server attaching to DB2 LUW

Started by Mike Jett in Application Development and DB2. Last reply by Mike Jett 12 hours ago. 2 Replies

I am trying to create a linked server from MSSQL 2014  to DB2 11.1  I am able to create one linked server and get it to work with one DB.  I have not been able to figure out how to add additional catalogs(DB's).  Is it possible to use just one…Continue

Tags: server, linked

Db2 12 Migration Planning & Customer Experiences - Part 1

Started by Sandy Flippen in What's hot ? Sep 12. 0 Replies

Register Now! NEXT CONTENT! Upcoming double webcast Db2 12 Migration Planning and customer experiences. Live Q &A session! PART 1 http://ibm.biz/BdzKxv 24th Sept 11:00 AM EST PART 2…Continue

PlanetDB2.com [Latest Blogs from the Biggest Names in DB2]

© 2019   Created by Surekha Parekh.   Powered by

Badges  |  Report an Issue  |  Terms of Service