Big Data technologies represent an opportunity to derive new insight from data on System z and to modernize the entire System z infrastructure regarding the following areas:

  • Enabling true mixed Hybrid Transaction/Analytical Processing (HTAP) workloads and delivering a single workload-optimized system with Operational Data Store (ODS) capabilities that integrates operations and business critical analytics into one streamlined system, e.g. by using IBM DB2 Analytics Accelerator (IDAA).
  • Complementing existing 'traditional' analytical capabilities with Big Data analytics, e.g. by making use of text analytics and Natural Language Processing (NLP) as part of IBM InfoSphere BigInsights.
  • Modernizing the zEnterprise Data Warehouse (zEDW) landscape by significantly reducing the number of traditional repositories, such as a landing zone, staging area, System of Record, data marts, cubes, e.g. by leveraging IDAA.
  • Exploring and visualizing big data and deriving to business outcome oriented insight prior to complex transformation, e.g. through IBM Watson Explorer.
  • Simplifying the often complex and expensive information supply chain, e.g. by using IDAA and through ELT on Hadoop with IBM InfoSphere DataStage Balanced Optimizationfor Hadoop.
  • Integrating Hadoop data repositories as part of a System z centric data reservoir, e.g. by leveraging IBM InfoSphere Information Governance Catalog.

 

This blog serves the purpose to exchange ideas and to better understand the above opportunities and zEnterprise modernization scenarios in the context of real customer requirements and use case scenarios. It should include architecture patterns and deployment models that are derived from real client engagements. The intent is to describe required integration capabilities that play at the intersection between the key products that were mentioned above. Furthermore, the objective is to articulate existing capabilities in DB2 for z/OS, IDAA, InfoSphere BigInsights and related products, but also to understand gaps that need to be addressed.

This blog should facilitate a discussion on modernizing the System z infrastructure as it relates to its monetization aspects and quantifiable business value. Deliverables should be descriptions of client use case scenarios, opportunities for new insight with data on System z, integration and capability gaps, and emerging System z reference architectures.

Kind Regards,
Eberhard.  

E-mail me when people leave their comments –

You need to be a member of WorldofDb2 to add comments!

Join WorldofDb2

Comments

  • System z is the prime example of systems of record. Large quantities of business critical data is generated on System z. That data combined with data produced by systems of engagement is the key ingredient in growing business for companies that realized that these combined systems of insight are the key sources of competitive advantage. In too many cases System z plays relatively passive role in these initiatives which does not match the relevance of data generated and managed on z. For example, data is taken off the platform and analyzed elsewhere, which results in increased latency between data generation and associated information consumption. In some cases this makes sense, and in some it does not. Instead of staying at the meta level in these discussions it would be very useful to analyze concrete use cases, assess possible approaches, apply broader experiences and even identify product enhancement needs and opportunities. 

  • Hello Bill -

     

    Thanks for your comment, I really appreciate.

     

    Well, the System z infrastructure is already used today for processing Big Data, and gaining insight from Big Data on System z. Beyond that, we see an increasing demand from clients including a number of use case scenarios, where a 'traditional' System z infrastructure needs to be augmented and modernized to allow new type of analytics to be executed on non-structured or semi-structured data (e.g. SMF Log records), which requires Hadoop systems with text analytics on this data. This data can still be generated by zEnterprise systems, but can also come from outside the Enterprise.

     

    Of course, gaining additional insight has to be done in an integrated way, ideally not requiring new skills to be acquired by System z personnel (e.g. learning MapReduce programming model). Already today, IBM provides integration between DB2 for z/OS and InfoSphere BigInsights (IBM's Enterprise Hadoop offering) to leverage DB2 for z/OS to interact with Hadoop in order to gain text analytics type of insight for instance from SMF Log records or VSAM files, etc. above and beyond what can be done with Analytics on System z. 

     

    Other examples can easily be derived from the list in the original blog. 

     

    Even today, System z plays a vital role in Big Data Analytics. IDAA for instance has opened doors for HTAP and a significant simplification of the information supply chain. With the InfoSphere System z Connector for Hadoop, we can use DB2 for z/OS to integrate with BigInsights (Hadoop) for text analytics on any type of non-structured or semi-structured data (regardless whether generated on System z or even outside the Enterprise). 

     

    We are interested in understanding real customer use cases, integration patterns and needs - and also sharing points of views and opinions. 

     

    Regards,

    Eberhard.

  • Personally, I believe modernizing the System z infrastructure is a requirement of using it for big data, not an opportunity. Everything you've mentioned is possible and has benefits, but the cost and work involved is not an "opportunity". To me, the real question is what environment is most appropriate for big data in the long term. System z is definitely on that short list and it's a really short list.
This reply was deleted.