This blog is the last part of a multi blogs story published by Aymeric Affouard, Guillaume Arnould, Khadija Souissi and Leif Pedersen. Here are the links to the previous entries:
Part 1: Is there a future for Analytics on IBM Z?
Part 2: IBM Db2 Analytics Accelerator
Part 3: Machine Learning on z/OS
Part 4: Operational Decison Manager
Last month, my Internet provider changed my connection from copper to fiber. The diameter of the cable is still the same, but in fact the copper connection uses a pair of copper cables twisted together, where the fiber connection uses a unique tiny fiber surrounded by Kevlar fibers to strengthen the full cable. The magic is that the light is used in 2 different length waves for the outbound and the inbound communication. This new fiber connection is 30 times faster for the downloads and 300 times faster for the uploads than my previous ADSL connection.
My mobile and my internet browser are very much happier. They can send more HTTP requests to the outside world and my user experience is really faster and funnier.
The question can be: what are those HTTP requests doing? What are those devices trying to achieve?
They just want to communicate with somewhere to exchange information. Is there a common matter to do this? Yes of course, on Internet the main language is HTTP: every internet address begins with http://blablabla or the secure version of it with a ‘s’ https:///blablabla. In fact, this standard is used by mobile devices and web browsers, what we call the SoE aka System of Engagement. But the use is of HTTP is broader than just inside the SoE. SoE developers already imposed this language to talk to data servers also called SoR aka System of Record or analytics servers called SoI aka System of Insight. They do that because it’s the language they are already talking, it is easier for them to reuse something already known, and not trying to understand and integrate the variety of different protocols used by those SoRs. They don’t want to spend time making their application talking those different protocols. That is how REST APIs came into life as the standard between the different systems: SoE, SoR and SoI. There are lot of good documentations on Internet on what is a REST API, but here is a simple definition: HTTP + JSON. Which means the communication relies on the HTTP protocol, and if you need to carry data, it will take the form of a JSON message, very less expensive than XML.
Bang! Isn’t it simpler? Everyone talking the same language, a common one, well improved.
Maybe we can push the idea further. Inside the SoR, the tendency is to explode monolithic applications into smaller pieces, easier to manage and to develop by different teams: microservices. What about the communication between those microservices? Can it be REST APIs? Well the answer to that question is not obvious. Let’s go back to HTTP. We saw this protocol is easy to speak by 99% of all the programming languages.
But does ‘easy’ means also ‘efficient’?
The answer is no and here are some counterparts on HTTP:
- HTTP is synchronous: after pushing the request, you wait the answer doing nothing in the meantime.
- HTTP is at the top of the OSI model at the layer 7, machines have to process those 7 layers.
So, REST APIs have a cost. And it’s for developers to find a good balance between the time we spend processing business logic, and the time we spend processing communication between the different business logic blocs. Memory to memory communication is far away faster than network communication like REST APIs. Proof is IBM introduced Shared Memory Communication SMC-R or SMC-D on mainframe to reduce data transmission network overhead when applications communicate through TCP/IP.
REST APIs are also stateless. Which means we can’t execute 2 REST requests under the same unit of work. It’s like a bye-bye to the two-phase commit, well known by mainframe people, for 2 or more REST API calls.
Indeed, let’s go back to mainframe, where business logic is processed in a form of a transaction. Mainframe is well known to process transactions quickly: let say 5-20 microseconds, in a large scale: let say 1500 transactions per second. If transactions are exposed to the outside world as REST APIs, what can be the cost of this web translation? Again, it depends, and it is for each one to choose where to put the cursor, and to choose the technology accordingly. The strategic IBM’s REST APIs gateway for mainframe applications is z/OS Connect Enterprise Edition. The cost of this gateway for web translation, if well used, is some microseconds added to the cost of the initial transaction.
What do I mean by ‘well used’?
The role of z/OS Connect EE is to expose mainframe assets. But those assets are secured inside the machine. z/OS Connect EE must be secured the same way, with encryption, authentication, authorization and maybe audit for every request. Each previous security piece has a cost. This needs to be taken into account when designing the solution.
For example, try to avoid OAuth 2.0 open standard for authentication where for every request, the mainframe will need to do an extra call to the OAuth 2.0 Introspection Endpoint to decrypt the token. Enhance OAuth 2.0 with OpenID Connect where the token can be consumed directly by the mainframe.
Another example is the encryption: when we add the ‘s’ to HTTP! HTTPS has 2 phases: the first one is the handshake where the 2 servers agree on the encryption key, and the second one is the data encryption where data is sent encrypted by the previous key. The cost of a secured handshake can’t be under half a second: 500ms, even on a mainframe with hardware cryptographic accelerators. This communication time is huge compared to the business processing time: 500ms compared to 5ms. The handshake is like the creation of a secured tunnel between the 2 servers. Once the tunnel is established, use it for a time. Do not recreate it for every request.
This a quite long story, starting from the new fiber connection to the use of REST APIs to access the mainframe. This is the path my HTTP requests are taking from phone in my home to a mainframe somewhere when I order items on my favorite retailer (running mainframe) from my sofa. I want those requests to be quick but also secured: to be sure it’s me, not a hacker somewhere. All of this using secured and well-designed REST APIs, but not so many otherwise my user experience will be changed to a wait experience.
