Isn't it interesting that AWS and SAP advocate Internet of Things (IoT), yet their IoT offerings have literally nothing in common?

Lets zoom into this:

• Analysis
• Bringing both Worlds together - Solution Architecture(s)
• Demonstration and Oberservations

Analysis

AWS - thanks to the amazing properties of large scale, global reach and a broad fleet of tailored services - a standard choice for any workload for the largest to the smallest companies.SAP - through the vast network of their customers deeply connected in the Enterprise, often running the core business backbone. Thanks to SAP HANA also offering a super high performing platform for Transactional and Analytical Processing in one combined.

Why should you think about a large scale cloud provider in the moment you hold a sensor in your hand?

1. Scale: If your IoT enabled product becomes a success, you do not want to tell the customer: "Wait a few weeks, while we provision more backend capacity." At the same time you do not want to tell your CFO that the pre-launch investment is only 5% utilized. Reducing cost of failure increases the rate of innovation.
2. Security: Larger scale enables higher investments into Security and Data Protection. AWS in general as well as AWS IoT in particular provide an enormous set of capabilities that are hard to match by anyone of smaller scale. SAP production databases run in the innermost security zone of a company. IoT devices are anywhere in the world - It's location can often not be controlled. The strength of IoT for enterprises originates from combined data processing. Security needs to be a key focus area for an Enterprise IoT project.
3. Partner Eco System: AWS is the standard destination where the cloud world does their business.
4. Choice: A large scale provider provides a broad range of choices to realize your needs. This leads to a better solution and often lower cost.
5. Agility: After all IoT is about innovation. Agility is a key dimension for innovation. A large scale cloud provider will provide you with a maximum agility.
6. Reliability: A downtime or data loss is not affecting your business, but your customers. It is therefore the most severe form of downtime impact. Choose a provider with a track record of being the backbone for Internet Scale, reliable solutions.

Yes, cost is not on the list, even though a large scale cloud provider will also provide you with cost benefits if combined with a good solution architecture and operations. 

Bringing both worlds together

In the following I will analyze a few options using which you can receive and process messages from your IoT device in SAP HANA.SAP HANA: As I mentioned last week in a side note, SAP HANA on AWS can process up to 2.2 million messages per second and still look good. Can SAP HANA on AWS also process 2 billion messages / second and continue to provide real time analytics? The short answer is yes, with the right solution architecture.

Below different AWS Options for receiving incoming messages from devices and processing them in SAP HANA. The AWS Monthly cost estimate is for a rate of 1 KB message per minute. (10x messages = 10x cost). Please note that regarding AWS cost the number of devices does not matter, only the number and size of the messages being send/received.

1. AWS IoT: Inbound and Outbound device gateway. AWS IoT rules will put inbound messages into one of the options below for further processing. AWS IoT provides a very sophisticated framework to handle large scale IoT projects with an extra layer of security.Monthly Cost: $ 0.22
2. Amazon Simple Queuing Service (SQS): A queue listener can store incoming messages in HANA with very low latency. (I demonstrated this in 2015 based on S3 Events.) A device or AWS IoT puts a message into the queue. Monthly Cost: $ 0.04
3. Amazon Kinesis Streams: Kinesis has been invented for Real-time stream processing. A Stream Consumer would store incoming messages in HANA. A device or AWS IoT would put record(s) into a stream. Kinesis has the concept of shards which can be mapped nicely onto SAP HANA Table Partitions. Kinesis also enables Stream replay.Monthly Cost: $ 0.0008
4. AWS Lambda: Server-less and state-less functions. A device would trigger a lambda function (asynchronous or synchronous). The lambda function (Node JS, Java, Python) would store the message into SAP HANA. As AWS Lambda does not support DB Connection Pooling, I will not pursue analyzing this option. Without connection pooling the DB connection would need to be closed at the end of every lambda call, which would lead to additional load on the SAP HANA Server.Monthly Cost: $ 0.02 (assuming 100 ms execution time per invocation)

For sending messages to SAP HANA, Amazon Kinesis is clearly the best choice. 

Architecture 1: Amazon Kinesis + SAP HANA

In this architecture the devices put messages directly into a Kinesis Stream.A Kinesis Shard is essentially a partition of the stream. A focus needs to be how to map Kinesis shards onto SAP HANA Hosts. SAP HANA partitioning appears to happen during insert execution. You will observe that the network is flooded with the HANA Hosts moving the records to the right host. I estimate that without this optimization you will only achieve 2 million messages per second if you have 3 x 2 TB RAM HANA Hosts with 10 Gbps (of 20) consumed for inbound processing. With optimization you will achieve more than 6 million messages. Accordingly you should bring the message to the right Kinesis Shard in the first place. Also Kinesis has a partitioning mechanism which you can control. Translating Kinesis Partitioning onto SAP HANA Table Partitioning is not trivial but possible.Without AWS IoT though you would need to realize this partitioning logic on the device. Obviously not a good approach. Naturally you only run into this situation with a scale out SAP HANA System. 

Architecture 2: AWS IoT + Amazon Kinesis + SAP HANA

With AWS IoT things are both more secure (sophisticated security options for an unsafe IoT world) as well as more comfortable thanks to the AWS IoT rule engine. Further on you can easily process the same message in different ways in parallel. In the diagram above we store incoming messages redundantly in Amazon S3. AWS IoT supports also logging requests to Cloud Watch Logs for log analytics, which is great both for developers as well as security teams.

Observations

AWS IoT Button to SAP HANA

The Hello World in IoT is the AWS IoT Button. In my case the path of the click message is:Button Click > AWS IoT > Amazon Kinesis > SAP HANA To enable the scenario I implemented a Kinesis-to-HANA adapter.

SAP HANA Studio perspective on the AWS IoT Button:

The IoT Latency column displays the latency in milliseconds from AWS IoT creating the Kinesis record and my Kinesis-to-HANA Adapter feeding that record into SAP HANA. I have not spend any time optimizing that latency, yet. 

Particle Photon

In 2013, when I considered starting a drone company, this was not something available in the market. A complete micro computer including Wifi measuring 20mm x 37mm (0.8" x 1.5"). That is the same size like the shift key on your laptop.

And while I am porting the experimental AWS SDK for Arduino to the Particle Photon platform, I recognize another value of AWS IoT: The IoT thing world develops with a speed that nobody can predict which chipset or which base boards will be relevant in 6 months from now. In the absence of a standard library for even such basic things like HTTPS, I do not want to estimate how much effort is being spend out there on getting existing libraries to work on a new chip/board. For IoT developers it is therefore of tremendous value if they have to deal with only one AWS Service instead of 60+. While IoT might not cover 60+ services, it allows to be one thing interface to the AWS Platform. 

AWS itself is faced with a similar challenge. Which chips / boards should be supplied with SDKs when the thing world is evolving with such an enormous pace? If I look at recent releases by AWS then the road appears to go this way:

• AWS provides the new embedded SDK for AWS IoT only
• AWS releases the device platform independent parts of the SDK.
• Device Hardware vendors take the effort of adding their AWS SDK device dependent parts as can be seen with Intel, Marvell and a number of others.

And now imagine you are not an AWS or Azure (who follows a similar approach) ...The discussion on what the IoT Cloud Platform of choice is/are, will be over long before the last machines are migrated to the cloud. 

Summary

SAP HANA on AWS does not have a limit on number of devices and messages to be processed. AWS is already now the most common cloud platform for IoT. Running both in the same cloud provides you with

• tighter security options (more integrated controls)
• closer to real time experience (IoT and SAP HANA in the same facilities)
• lower network cost (those can be significant otherwise)
• global reach
• safest business bet, if you want to stay close to the Warp Speed of IoT market innovation

Hope you like the post. Let me know what you think.

Continue Reading

• Provisioning SAP HANA on AWS with the power of your voice
• Bringing High Performance to SAP Mission Critical Systems

Questions? Contact us: questions@ocean9.io