We are learning what each of the cloud providers has to offer when it comes to streaming real-time data. After looking at what Amazon has to offer with Kinesis, we wanted to see what Microsoft is up to with Azure. Who seems to be really focused on the analysis of data flowing through big pipes, with their Apache Kafka for Azure HDInsight.
Kafka for HDInsight is an enterprise-grade, open-source, streaming ingestion service that’s cost-effective and easy to set up, manage, and use. Build real-time solutions such as the Internet of Things (IoT), fraud detection, clickstream analysis, financial alerts, and social analytics.
Azure’s Kafka solution handles all the partitioning, scaling, storage for you, and providing you with alerting, monitoring, maintenance, and disaster recovery. Delivering a cloud-based Kafka solution that handles any load you want to put on it, and pay for. Using Kafka to fuel your HDInsights managed, open-source analytics service for the enterprise. Focusing more on analysis than Amazon does with Kinesis, but also providing you with a pretty compelling, out of the box, industrial-grade, scalable cloud version of Kafka.
We’ll look around at what else Azure has to offer when it comes to managing real-time data Then we are going to take a fresh look at what Google is offering in this area. Once we have evaluated all of the real-time data offerings from AWS, Azure, and Google, we will compare their features a little more to see what the pros and cons of each are, and see where they overlap. Ideally, there would be some interoperability between each of the provider, that facilitates the operation and migration of infrastructure between/across providers. However, we predict each provider will “have their own unique way of doing it”.
