We are always keeping an eye out for how existing API providers are publishing their streaming APIs, and trying to understand the motivation behind how each platform delivers real-time data to their most demanding consumers. We learn a lot by studying the existing streaming APIs out there and develop blog posts, white papers and other valuable industry research along the way. As we were profiling the Meetup APIs, we were intrigued by where Meetup felt the need to provide streaming APIs for events, and specifically that they decided to go with three different flavors of streaming APIs. For each Meetup event, in addition to access via their regular request and response web APIs, you can stream the following three areas of each event:
– RSVP – Streaming the RSVPs that have been for each event.
– Photos – Streaming the photos that are associated with each event.
– Comments – Streaming the comments associated with each event.
Meetup does a good job at augmenting their existing web APIs with streaming editions. We’ve found a couple more that we think would also make good streaming APIs, and will be proxying with Streamdata.io, but their approach is still worth noting. Not just because they have streaming APIs augmenting their existing ones, but also because they provide multiple approaches to getting data in a real-time way.
– Long Polling – This long-polling stream can be easily consumed using JavaScript in any modern browser. Event RSVP, photos, and comments are received when they happen, or as soon as your script finishes handling its last notification.
– Websockets – For browsers that support it, WebSockets is a more efficient alternative to the long-polling stream. This is a push only endpoint and will discard any messages received from the client after the socket is open. Because browser support for WebSockets is limited, we recommend that you consume this stream through the must.js client, which can fallback to long-polling.
– Chunked – Live HTTP stream of event RSVP, photos, and comments within public meetup groups. This method uses chunked transfer encoding to maintain a persistent connection with the client. This connection will only be terminated for server maintenance or a connection error.
Offering a seamless switch between long-running HTTP connections as well as TCP using Websockets, as well as getting data in ongoing chunks of data. We prefer delivering Server-Sent Events streams on top of the existing Meetup web APIs, as it is the most efficient, and easiest to implement approach, but we feel their approach is still relevant. Allowing Meetup API consumers to shift the protocol and message when it comes to getting at Meetup event data in real-time, satisfying the demand for real time event information.
We have profiled Meetup using OpenAPI and added to the Streamdata.io API Gallery. After profiling the Meetup APIs we benchmarked all of their APIs, looking for which ones produce the most data. While the event RSVP, photo, and comments APIs all rate high, we found some others that are worth being delivered as real-time streams. Next, we are profiling their existing streaming APIs, as well as the regular web APIs we are proxying with Streamdata.io using the AsyncAPI specification, and publishing all of them the event-driven API gallery we are working on. Stay tuned on the blog, and via the Streamdata.io API Gallery for updates over the next couple of weeks.
Follow us on social