I’ve been scaling up my monitoring of Github as the API Evangelist. The platform provides me with a wealth of information regarding what companies and individuals are up to in the API space. Each Github user possesses an RSS feed via the platform, making it pretty easy to tune into what any individual user is up to via the platform. I wanted to build me a dashboard for monitoring activity across the platform using Github activity feeds, but as soon as I got to work, I saw there were two major hurdles I needed to get over before I could deliver the real time dashboard I envisioned.
1) Aggregation – I didn’t need a single user, I needed to monitor hundreds, and even thousands of users–I would need to be able to aggregate multiple feeds into a single.
2) No JSON – Github provides RSS feeds for each user on the platform, and to be able to proxy with Streamdata.io, I would need them to be JSON feeds.
To help me achieve my goals I got to work developing a simple RSS aggregator, that would also convert the XML returned into simple JSON. I built my prototype in PHP, but will also be replicating in Node.js, and Python pretty quickly so that I can deploy on a variety of platforms, including as serverless functions running on AWS Lambda.
Hopefully this article about Github activity feeds provides you with an example of how you can quickly augment existing RSS feeds with JSON APIs, and then aggregate and convert to JSON, which can then easily be made real time using Streamdata.io. We are a big fan of keeping Streamdata.io doing one thing and doing it well, and leave it to people like you to help us create the dead simple JSON APIs which can act as feeds for streaming APIs published via our service. The reports of RSS’s demise have been greatly exaggerated, and there are endless opportunities to publish, aggregate, and translate RSS into more meaningful streams, delivering real time news and other content where it matters.