How Do You Rank A Good API Or Bad API?

As we are working to populate our Streamdata.io API Gallery with new and interesting APIs, we are faced with the question: how do you tell a good API from a bad one? We want to include the best APIs in our gallery, but as time progresses we want to also be able to regularly evaluate if an API still meets our standards. This seems pretty simple to do when you first get started looking at APIs, but then after you’ve looked at hundreds, or thousands of APIs the question begins to get much more difficult to answer. To help us understand what are the best APIs out there today, we wanted to begin writing down some of the things that make for a good API, that we could possibly automate and help us better understand the good API from the bad.

As we’ve been profiling APIs for the gallery, here are some of the questions we are asking about the APIs we’ve decided to include:

– Discovery – Were we able to find the API easily?
     – Applications – Did we find new applications built on top of the API?
     – People – Are there people who are doing interesting things with the API?
     – Organization – Is it clear that companies, groups, or organizations are working with the API?

– Versions – What versions are currently in use, what new versions are available, but not used, and what future versions are planned and on the horizon? What is the time since the last release, or between releases?

– Paths – What paths are available for all available APIs, and what are changes or additions to this stack of resources?

– SDKs – What SDKs are available for the APIs?

– Repositories – What signals are available regarding any Github presence (ie. commits, issues, etc.)?
     – Contributors – Who are the contributors?
     – Stargazers – What is the number of stars on SDK?
     – Forks – The number of forks on an SDK.

– Communication – What is the chatter going on around individual APIs, and across API communities?
     – Blog – The latest from each API blog. Is there a feed. a number of blog posts?
     – Press – Any press released about APIs?
     – Twitter – The latest from Twitter regarding API providers?
            – Tweets – The tweets from API providers?
            – Mentions – The mentions of API providers?
            – Followers – Who is following their account?
     – Facebook – The latest Facebook posts from providers?
            – Posts – What posts are available?
            – Likes – How many likes does API have?
            – Followers – Who is following their account?
     – LinkedIn – The latest LinkedIn posts from providers?
            – Posts – What posts are available?
            – Likes – How many likes does API have?
            – Followers – Who is following their account?
     – Reddit – Any related Reddit post to API operations?
     – Stack Overflow – Any related Stack Overflow post to API operations?
     – Hacker News – Any related Hacker News post to API operations?

– Support – What support channels are available for an API? Are they supported?

– Forum / Group – What is the latest from groups dedicated to APIs?

– Issues – What are the issues in aggregate across all relevant repositories?

– Change Log – What changes have occurred to an API, that might impact service or operations?

– Road Map – What planned changes are in the road map for a platform, providing a view of what is coming down the road?

– Monitoring – What are the general monitoring statistics for an API, outlining its overall availability?

– Testing – What are the more detailed statistics from testing APIs, providing a more nuanced view of API availability.

– Performance – What are the performance statistics providing a look at how performant an API is, and overall quality of service.

– Authentication – What are all of the authentication approaches available and in-use. What updates are there regarding keys, scopes, and other maintenance elements of authentication.

– Security – What are the security alerts, notifications, and other details that might impact the security of services, or the approach taken by a platform to making sure a platform is secure.

– Breaches – Has there been any breaches lately? Or at other times in the past?

– Terms of Service – What are the changes to the terms of service, or other events related to the legal operations of the platform?

– Privacy – What are the privacy-related changes that would affect the privacy of end-users, developers, or anyone else impacted by operations?

– Licensing – What licenses are in place for the API, its definitions, and any code and tooling put to use, and what are the changes to licensing?

– Patents – Are there any patents in play that impact API operations, or possibly an entire industry or area of API operations?

– Plans – What are the plans and pricing in existence, and what are the tiers of access–along with any changes to the plans and pricing in the near future?

– Partners – Who are the existing platform partners, and who are the recent additions. Maybe some finer grain controls over types of partners and integrations?

– Investments – What investments have been made in the past, and what are the latest investments and filings regarding the business and investment of APIs?

– Acquisitions – What acquisitions have been made or being planned–showing historical data, as well as latest notifications?

– Events – What meetups, conferences and other events are coming up that relevant APIs or topics will be present?

– Integration – What integration platform as a service (iPaaS), continuous integration, and other orchestration solutions are available for helping to make sense of API operations within this world?

– Deprecation – What deprecation notices are on the horizon for APIs, applications, SDKs, and other elements of API operations?

While not all of this can be automated, the majority of them can. Most of the channels we’d find these signals on have APIs. Twitter, Github, LinkedIn, Facebook, Stack Exchange, Reddit, Crunchbase, AngelList, all have APIs that would let us pull the relevant signals we need. Most of these signals are within control of the API provider, which means they could be gamed, or inflated. However, some of them are not, and would actually require genuine traction, engagement, and consumption to trigger the targeted signal. Many of these areas also fall into disrepair as APIs go dormant–an out of date blog, silent Twitter, or dormant Github repository are all excellent signs an API might soon become unreliable.

We aren’t going to be turning on all these metrics for every API in the gallery right away. However we will begin monitoring some of the most obvious channels for signs of life, and ensuring APIs are still supporting their consumers. Otherwise we may reconsider highlighting as part of the API gallery. Currently we have a simple score for each API provider, based upon their overall quality of service, which we will periodically re-evaluate, update, and use to continue determining each APIs value. We will be introducing a handful of controls that let our users determine where the bar should be set regarding the quality of APIs they are looking to stream, allowing them to decide how real time they’d like an API to be, as well as what the quality of the API service should ultimately be. This will be our approach of good API and bad API.