API Management

What are API metrics and how do you measure them?

What are API metrics and how do you measure them?

APIs are priceless. APIs allow companies to tap into their complex technology make-ups to create new digital experiences, build new services to grow revenue and provide security. API adoption leads to lower costs and an increase in sales, operating income and even a company’s market value. In addition, the creation of API developer portals is associated with decrease in R&D expenditure.[1] But how can you measure API success and what API metrics should you track?

What are API metrics?

API metrics help companies understand the operational performance of APIs, and their contribution to the business. Business users will want to know the impact of APIs on business outcomes (how many new customers have signed up through an API channel), while enterprise architects will be more interested in metrics indicating the performance of the API platform (number of API calls).

API monitoring is crucial given the importance of APIs to a modern business. It can indicate digital business health and allow businesses to take immediate corrective action if an important app is down. It can help determine which new services are performing well and which aren’t. That’s why we’ve talked to our customers and experts about business and API metrics and here are the findings of our research.

What can you measure?

Measuring the value of APIs in your own company is not easy. There are four types of API metrics that companies can track. Let’s think of it as a pyramid. At the bottom of the pyramid there are operational API metrics, followed by business API metrics, monetization and then, at the top, the ecosystem value.

Operational metrics

The most basic set of API metrics is operational. They measure API performance, such as the number of APIs, the number of API calls, CPU or memory usage. Operational metrics give an indication of the operational stability of an API platform.

Business metrics

The next set of API metrics are business performance metrics. They give a better idea of the business value of APIs as they can measure the speed of onboarding, retention, customer satisfaction or a number of partners using your API. These metrics look at how an API contributes or supports a business value indirectly; a core proposal of a business is not speedy onboarding of their partners, but we can’t imagine a thriving business without that capability.

Monetization

Monetization tracks incremental revenue from an API, direct or indirect. Only a minority of APIs can be monetized directly, i.e., API consumers pay directly for their API consumption. Different pricing models exist to charge for API calls (e.g., pay per call, subscription). Getty Images or AccuWeather are examples of companies based on direct monetization models.

Ecosystem value

This is where the true value of APIs lies. Some refer to it as indirect monetization—enabling new channels for existing business activities. It focuses on revenue that is enabled by APIs. The question isn’t “What is the direct value of an API?” but rather “What is the value of the business?”. It shows how much business an API enables, how much business flows through an API. For example, 90% of Expedia’s and 60% of eBay’s revenue come via an API.

What to watch out for

Don’t be too operational: Companies tend to focus on operational metrics at the expense of metrics indicating the value of APIs to the business. Why is that? A few reasons.

Firstly, many API initiatives are driven solely by IT, so they become too IT-centric during implementation. Axway’s Catalyst team shares this experience; a lot of companies tend to think operationally (e.g., the first thing is to authenticate a user), so the Catalysts encourage companies to focus on the value instead (e.g., why does the user need to be authenticated? What is the value of that?).

Secondly, operational metrics are easier to establish and track than business metrics. And finally, there’s a misconception that operational metrics are an indication of business metrics—so two birds, one stone.

Pressure to prove ROI: during our research, a trend emerged: the more the IT department cooperates with a line of business, the less pressure they feel to prove the value of the API investment. Where the request to create an API came from a line of business, or where the line of business tracked the metrics, IT didn’t feel pressured the prove the API value and come up with metrics.

One customer was struggling to come up with KPIs for an API Portal, because it was purely a technical investment and he assumed he had to come up with an ROI justification for the line of business.

Monetization can create friction: Only a few companies’ business model is based on charging for API consumption. The value of APIs to the vast majority of companies is the fact it enables their participation in the wider ecosystem. Charging for these APIs would create unnecessary friction—in fact, it may be more valuable to incentivize others to use your APIs!

For example, 20-25% of JustGiving’s revenue comes from its APIs that is used by developers to build applications to help raise money for charitable organizations. [2]

What API metrics do companies track?

Indeed, most companies tend to track operational rather than business metrics. In a survey on API KPIs conducted by SmartBear[3], 76% of respondents focus on performance, followed by usability/developer experience and uptime/availability (58%), and number of API calls (53%). “Gartner’s 2019 Research Circle Survey on API Usage and Strategy found that 50% of respondents don’t measure anything directly attributable to business value and only measure runtime metrics such as the number of API calls. Nearly a third of respondents don’t measure any metrics at all.”[4]

From our customer interviews, I found two interesting things:

  • One customer measures the “reuse factor,” i.e., how many times an API is reused. It doesn’t measure WHAT the saving is, but the NUMBER of times a saving has occurred because they didn’t have to build something new. Another customer independently complained that it is a pity they don’t have an API metric to track how many times an API is reused.
  • Another client said that IT doesn’t really measure business metrics for APIs. Business users initiate a request for a particular API to be created, and they look at results themselves in different databases—they use different databases because it differs with each API created.

How do you decide what API business metrics to track?

The best way to decide what API business metrics to track is to start by cooperating with your business counterparts. You need to understand who you are measuring the metrics for and why.[5] IT needs to understand the key business goals that the organization or a particular business unit has (this could be, for example, increase in online sales). The next step is to translate the business objectives into measurable API metrics (e.g., online sales via an API channel), and select just the top five. Finally, assign APIs to API metrics. Now, you just have to track and report the results to your stakeholders.

The TL;DR commandments

Measuring APIs is difficult. But there are a few things you can follow and watch out for to make your life easier. Here’s some of the key takeaways to remember from our research:

  1. IT and line of business should cooperate in coming up with success metrics for API programs
  2. Don’t be too operational—focus on the value that APIs bring
  3. Don’t monetize everything
  4. The true value of API is its ecosystem value

Read about API product intelligence with these 10 KPIs to support your strategy.

[1] Seth, G., Benzell, G.L., and Marshall V.A. (2017): The Impact of APIs on Firm Performance

[2] Boyd, M. (2014): API Strategy and Practice Day Two: The Values Behind an API-enabled Sharing Economy

[3] SmartBear (2019): The State of API 2019 Report

[4] Gartner, How to Use KPIs to Measure the Business Value of APIs, Shameen Pillai, 29 January 2020

[5] Musser, J. (2014): KPIs for APIs: Why API Calls Are the New Web Hits