Observing and acting on API business and operational metrics

Starting from the vision that only what can be measured can be controlled, we must learn to capture metrics that can help us observe and act on how API-based products behave, for the good and the bad of the company’s business and operation.

APIs, for those who are not familiarized with the term, are the basis of our digital economy, allowing us to connect the most diverse channels and digital experiences. APIs are the bridge of modernity, allowing to establish connections that are based on ease and agility, which allows expanding partnerships and businesses, besides rapid innovation.

The vast majority of our systems, from apps on our smartphone to our e-commerce purchases, among other actions, depending on an API behind the scenes to work.

API business and operational metrics

As in any business, we need to observe how the market and customers react to our products and services, taking the necessary corrective or changing actions to achieve the intended objectives: more customers, more revenue, more success, not losing business, etc.

To gain valuable insights, we can follow a method or sequence of steps:

“Observer” is the entity looking for the insights that it will help in making any decision or action. As an example, an executive, a product manager, a support analyst, an infrastructure administrator, and others.

We must always set goals that answer business questions, otherwise, we can have insights that say nothing and lead nowhere.

 “Objectives” define what we want to achieve with the data, for example: never losing a deadline, not losing customers (churn), doubling the number of customers, reaching one million dollars in payment transactions in 2020, etc.

“Concerns” are directly situations that can occur when reaching or distancing from the objectives: we reached one million dollars in payment transactions in 2020, we reached twice the number of customers, we are losing customers and distancing from the goals, etc.

“Actions” are triggers and alerts that tell us when something is out of control, or is going out of its predicted behavior: we won 10 customers, reached one million dollars in transactions, lost a transaction of one hundred thousand dollars per timeout, etc.

We collect and correlate data to serve different observers

API observers vary widely, but we can study four profiles that broadly cover what we can observe:

• API Product Manager: this is an essential persona, but still in adoption in companies. As the name says it is the person responsible for the roadmap and success of the product. In the API economy, APIs are the products, and they need to be translated in this way, analyzing the market and customers’ wishes and reflecting these wishes in the product roadmap. The product manager expects insights that show that the API product is in the right direction in customer satisfaction and adoption of the product and its features. Example: Did the new feature increase payment transactions as expected?
• API Evangelist is an evangelist responsible for keeping in touch with users and customers to adopt the APIs and identify whether they need help to engage in using them. In general, the evangelist expects to know the engagement and behavior of users and customers, a result directly influenced by his efforts. Example: How many customers are consuming the API product?
• System Administrator API is responsible for the platform that transacts the APIs, being able to manage systems such as API Gateways, API back ends and even front ends that consume APIs as applications. The administrator expects to have metrics that identify that the platform is delivering the agreed service level. Example: Are there transaction latencies and platform availability in line with expectations?
• API Owner is the owner of the API. He/She has the responsibility for the business or technician who keeps the products published and aligned with the general interests of the company. You expect to gain insights into performance APIs in line with business metrics or API development. Example: Is the API consumption as expected?

If we know what we want to observe, we will know what we need to capture

An important step is to identify the data to collect, and thus to correlate the information to generate the primary insights that will help us to observe and take the necessary actions.

APIs like other transactional forms bring to us two concepts: one we can call the envelope, and the other we can call the content. A perfect analogy to a letter where we have the envelope to indicate how the information gets from the sender to the recipient.

The content does not matter to the messenger (the postman) who takes the information from the sender to the recipient. The sender and recipient will establish a connection once the recipient identifies the content of the message.

In an API, the “envelope” information will indicate sender, recipient, delivery and date and time of arrival, quantity, delivery status, etc. Content information can differ but can be payment transactions, statements, a product order, shipping notice, customer registration, etc.

All of this information can be given as “empty” or no value depending on who is observing, the objectives, the concerns, and the actions that are expected to be taken.

Technically, we need to capture the information from our API, which goes from the header data, payload data (content) and HTTP status. These data will provide us with the most diverse information:

• Identify the requesting application through an identifier such as a key or token.
• The content of the transaction, which through introspection can tell us the amount of payment, quantity, and types of products purchased, etc.
• The status of the transaction, whether it occurred or not.
• Several transactions or events.
• Several occurrences.

Based on the application identification, we can verify the application and the owner of this application, and through this information observe numerous variables.

As an example, with the identification, we can track that the application of a business partner, like ACME Fintech, increased in the last week the number of payment transactions calling our payments product by ten times.

This information may show that our product has a return of investment with the business area from this new business partner, including the fact the marketing actions that were taken to expand the adoption of this payment product are working.

To illustrate with views of these insights, I will use the Axway API Management solution with some registered sample APIs. Axway’s solution relies on Embedded Analytics for APIs, a pre-configured analytics tool with data and views for quick insights into the operation and the business. Read the API Management Software Customer Success Report and how they named Axway as a Market Leader.

We will not cover all the functions and dashboards of the solution, but it allows us to analyze different behaviors in real-time, in the past, and in comparison, to periods like a time machine. By bringing pre-designed data and dashboards, it saves us time to extract, correlate, present, and define alerts for taking actions.

In the image above, we see a dashboard with health information for all the company’s APIs in the last 24 hours. In this dashboard, we see the most used API methods, the most accessed back-ends (back-end systems), an overview of performance and activities, as well as a quick view if any API products present problems to the operation and the business.

With this dashboard, an API Owner and a System Administrator can quickly identify that an API behaves inappropriately with high response time, probably impacting a customer and even the completion of a transaction that can directly impact the company revenue.

As an example, we can have an e-commerce payment transaction, where the company invoices a percentage of the value of the transaction, or an order transaction, where the loss of an order may represent that a potential customer moved to the competitor product to complete the transaction.

In the screen above we have a dashboard that brings us information related to “applications.” Applications are exactly the systems, apps, IoTs, etc., that consume the API product or several API products respectively, individually or in groups.

We can quickly identify that an internal system, an application to the client (app or Webapp) or a B2B partner, are consuming the products according to how we anticipate and aligned the business metrics.

On the dashboard, I can identify that the “Mobile-Patient Care” application, and on another dashboard, I can identify that it belongs to the B2B partner “ACME Health,” it has made frequent use of our product, and it has encountered some problems with the response time.

We can also identify the consumption of 830 transactions in the last hour, in line with the business expectation of maintaining a relationship with this customer. In our example, ACME Health is a hospital that outsources services from our company for laboratory tests.

The dashboards show us that ACME has successfully used our patient information products and its exams. The number of 830 transactions is above the forecast of 500 transactions per hour, meeting the concerns of the business area when it closed the contract with ACME.

In the dashboard above, we can directly evaluate a single product in use by a specific customer.

First of all, we can conclude that in general, we deliver the experience that we promise to the customer, except for a product function called “Update Healthcare Situation” that presents an unexpected processing time.

With this information, we can act on the infrastructure looking for problems, as well as we can go back to the product definitions, and perhaps identify that the product’s function was not well designed, and it delivers bad usability that makes it difficult for the user to update the patient information correctly.

Here there is a good chance to understand the customer experience and take actions to improve the experience and satisfaction, the customer wins and the company wins.

In another view, we could reach this conclusion: the ACME client has been reducing the transactions in our system week after week since the first day of last month.

The API Product Manager is evaluating, and it may be that the update was made to restrict the function that queries patient data matches exactly the week that the customer started to reduce usage.

In the examples above, we extracted valuable insights without having to identify or correlate data from API content. We can go a step further by correlating data from API content with the information we already have, and in this way, we could obtain these insights:

• The transactions to update the patient’s health shows a concentration in blood exam tests originating in a unit in the south district of the city.
• The order transactions are not succeeding by a timeout error impacting revenue thousands of dollars in revenue. If we don’t take quick action, our revenue for today will be deeply affected.
• The registrations through the registration API grew in the last week by 50%, and we can identify that these registrations come from a new partnership with a new health adhesion plan.

So, insights can tell us a lot about what APIs deliver to our business. Many insights can be captured from other systems and presented in corporate Analytics applications that companies have, but it is worth remembering that everything depends on the point of observation and the point of view of the observer.

Connections that the APIs establish are a point of view that companies rarely look at. They end up running out of information about something that happens at the connection level.

We can look at connections as an action of software to move information between two points. Yet, we must look mainly at the view of the relationships of companies, people, and as a consequence of all the complexities of the human relations involved.