Streaming Kong API Endpoint Activity With Server-Sent Events (SSE)

Staying in tune with what applications are doing with our APIs is the heart of modern API management. Proxying APIs using API management solutions like Kong is standard practice for all API providers. Staying aware of what our applications are consuming, and how they are using our API-driven resources is essential to healthy API operations, and API security. The status of API management endpoints is critical to ensuring APIs are doing what is expected of them, and allowing us to participate in the API economy. As we work to put Kong in place for our customers, and experimenting with what is possible with Kong API management, we wanted to understand which of the Kong administrative APIs would benefit from proxying, and streaming using Streamdata.io. Making API operations more real time, and keeping providers aware of the usage and health of each endpoint. Kong provides an endpoint status API as part of their administrative APIs, which you can call for any of the nodes you have setup using Kong, returning the following response:

{
"server": {
"total_requests": 3,
"connections_active": 1,
"connections_accepted": 1,
"connections_handled": 1,
"connections_reading": 0,
"connections_writing": 1,
"connections_waiting": 0
},
"database": {
"reachable": true
}
}

Using Streamdata.io, you can easily proxy any Kong endpoint with the retrieve node status API, using the following command line:

curl -v "https://streamdata.motwin.net/http://{kong_node_host_or_ip_address}/status?X-Sd-Token=={streamdata_key}"

Once you have a streaming API setup it will begin providing the following details about each Kong API instance:

– server: Metrics about the nginx HTTP/S server.
– total_requests: The total number of client requests.
– connections_active: The current number of active client connections including Waiting connections.
– connections_accepted: The total number of accepted client connections.
– connections_handled: The total number of handled connections. Generally, the parameter value is the same as accepts unless some resource limits have been reached.
– connections_reading: The current number of connections where Kong is reading the request header.
– connections_writing: The current number of connections where nginx is writing the response back to the client.
– connections_waiting: The current number of idle client connections waiting for a request.
– database: Metrics about the database.
– reachable: A boolean value reflecting the state of the database connection. Please note that this flag does not reflect the health of the database itself.

Once you have your API management status streams setup, you can then publish the results to a dashboard, and any web or mobile application. Keeping in sync with what is happening with each Kong API management node. Something you can replicate for each Kong node you have setup, providing real time insight into the usage and health of your entire API management infrastructure. Allowing humans to monitor what is happening, but something you could also hook up to other types of alerts, notifications, automated ways of responding to API activity.

We are going to continue going through the entire Kong administrative API, as well as the different dashboards built on top of the API management platform, looking for other opportunities for turning on real time streams of information. Discovering other ways we can stay in tune with our API operations in real time, using simple, plug and play Server-Sent Events (SSE) powered by Streamdata.io.

For reference, you can also read our tutorial to build an HelloWorld Kong plugin.