You can’t always depend on teams to report the APIs they have developed, and are using to operate the applications, devices, and system integrations they are responsible for. Many large enterprise organizations already possess directories, catalogs, and repositories of web services and APIs, but rarely are they ever up to date, accurate portrayals of the digital capabilities of the enterprise they are supposed to reflect. In reality that makes log files, and specifically the Apache logs, a pretty ripe source of signals when it comes to what APIs are actually in use across the enterprise.
Every call made to any web server across the enterprise, which often runs Apache records the domain, path, headers, and query parameters being made. When organizations ship these logs off as part of an overall log shipping strategy, which every enterprise organization should have, it makes for a pretty rich repository of intelligence regarding what APIs are in use across the internal and external enterprise landscape. Using open source tooling like Filebeat and Logstash, log files can be shipped in near real-time from across all the web servers in use across the enterprise, and aggregated in a single location for API signal scanning. Allowing for API consumption to become self-identifying as part of a larger, more automated API discovery effort.
Automated API discovery won’t solve all of the challenges an enterprise organization will face trying to quantify their API landscape, but when used to augment a strong self-reporting mandate, as well as a well-known, active, API catalog, directory, and search engine, it can help tame much of the landscape. Providing a more comprehensive approach to defining what enterprise capabilities exist, and help move a more known API landscape forward in concert, across existing enterprise infrastructure in use. The log files for Apache web servers is just one possible approach to establishing more API observability across an organization by leveraging output from existing systems, helping translate existing traffic and operations into actionable intelligence that can be used to drive the next generation of API definitions, design, and deployment–ensuring that enterprise organizations operate as efficiently as they possibly can.
We are working our way through a variety of the common building blocks available for deploying APIs today, and identifying ways in which existing infrastructure like the Apache web server can be used to help automate the discovery of existing API infrastructure. Leveraging existing tooling that is already in place to help us understand what APIs exist, and are actively being used in other applications and systems. Helping us better understand how web services and APIs are in use across the enterprise, so that we can be more effective in our overall API and microservices strategy. The real power of APIs won’t ever be realized until enterprise organizations can clearly articulate what their digital capabilities are, where they exist, and able to put them to work in any situation with as little friction as possible. Do you know where all of your digital resources are?
Photo Credit: Clive Darra
