We spend a lot of time profiling existing web APIs, looking for event-driven opportunity. Streamdata.io is in the business of providing event-driven architecture, so we are always on the hunt for platforms that need our services. This involves profiling the APIs a many different platforms, looking for a lack of event-driven infrastructure across many of the existing APIs that operate within in variety of industries. One thing we are seeing as a result of our effort profiling platforms for the API Gallery™, is a heightened awareness regarding the layers of event-driven API architecture. Something that isn’t just about using Kafka, or implementing a WebSub solution, and is something that might be as simple as being aware of all of the resources you make available, and how often they change.
We’ve begun seeing several layers emerge from the research we are doing on the existing API landscape, helping us get better at defining the event-driven API opportunities that exist across many business sectors.
– Resources – Having a clear inventory of all digital resources made available via an API, having OpenAPI definitions available for the entire surface area of how these resources are accessed, as well as JSON schema for each resource’s definition.
– Tagging – Being able to tag APIs, gateway, management, orchestration, database, and other architectural components based upon the resources they serve up. Managing a centralized list of tags based upon the catalog of resources that are available.
– Change – Having visibility into how often resources are changing, understanding when they are added, updated, and deleted. Mapping out which resources are changing, and details regarding the granularity of those changes.
– Event Types – Producing meaningful definitions of the types of events that are changing, going being the CRUD resources, and possessing conversational descriptions of all events that are occurring across a platform.
– Push Technology – The ability to push data out via webhooks, based upon events occurring, making sure that data is where it needs, when something has occurred.
– Persistent Streams – Having the infrastructure to reliable deliver real-time streams of data at scale, delivering small, medium, or large volumes of data within the enterprise, as well as the last mile outside the firewall using the web.
– Subscriptions – Allow consumers to subscribe to specific events and resource topics, and choose the method for receiving updates as simple API requests, webhooks, or as streams.
– Discovery – Having API infrastructure, as well as the meta data about resources, event types, schema, and other aspects of operating event-driven infrastructure well documented for humans, as well as other systems in a machine readable format.
This isn’t just snapshot of specific event-driven architectural services or tooling. These layers are meant to reflect the awareness that comes along with event-driven API investment, expansion, and evolution. As we map out the existing API landscape, we are using the presence of these layers to quantify how far along in their event-driven API journey the platforms we profile are progressing. If a platform can articulate what their resources are, have them well tagged, and begun defining event types, as well as providing webhooks, they are clearly moving forward in their event-driven efforts. Those who are investing in streaming technology, and moving large volumes of data around within the enterprise using Kafka, and making available to partners, and 3rd party consumers via real-time subscriptions, are the most mature.
We are working on more examples of these event-driven API layers out in the wild. Gathering up examples of platforms that do it well, profiling their APIs, the protocols used, and how well they document their event types, topics, and the other movement meta data parts of their operations. As the number of APIs in our API Gallery expands, our understanding of these layers increases, allowing us to better produce landing pages that help highlight event-driven architectural components like webhook implementations. Allowing us to identify and aggregate event-driven architectural patterns we find across the space, and organize into sub-projects that help us illustrate what is possible when you invest in event-driven approaches to doing APIs.
