There’s gold in them thar hills

Recently I was working on a pitch to my executive board, and I wanted to include some content that I thought would be perfect that I recalled from an old document. To my dismay, trying to find the file was like going on a quest for the Holy Grail–I looked in Syncplicity, Google Drive, various machines (laptops, desktops), external hard drives, nightstands, you name it… until, finally, I found it, on an old backup that I’d saved to my trusty Iomega Zip Drive that I hadn’t used in years!

When I told a colleague about my quest to find the lost file she laughingly pointed me in the direction of the following xkcd cartoon:

What struck me about the cartoon above was how true it was for me, but it also occurred to me how true it is within an enterprise. Within an enterprise, there are layers upon layers of technologies deployed over time. Many of the older technologies still bring value, making it far from obsolete, but reliance on these older technologies brings cost and debt. On the other hand, there is risk and cost when one considers a rip and replace approach. When I take a step back and look at my company and our customer’s existing architectures, I am reminded of the horrific geology classes from my university days–it is like looking at a geological cross-section of the earth!

Even though many enterprises are undergoing digital transformations, the technology and associated data that is contained within their old and deep layers still provide value. They cannot simply be removed, because after all it is the layers beneath that create the foundation for new pillars to be placed. Many of these layers will often have legacy interfaces (CORBA, RMI, SOAP etc.), or proprietary interfaces only accessible via SDK or they may well be closed systems with no programmable interface. This technology and data still need to be accessible, allowing for it to be repurposed in new ways, such as mobile, IoT, Big Data, machine learning etc.

But, how can these layers be mined? How do we get to these nuggets of gold that are deeply buried in the enterprise? How can we unlock this data so it can be made available to new business opportunities?

The answer: integration technology!

We need integration technology that is capable of leveraging the underlying layers in order to expose these capabilities as a service, or perhaps to simply unlock the data from its primordial datastore and make it easy to consume. Whether you usurp the legacy interface or modernize the data access, both scenarios allow for the legacy system to be accessible via a modernized API.

Legacy application integration, just like mining, provides many different ways to extract data from the layers and make it accessible (API enablement).

The value of an API Gateway

You could attempt the traditional coal mining approach and dig a shaft into the earth where all that appears to the outside world is the top of the shaft, and meanwhile deep underground you’re extracting the coal (value) from the various layers. This is analogous to deploying with a traditional API Gateway. The only exposed footprint to the outside world is the Gateway in the DMZ (i.e. top of mine shaft). The Gateway facilitates the exposure of clean/easy APIs, which allows external API consumers to create their apps and integrations without requiring intimate knowledge of the system complexities or pitfalls. As a result, the technical debt is lessened allowing the legacy technology to become a player within the API Economy.

The value of an ESB

The vertical shaft of the mine is like an Enterprise Service Bus (ESB). This is the backbone to the mine and where miners (humans) and coal are extracted to the surface. An ESB acts as a shared messaging backbone, which is connecting applications and other services throughout the layers of the enterprise. The ESB provides mediators to transform protocols, payloads and data enabling an enterprise to employ various integration scenarios.

The value of microservices

Yet, another way to get access to the layers is to change the architecture towards microservices and structure of your organization in order to adopt a microservice architecture. As you might imagine, adopting a microservice architecture is an extremely disruptive approach and comes with a cost.

A company attempting this type of transformation must be operationally mature. Adopting microservice architecture is the equivalent to strip mining. If you are successful you will be able to unlock value efficiently, but when you are finished you will not recognize the organization. The organization will have increased in agility, culturally changed in order to adopt a DevOps mindset and teams will be organized around services. Conway’s Law taught us that If you want to change your architecture, then you must first reorganize your company to match.

Due to the environmental scarring that strip mining leaves behind, it obviously has a very negative image. However, microservice architecture should not be perceived with the same negativity. Organizations that successfully adopt microservice architecture enjoy and benefit from increased agility and speed to market, because microservices do not require teams to rewrite and deploy the whole application when they add new features. Also, the smaller microservice codebases make maintenance easier and faster. This saves tremendous amounts of development effort and time, and in doing so increases overall productivity. And finally, since its success requires operational (DevOps) maturity this means the time from idea to value in a customers hand will be as fast as your CI/CD pipeline.

The value of iPaaS

In today’s enterprise architecture landscape, more and more we are seeing a hybrid approach where you have a mix of on-premise and cloud. Gone are the days where we see infrastructure and software deployed on-premise within our own managed data center. Today, companies are able to modernize by picking and choosing new layers found in the cloud and then joining the new technology with their existing traditional offerings. This joining is really an integration, and in order to harness the power of the ground to the cloud to ground and even cloud to cloud integration a new type of “as-a-service” has been introduced… Enter the Integration Platform as a Service (iPaaS).

iPass can be likened to harnessing the power of wind in renewable energy. The wind turbine, for example, adds power to the existing infrastructure (power grid) but does not provide enough power on its own. The existing legacy power is still needed, but together they can improve the overall power offering. An iPaaS provides power by providing a means to integrate disparate technologies across network surfaces (on-premise and cloud). The iPaaS turbines provide integration templates, mappings, transformation, mediation, etc along with solutions for common cloud integration patterns, normally targeted at API/JSON-based integrations. Just like the power grid, light-weight gateways exist between the different cloud to cloud and cloud to ground endpoints, much like a substation exists within our current power grid.

The value of the API Composition tools

Even when you have mined the legacy data and exposed it as service, it still needs to be refined; just like many mined resources get refined. The refinery process is to make the API more friendly to the API Consumer. For example, the payload may need transformation, you may want to provide a single facade that aggregates or orchestrates other services. Axway API Builder is a tool that facilitates the refining of both existing APIs and data sources so that it is more friendly to consumers.

The enterprise is full of existing value which can be and mined and refined using many different techniques. “There’s gold in them thar hills,” which can be excavated and exposed to new business opportunities and as we’re seeing, there are many techniques to get to that buried value.