The Rise of Operational Intelligence

The rise of operational intelligence for immediate digital insight and action has exploded much like using Waze for turn by turn navigation directions, downloading digital content immediately from Amazon and leveraging a ride-sharing service like Uber.  The ability to see where your Uber driver is in the proximity of your current location and his or her ETA while receiving the real-time best-case route mapping from Waze with construction or accident slowdowns are perfect examples of getting the right amount of actionable information at the right time in a self-service manner.

Do you recall when using a mapping application and it would estimate time-based on some standard formula but not the formula needed at THIS moment? Do you remember taking a taxi in a new city from the airport to a hotel?   Do you remember going to your favorite music store to buy that hot new CD or dare I say album?

I admit it.  I loved going to a local store to listen to the CD sample, chat with the manager about favorite concerts while dissecting the latest CD.  Good times!  In these situations, though, did you get the immediate satisfaction, the right insight you needed at the THAT moment you needed and possess the control to make the decision intelligently yourself based on data?

In the case of a static mapping application, you were presented with more of an average best case and you could have picked from other routes.  Once you were in the taxi, you tend to lose your control of the situation and must rely on the driver’s knowledge to get you where you needed to go.  If you’re lucky (or unlucky) you may hear a great story about the city or a wild and crazy ride the driver experienced or maybe sneak in a nap (inside joke).

To me this transgression symbolizes the transition from business intelligence (BI) to real-time operational intelligence (OI).  BI for years has served its purpose and will continue doing well for that purpose assuming it is aligned to clear objectives, has executive sponsorship, performed against trusted data, etc.

However, how often do you feel the information was truly actionable in the moment of what you wanted?  Was the information too much, too little, too late?  Would you rather be presented a map with the best possible route or a route presented based on averages throughout different times of the day?    I’ll let you think about that for a bit.

Rise of Operational Intelligence

A couple of years ago, I made the shift from BI to OI. Fortunately, I learned how to apply the appropriate methodology to business process operational intelligence where my focus is now.  This philosophy is like my BI practice, but different because the goal is to provide just the right amount of real-time proactive insight and context to a highly complex and constantly changing business process.

Use cases include payment processing, supply chain order fulfillment, pharmaceutical manufacturing, call center operations, and more. The starts and stops of events, the signals they represent across multiple applications and systems call for a robust analytics solution with an in-the-moment business mindset not a system or application performance mindset.

I recently performed research on the analytics alphabet soup — OI, BI, BAM, Big Data Analytics, APM and others, and quickly lost track of the volume, variety, velocity, and veracity of vendors in the space. Instead of getting into the minutia, let’s focus on five keys to providing proactive real-time operational intelligence for business processes.

SLA or Deadline Driven

The top objective for business processes typically involves manufacturing, producing and/or delivering the perfect good or service timely which means completing the entire process and individual steps before a pending deadline and/or SLA.

This applies across industries whether it is a high-value payment process in banking, trade settlements in financial services, supply chain order fulfillment processing or simply needing to exchange B2B files among partners for pricing transportation logistics bids. Missed SLAs typically are costly from a penalty, lost potential business revenue and/or brand or customer experience perspective.

Data, Signals and Events in Motion

When providing insight into a business process, we are working with data in motion meaning transactional events exchanged across APIs, message queues, B2B exchanges and more.  The data is moving extremely fast, rarely at rest, and often machine-generated. Inspecting these signals and data exchanges to understand the business impact is vital. Understanding what customer, partner, plant, warehouse, store, bank, the associated dollar amount of the transaction with a pending deadline are key elements to understand and present in a meaningful way so business operations, customer service and more understand the impact to the business.

To access data that is in motion and occasionally at rest requires a strong and versatile data integration layer. Common integration points include message queues, B2B gateways, APIs, log files, databases, and more. Considerations here include how best to integrate with the data, doing it in an unobtrusive manner and often providing bidirectional integration.

For example, when a file transfer appears to be late, a ticket is issued in Service Now by business operations for the file transfer team to investigate the issue. When the file transfer team performs the investigation via a dashboard, wouldn’t it be nice if you select a button on the dashboard to write back to Service Now stating the fix is in place? Just one example of time savings and productivity gain.

When you look at an end-to-end business process flow like those mentioned above, the word FLOW is key. How can your favorite application performance or system monitoring tool help?  Well, it can monitor system outages, firewall attacks, CPU fluctuations, whether a piece of the application is down. But what good does that do when the business user has a customer screaming at him or her and wants to know where the order, file, or payment is in this complex process?

Remember, business processes = multiple steps + multiple applications + data exchanged across multiple disparate systems. Application performance monitoring tools aren’t going to tell you that your payment cut-off to the Fed is at risk and which customers and what dollar amounts are impacted.  Talk about the time drain on your IT Operations to perform the manual log tracing. Insert that important process you are managing and image the time drain and productivity loss.

This is where real-time operational intelligence comes into play. The need for a layer above APM tools is crucial to look at business processes holistically, highlight POTENTIAL late payments, orders, files, and more; and label the possible business impact proactively.

Key metrics for this include objects that are stuck, sitting in one piece of the multi-step process too long, and piling up where too many objects reside in one piece of the process.  By highlighting these capacity planning-related and performance issues, this is how the rise of operational intelligence provides proactive insight before impacting the customer.

Smart-Volume Analysis

Have you had a situation when the business calls IT days later and says, “I don’t recall getting files, orders or payments from this partner or customer the last few days?” By now, deadlines are impacted, reputation is at risk, but no one had a clue regarding this important exchange of information which impacts downstream applications.

Operational Intelligence monitors expected volumes in real-time vs what is expected leveraging predictive analytics. These real-time predictive monitors can learn on the fly and provide a dynamic threshold to establish alerts where actual volumes are too high or too low notifying the right people immediately of the situation. Knowing this information ahead of time permits you to adjust capacity planning and save the customer or partner experience.

The Essence of Time

Capturing real-time data with historical context is vital to the success of an operational intelligence project. Not only do you need the right information at the right time which is now. Going back in time to learn from past mistakes is helpful in growing more efficient processing going forward and appeases the auditors when asked about a certain time-based event.

For example, while banks process payments from multiple channels, their goal is to complete the processing of all current-day activity before the Fed cut off. Missing these cutoffs leads to holdover situations and associated penalties.

What happens when a high-profile client sends a large payment for processing just before the cut-off? Normally, the bank won’t have an obligation to process it that day, but they choose to make best efforts to do so and request that the Fed window is opened longer.

During an audit, an auditor would likely ask about what happened on this day and the bank would need to show that not only does it have complete oversight and control over the payments that it is processing but also be able to prove that the transaction arrived late from the client.

Conclusion

As you can see and probably experienced, business disruption is driving digital transformation across all industries and resulting in time-sensitive demands from your customers and partners. It is no wonder the rise of operational intelligence with predictive insight is increasing. How well do you think business intelligence is helping retailers in their battle with Amazon or cab companies in their battle with Uber?