In 2011, DoSomething.org, a non-profit organization, posted a video on YouTube featuring celebrities asking young people to donate used sports equipment to youth in need. That video was viewed by 1.5 million people. Success, you might think? Well, only eight viewers signed up to donate. Of those eight, none actually donated.
What happened?
How could such a huge number of views not produce any donations? As explained by Jeff Bladt, Director of Data Products & Analytics at DoSomething.org, “We were concerned with the wrong metric.”*
He went on to say, “In the case above, success meant donations, not video views. As we learned, there is a difference between numbers and numbers that matter. This is what separates data from metrics.”
Track metrics, not data
Tracking the wrong data left DoSomething.org with no donations. Likewise, an Operational Intelligence (OI) solution collecting the wrong data can lead a company producing ineffective and, worse, potentially misleading actionable information. There are various ways to pick the right metrics for a company’s OI purposes. Let’s look at one method. Read about the benefits of real-time analytics for immediate and instant payments.
Steps to measure what matters
1. Look at an individual’s operational objectives.
2. Map his objectives to the company’s commitments.
3. Finally, align the metrics.
1. Look at an individual’s performance objectives
The process of defining meaningful OI metrics starts with an individual. Each OI dashboard has been designed for one person (or a category of persons).
That individual is expected to achieve measurable operational performance objectives. Let’s say that the individual for whom this dashboard has been designed for is named Edward. He is the supervisor of a back-office operation for high-value payments in a bank.
Edward’s operational objectives are to never miss a cut-off, always meet the Fed’s deadlines, and achieve higher satisfaction levels on customer surveys. These objectives are mapped to the bank’s customer commitments.
2. Next, map the person to the bank’s commitments
Map the individual’s objectives to the bank’s commitments, obligations, and deadlines. In our example, Edward’s objectives map to the bank’s:
- Customer commitments, (e.g., process all payments by cut-off)
- Regulatory compliance obligations (e.g., meeting the Fed’s deadlines)
- Current strategic initiatives (e.g., improve customer experience)
- Once that has been mapped, the OI metrics need to align with these objectives.
3. Finally, align the metrics
Now select the metrics that align with Edward’s objectives and the bank’s goals. Let’s take the example of Edward’s objective to never miss a cut-off.
The goal is to proactively identify and resolve process issues prior to putting such a cut-off at risk. Therefore, the OI measurements and metrics must create the necessary situational awareness to alert Edward when a situation requires his immediate attention.
For instance, measuring abnormalities that could lead to putting a payments cut-off at risk.
In this case, an Operational Intelligence (OI) solution would measure payments stuck in a step for too long. And it would measure when an abnormally high number of payments fail their STP route and go into a manual repair queue. Another thing that would be measured is files or messages not received on time from a customer.
Any of those situations will alert Edward and trigger him to take proactive actions leading to the avoidance of missing the cut-off.
In addition to the operational objectives, there’s one more measurement to take. To be sure that the metrics are the ones that matter, take the step of measuring the OI solution.
Measure the success of OI itself
It can be valuable and instructive to track the progress and success of the OI solution. Do this by measuring the data before and after the Operational Intelligence (OI) solution has been in production.
In our example of Edward, the back-office operations supervisor, there could be two metrics for measuring the ‘before and after’ data. One metric could be the number of payments that missed their cut-off as a percentage of the total payment processed daily. The second metric would be the value ($amount) of missed payments as a percentage of the total value processed daily.
At first, glance, getting 1.5 million YouTube views seemed like a success for an organization. But it was a failure because their goal was donations, not views. Likewise, it’s important for a bank to look at its goals, and to map its OI solution to track the results that will achieve its goals.
One method of doing this is by looking at the operational objectives of the person the OI dashboard was designed for. Map those objectives to the bank’s commitments, obligations, and deadlines. And, from there, align the metrics. When you go through those three steps, you will find the measurements and metrics that matter
Learn about Axway’s Operation Intelligence Tool, AMPLIFY Decision Insight.
*Source: DoSomething.org blog post Know the Difference Between Your Data and Your Metrics