Powerful insights are almost always perishable. Hence there is a need to have a dynamic system to turn insights to decision tasks instantaneously. Analytics initiatives are, by design, rolled out with this intention. And like any other initiative, this too must be justified with the immediate impact created, and mostly on the financial return. Like every piece of luggage in the airport needs a TSA approval, every corporate initiative needs an ROI stamp. However, estimating the impact of analytics projects is not straightforward, especially, if it’s not measured against the intent with which the projects were initiated.
This must be done by first assessing the purpose of the project within the strategic framework of data intelligence.
Vision:
Analytic systems can enable decisions, such as staffing a store through labor forecasting, by providing the right data at the right level. Such decisions, in addition to having an immediate financial return on investment, also can have a lasting impact by improving relations and overall brand image.
Enabling new processes, like collaborative decision-making, can benefit the organization with 'time to decision' and 'time to action', making intervention tactics more targeted and impactful.
Impact measurement in this area includes interviews with key stakeholders and process champions, and a detailed supporting survey that captures the following key attributes:
- Type of decisions and whether they impact the top or bottom line
- A decision process map and potential impact points across the business value chain
- Validate viability of specific measurement techniques (like financial ROI, satisfaction score, image indices) across those impact points
- Establish 'time to decision' and 'time to action' as separate KPIs and track them pre- and post-implementation
Governance:
Many analytic initiatives impact the richness and the speed of decisions. Building comprehensive systems by rationalizing reports and improving the accuracy of insights achieves richness. Automating reports to eliminate manual work enhances efficiency. Creating SSOT (Single Source of Truth) advances consistency.
The impact of these is measured by estimating the effort differential – pre versus post. A survey with a detailed questionnaire to the key users should accompany this evaluation covering the following key attributes:
- Establish a confidence index that measures end-users trust and usability of the data and KPIs.
- List and quantify the replaced systems
- Highlight areas of duplication removed and consistency gained
- Take stock of status quo reports that the new system rationalized
- Quantify the number of person-hours required to develop the existing reports and dashboards, to contrast against post-implementation
Consumption:
Analytics effectiveness depends on whether it supports the organization to have a data-driven or data-led culture. Success starts with democratizing the data so that the right data at the right level is available for the concerned members. Data modelers, analysts, directors, and executives need data, but the source/granularity/form depends on their roles and decisions they need to make. Good analytic systems also focus on user experience by improving how users interact with the data. This focus has the additional benefit of promoting the adoption of data-led decision-making.
It is vital to measure the system's usage metrics, such as number of users, clicks, and session times, to measure the impact of these prior mentioned objectives. Systems should track these key usage metrics:
- Number of users accessing the new system
- Usage patterns by key personas
- Assess 'time to decision,' by mapping the potential follow up analysis areas
- Set up methods to capture the critical objective for the user session, and during log-off, record the session's success in enabling the user to meet their goal.
Project ROI measurement removes waste and encourages accountability. While it is necessary to measure, compare, and contrast different projects' ROI, a standardized framework cannot be the only measurement of success. Otherwise, innovation and growth will be stunted as all projects try to fit into arbitrary goals. The data intelligence framework presented here and summarized in the table below is a reliable tool to measure success and identify potential improvement areas for your Analytics projects.
