Continuous versus traditional data analysis
This is in particular an issue for applications involving ongoing monitoring of IT operations e.g. within a service integrator. One potential problem with such monitoring applications is the tendency for managers to view a continuing stream of analysis and reports without making any decisions or taking any action.
‚Management by Exception‘ is key
For ongoing monitoring work in the context of a shift from iterative to continuous data analysis, there should be processes agreed between the different parties of a value network for determining when specific decisions and actions are necessary—when, for example, warning levels or error report data fall outside certain limits. This management principle is called Management by Exception (MBE) and I believe this will be a more and more important aspect of business intelligence compared to traditional management concepts like ‘management by objectives’ (MBO). Management by exception simply means we focus on anomalies. But with Big Data we might also want to start thinking about cohesive pools of anomalies that must be categorized across providers. Concretely we need to define what exception will have information, warning or error indication character. We still need to analyze those pools of anomalies over time to trigger the right actions of the right teams.
Security Management as a use case for big data
In a mature Service Management organization one potential application of big data would be the monitoring of data provided by a set of key service suppliers and this way creating a social network for hypothesis formation, evidence collection, and collective decision-making. The idea is that as data begin to reveal a trend or finding—say, for example, problem management data suggesting a security breach that could lead to service downtime —an analyst on the line would post the thread and the data on which it was based, and suppliers could weigh in with new analyses and data. Such suggestive hypotheses have been described as “digital smoke signals”. One goal is to determine how likely the thread is to be worthy of detailed analysis and action. But what is more important in my view is that the idea that a service integrator would have a system for circulating data-driven problem indications across suppliers marks a major change in that organization’s culture.
Whether the analysis and decision processes are social within a value network or individual within a service integrator’s value chain, the continuing stream of big data suggests that IT organizations need to think about new ways of making decisions. If it’s worth investing in the collection and analysis of big data, it’s also worth thinking about how the outcome will have an impact on decisions and actions.