The Myth of BI for the Masses

Glenn Alsup over at the Viewmark Blog has a great post on how it's next to impossible to properly satisfy all user needs with a single analytics tool/data warehouse. I've blogged on this theme before, but it's worth hammering home time and again, since it's diametrically opposed to what the vendors want you to think. The problem with BI is knowing the information requirements prior to users using the system. Given the role of BI is to support decision making, people don't know what they need until they start to work through the issues - the very thing the BI system is supposed to support. No data warehouse design, however big, is ever going to be able to anticipate the specific information requirements needed by a decision-maker, especially if the decision is a strategic one (the real sweet-spot for BI).

My personal view is that it's better to spend money on a team with some great analytical skills than a software solution that ends up as a glorified enterprise reporting tool. In many decision situations, small-scale, non-permanent 'ephemeral' systems (so-called 'personal DSS') thrown together to answer specific questions are more useful than multi-million dollar BI systems. But then that doesn't sell software licenses, consulting or support contracts.

BI Marketers are eevil. As in fruuuuits of the deveeeil.

Just received via spam from a well known vendor:

Markets Demand Accurate Forecasts - Learn how to improve your forecasts by a minimum 50%

If I'm likely to fall for a line like that, I'd be better off doing a basic stats course to improve my forecasting accuracy. It's silly statements like this that give BI vendors the reputation they have today. Shame on you.

CRM: Seeing things from other people's perspectives.

We've just finished teaching for the semester here at Monash, and one of the subjects I taught was a Masters level unit on customer relationship management. As part of teaching the unit, I created a blog, which I've just switched off, but thought this post was worth saving, and relevant to this blog here. It was originally posted on the 2nd of August, 2007, and appears slightly edited (for context) here.

Sometimes we lose sight of the point of our business initiatives, failing to put ourselves in the shoes of stakeholders like customers (or users, in the case of BI).

Fournier, S., Dobscha, S. & Mick, D.G. (1998) Preventing the Premature Death of Relationship Marketing, Harvard Business Review, Jan/Feb98, V. 76, I. 1, pp. 42-51

I came across the HBR article above today and was struck by its relevance to the Barry Schwartz video below* (and the one viewed in this week's seminar). Although it's a bit old now (nearly 10 years), the article talks about how the idea of relationship marketing (the underpinning of CRM) is often subverted by the very activities marketers engage in to fulfil it. The relevance of Schwartz's idea of the paradox of choice to CRM is that it raises doubts about this ideal of the one-to-one relationship between a customer and a company. The article above builds on this theme very nicely. From the article:

Every company wants the rewards of long-term, committed partnerships. But people maintain literally hundreds of one-to-one relationships in their personal lives - with spouses, co-workers, casual acquaintances. And clearly, only a hadnful of them are of a close and committed nature. How can we expect people to do anymore in their lives as consumers?

"It's overkill," said one woman we interviewed, referencing the number of advances from companies wanting to initiate or improve their relationship with her. "One is more meaningless than the next."

The article points out that consumer-satisfaction is at an all-time low, despite all these relationship-marketing efforts. I reckon a lot of that has to do with the phenomenon that Schwartz talks about, but relationship-marketing, aided and abetted by CRM, seems to only exacerbate it all. You have to wonder how effective our BI systems are at doing what they try to do: improve the decision-making process.

[* The Schwartz video referred to was posted on the original blog - it is a presentation by Barry Schwartz to Google on his concept of the Paradox of Choice. You can watch it here. It's worth watching in it's own right. ]

10 Keys to BI Success

This CIO magazine headline (10 Keys to a Successful Business Intelligence Strategy) popped up in the links to the right recently, and it's worth the read. It's spot on about most things, especially recommendations nine and ten that you should start simple, go for 'low-hanging fruit'. Author Diann Daniel is also quite right in pointing out that the main driver needs to be a c-level executive other than the CIO.

I'm not a fan of the 'X keys/steps to "insert good thing here"' approach to journalism, but good stuff nevertheless.

SAP & Business Objects

Everyone will have heard by now about the massive buyout of Business Objects by SAP. At just under US$7 billion, this is almost twice as much as Oracle paid for Hyperion several months ago (ok, 7 billion US is not that much anymore, but 4.7 billion euro is a nice stash of cash). While the deal is still awaiting approval by shareholders and regulators, it's probable it will go ahead.

The most obvious observation to be made about the deal is that it's a reaction to the Oracle/Hyperion deal, which it no doubt is, in part. There is another angle, though, touched on by Joshua Greenbaum over at Enterprise Anti-matter. One of the points he makes is that the ERP market is starting to stagnate as a result of market saturation. This is certainly the case in the market here in Australia: a conversation I had with a contact in a large BI/Data Warehousing company highlighted the fact that a lot of former ERP consultants are applying for jobs in business intelligence. If the industry research companies are to be believed, BI is still experiencing growth and is apparently on every CIO's 'radar' for the next year (I am so sick of military metaphors for business...).

Joshua also mentions the different perspectives that ERP and BI companies have on what it is that they do, with ERP obviously focussed on transactions, while BI is tools-oriented. This is certainly true, but I think that it goes further than that.

Transaction-oriented systems like ERP or any OLTP system, have inherently different design requirements from systems designed to support decision-making. Transaction systems need to process lots of small packets of data very quickly, are designed for efficient data entry and storage, and are mainly used by people with relatively little organisational authority. They are also, comparatively, easy to design. The workflows and processes supported by these systems (or in the case of ERP, imposed by these systems) are well understood and possible to document in minute detail.

Decision support tools, though, are designed to answer questions and facilitate a decision-making process. Making decisions is a fundamentally different kind of activity to keeping track of business events. The need for a decision-support tool necessarily means that the users (ie. decision-makers) are functioning in an environment characterised by uncertainty. They may have an idea of the kinds of questions they need to answer, but they can't be absolutely sure that this list is complete (and practice shows that it never is). Indeed, in answering those questions, a whole range of new questions invariably arises. In short, the use of a decision-support tool is fundamentally a learning process:

1. The users don't know exactly what they need
2. They don't necessarily need what they want
3. Use of the system itself changes their understanding of what they need
4. The system therefore needs to change to help answer new questions (and back to 3 again..)
   

Learning is an evolutionary process, which means the development and use of decision-support tools also needs to be evolutionary. The difference between ERP and BI is not just a tools-based perspective, it's a need for a fundamentally different mindset on the development and use of the system. When you throw users with a massive amount of organisational clout (senior executives) and cognitive factors associated with decision-making into the mix, the practice of BI and ERP are worlds apart.

Unfortunately, a lot of IT folk don't understand this difference, and we see time and again the use of old-fashioned engineering approaches to BI projects. Such approaches may work fine for transactional systems, like ERP, but doom many BI projects to failure. SAP's own BI module, SAP/BW is widely loathed as a decision-support tool, in large part because it was developed by people with a transactional-mind set. I'm aware of one very large BI project that is currently underway (not using Business Objects or SAP/BW) that exhibits exactly this engineering approach: 18 months into the project, not a single user-facing aspect of the system has been delivered to anyone, with all the work going to the back-end infrastructure. Without something for the users to react against, the learning process can't kick-off, and the requirements (against which all this infrastructure is being designed) cannot possibly be properly understood. Down the track, either one of two things will happen (unless something changes): either the requirements will change and the system will have to be redesigned; or requirements will change, the system won't be updated, and no-one will use it because it doesn't answer their questions. The only way around this is to deliver some quick wins in the form of one or more pilot projects that focus on easy, yet strategically important business areas - get the system in front of some user eyeballs, and the requirements elicitation process will be driven by their feedback.

In any case, to bring this back to SAP and Business Objects, the move makes a lot of sense for SAP. They pick up expertise on developing systems for decision support, not just recording transactions. Hopefully, they recognise this, and the move is not just motivated by a reaction to Oracle and Hyperion, or to tap into a growth market. The fact that SAP are saying that Business Objects will continue to operate as a separate business suggests they want to keep the skill base around, and hopefully it will translate into some skills transfer in the direction of SAP.