I was just browsing the excellent Juice Analytics blog, where I read a post talking about the creation of a wiki discussing the basic core skills needed to use Excel for data analysis. Surf on over to the Excel Core Knowledge Wiki, learn some things, and perhaps contribute!
Former executive editor of Harvard Business Review, business writer, speaker, and unfailing trouble maker, Nicholas Carr is at it again. Carr, author famous (or should I say infamous) for his 2003 HBR article "IT Doesn't Matter" appears poised to stir fresh controversy in the IT industry, warning organizations to stop spending on technology. Read more.
As Marcus has pointed out, Nicholas Carr is at it again, as reported on ZDNet.com. I agree with Marcus' assessment over at 401 Percent that there are some holes in Carr's original article, but I tend to agree with his main point as reported in the ZDNet article: many companies spend too much on technology in the hope of a silver bullet, without considering alternative, even low-tech ways of achieving the same end. In the ZDNet article, Bob McDowell of Microsoft was quoted as saying "There was over-hype in the 90s and there was overspend... [we're] still paying the price now." The problem is, McDowell's assessment is too limited - the IT industry is still making the same mistake, driven by vendor marketing departments that oversell their product, whilst under-delivering. Nowhere is this more evident than in the BI sector.
Time and again I go to vendor presentations and seminars, and the sales pitch is the same: buy our product, and all of the analytic and reporting (read decision-making) needs of your organisation will be solved. Let us tell you about our volume licensing arrangements!
The problem is, few of these products solve any of the really thorny decision-making needs of organisations - especially highly strategic, novel, 'wicked' decision problems that require creative, lateral thinking for their resolution. The needs of decision-makers facing these problems are wide, varied and don't submit to the usual requirements elicitation techniques used during system analysis and design. The tricky part about supporting these needs is not the technology - it's understanding the human aspects of decision-making, including their limitations.
Carr's central point, that companies spend money on IT for a strategic advantage that isn't there, holds for BI too. Maybe, rather than forking out for the new, you-beaut dashboard addon in the latest version of a reporting tool, a manager would be better served by spending that money on some people to help with the decision process. Imagine, instead of spending millions on a BI package, an organisation spending that money on the salaries of a flying-squad of decision/business analysts, skilled in using IT as well as in strategic decision making - a kind of internal consulting company, able to build personalised decision support tools for individual decision-makers.
As an IT person, Carr's attitude bothers me, but that doesn't mean the bugger is wrong. Maybe BI folk should be arguing, too, for organisations to spend less on BI technology, and more on people who know how to get the most out of what's already there.
Just came across another, more established BI Blog called Bounded Rationality, by Mark Madsen. Bit more technically focused than us, and has some interesting things to say about the industry. I particularly like this post on some of the overblown claims some of the vendors put out there.
Listening to your customers (or end-users) is a must for BI systems, or any system for that matter, right? If the users tell you they want a feature you should give it to them - and quickly if possible. Well, no. We often talk in our lectures that the role of an analyst in a decision support setting should be - what we call - an active one. The analyst shouldn't just be a passive interpreter of the user's requirements, translating everything the user's ask for into system designs. They need to take into account what the user community wants, but every now and again they need to intervene and change something. They might need to explain a better way of doing things, or explain that what the users are asking for is silly is somehow or another. All this takes great skill and care not to seem like a typical know it all IT person.
Here is an unusual example of taking what your user community wants too far - and sadly this is going to have a major impact on just about everybody working in BI. There is a feature in the upcoming version of Excel that is just plain wrong.
Microsoft has gone out of its way with its next release of office to take usability seriously. They have tried to embark on a brave remake of the product set. One of the potentially great new features they are including in Excel is the ability to quickly create bar charts that sit (ala Tufte's sparklines) in the spreadsheet cells, next to the numbers and text, not as a separate chart object. Great idea. However, the feedback they got from focus groups is that users don't like the look of these charts when one of the cells is 0 or very low compared to the others, so they have adjusted the bars displayed so there is a minimum that shows even when the quantity is 0. Oh dear. For some example charts and a discussion visit the (excellent) blog at Juice Analytics.
This is perhaps not really appropriate as the first substantive post on this blog, given that it's not really about the technology of BI, but thought I'd post it anyway, as I'll forget if I don't do it now. I've been recently getting into some video podcasts from the TED conferences (Technology, Entertainment and Design). One recent episode was a presentation by Malcom Gladwell (author of Tipping Point) whose main theme was the importance of choice - that is, freedom to choose is closely tied to happiness.
Another, more recent episode, was a presentation by Barry Schwartz (author of The Paradox of Choice) whose point was that too much choice can have a number of strongly negative side-effects. In short, too much choice, amongst other things, leads to paralysis and post-decision regret of various kinds.
So why is this relevant to BI? BI systems are developed to help managers make decisions, and by providing them with information, we increase the number of choices available to decision-makers. Schwartz's point, though, would indicate that a naive approach to this is unproductive: simply plugging a manager into a data warehouse isn't going to help them make better decisions. In fact, it may actually make them worse decision makers.
I don't think too many people would disagree that a good BI tool doesn't give managers more information (well, it might, but not as a primary design aim), it helps managers to understand information to be able to make a better decision. BI systems are there to help provide a supportive decision making environment, and do so by adding 'structure' to the decision problem: that is, they help decision makers to focus on important information, while helping to filter out less important (but maybe still relevant) noise. Mark Silver calls this 'decisional guidance'. On the other hand, though, is still the need to allow the decision maker to make a real, free, and often unanticipated, choice.
My point is this: as BI developers, we need to balance the needs of the decision maker to make a free choice, while not overwhelming them with potentially choice-enabling information. BI isn't about building and deploying a data warehouse and an analytic tool suite. It's fundamentally about providing decision support, and sometimes that means restricting choice, rather than enabling it. Getting that balance right, of course, is often easier said than done, and raises all sorts of ethical issues.
Check out the videos, or subscribe to the podcast, at the TED Blog. I suggest watching the Malcolm Gladwell one first, followed by the Barry Schwartz video.
