A shortage of data scientists - really?

One of the standards of our industry at the moment is that there is, right now (and in the future), a crucial  and serious shortage of data scientists. I was at an industry seminar last week (a roadshow that had been all over the country) where a senior executive from a BI company repeated this "axiom". She showed some charts and numbers to back up her comment. All was accepted uncritically by the audience. It's what we want to believe.

... but ...

Her numbers were based on predictions that were relevant to the US., and she'd just made some simple (and reasonable) adjustments for the size and nature of Australia's economy to come up with numbers representing the shortage in Australia. A reasonable approach, but, I have a problem those U.S.-based predictions in the first place. I think - based on the data we collect at Monash (and I say "we", as I'm a guest at Monash now, and not a full-time staff member) - they are way out.

No doubt there is a significant market for data scientists in Australia. Just log on to, say, Seek.com today and you'll see 200 odd jobs. I think - and I didn't collect the data previously, so I'm guessing - that most of the jobs listed aren't "new" in the sense that most people want to believe - that there's a revolution happening. Most are for jobs that only a few years ago would have been called statistician, and econometrician or operational researcher. Just now these roles labelled being labelled "data scientist". If you delve into the jobs, you'll see listed as required qualification, a degree in "Data Science, Analytics, Operational Research, Engineering, Econometrics, Applied Mathematic or Computer Science" - except for "Data Science" in that list (from an actual current role) all those degrees have been around for many years. The job label "data scientist" is new, and that's good, but relabelling a category of job (even it it represents a maturation or appreciation of the role of IT and of data in those roles) isn't a revolution.

Regardless of what they are called, what of the numbers of jobs themselves, are they growing? In 2015 a feature article in the Australian Newspaper agreed with the still prevailing wisdom that the market for data scientists was growing fast. A 77% annual increase is noted in the article. Well - that might be true, I started collecting data on how many jobs there where for roles matching the description "data scientist" at that time, and I think at the time the article was published there was actually about 70 odd jobs listed on-line, so that could be consistent with the numbers quotes if the growth came from a very low base. What about now, 2 years later? Well not much has happened. The growth has been solid but slow. There are 200 odd listed right now (jobs tend to list for 30 days - so the total count is kind of a moving average), but that's up dramatically in the last few days due to a lot of cross posting for roles in the federal govt. Dept. of Human Services (Data Science jobs seem to get cross posted - list with more than one agency - at a much higher rate than other jobs). So there's a bit of a spike right now. So the longer term trend is slower growth from nothing to a couple of hundred in the last few years. Is that the start of a major gap between supply and demand? I don't think so. Universities all over the country now have graduate and undergraduate offerings in data science. These courses are popular, and a lot of graduates are being produced. Graduates with qualifications in computer science, statistics, mathematical modelling are also being still being produced. Most of the students attracted to the graduate courses have come from overseas, so it's not like there has been displacement from one University course to another at the expense of the "older" courses.

The long term job trend for "data science" - well - I think it's pretty flat. Its largely - for the past 18 months - been between 100 and 200 active listed vacancies. To give some context - there are about 700 active listed roles in traditional BI, and about 2,500 for JavaScript programmers (the leading required language skill for programmers). So no, there's not a massive shortage of people with data science qualifications, and there isn't a massive job market for data scientists.

Feel free to check my numbers here: http://dsslab.infotech.monash.edu.au:8080/datajobs/, or follow the Twitter account @MonashBIIndex

Arrgh. Why do people believe stories that are too good to be true?

OMG help me! It's a simple rule – if it's too good to be true it probably isn't. I'm hot under the collar right now from reading an academic paper that has fallen hook, line and sinker for an urban myth.

It's hard not to get lost in hype. We all believe what we want to believe. We are hard-wired to 'see' the world in a way that confirms to our existing beliefs (it's called the confirmation bias). 

In IT, there is plenty of hype and lots of selective (and sometimes unconscious) use of evidence as justification for positions and beliefs. That's just the way the the world is, and I love that as an academic I have the freedom to rock the boat occasionally and poke fun at some of what goes on in industry from time to time.

I think academics have a lot to offer industry, even though we use often use language that is inaccessible to practitioners (largely because we are writing for other academics). We provide thinking that is objective, theory rich and evidence based. And when it's explained or presented in the right way, this can provide a useful perspective for practitioners to deal with real problems and issues.

However, in the last little bit, many academics have been swept up in the hype surrounding Big Data and Data Science. Davenport's famous statement that being a data scientist is 'The Sexiest Job of the 21st Century' is thrown into conversations, papers and presentations uncritically by academics and practitioners alike. We desperately want this to be true, but seriously, if committing R-code to a GIT repository is sexy, then I'm in need of a whole a new definition of sexy. As a result, we've forgotten what our role should be and have unthinkingly dived head-first into the role of Data Science evangelism. 

That 'need to believe' is going to cause us problems. Data Science is, of course, great, but it has its limitations, and there are many many problems with what used to be called the 'normative approach' to decision support. These problems have been well understood since the 70s and 80s and aren't changed by the use of Hadoop or R, or whatever is the new silver bullet technology for crunching large amounts of data. Nobody that I've seen working on Data Science is seriously addressing these issues. (Have a read of Peter Keen's insightful review of problems facing the approach - written in 1987 - "Decision Support Systems: The Next Decade"). The work being done on Data Science is almost exclusively focussed on the development of new technologies and techniques for crunching data – that's nice, and makes for neat applications, but it doesn't change the kinds of problems that will be solved (which are generally narrow, structured, well defined and almost always operational.)

In reading about Data Science today, I ran head-on into the kind of mistake that an academic shouldn't make. I was reading a paper in a very good journal – highly rated, peer reviewed – on the ROI of data science. It was the kind of journal where, if you are able to get a paper published, your career is made (at least for short while). In its abstract and main body of discussion, the paper repeated an often told story of a large US-based retailer (Target) that used an algorithm to predict which female customers were pregnant, and used this information to send them offers. It's a great anecdote that illustrates the power of predictive algorithms while also showing the ethical line that can be crossed by Big Data analysis. As the paper's authors stated, the predictive algorithm "proved to be an invasion of privacy into the life of a minor [a teenage girl who was correctly identified by the algorithm as being pregnant] and informed her father of her untimely pregnancy prematurely". 

It's a great story, but ... sadly not true. Despite being widely reported in the trade press, it has just as equally widely been refuted. (Fake news!) 

Academics should be better than this. So should peer reviewed journals. Our job should be to seek truth, not to add credibility to made up stories.

Big Data: Massive Ado about ... Nothing?

Some of the hype circulating right now around the topics of data science and big data are really annoying me - used to find it amusing, but now it's starting to make me angry. There is a lot too the movements of data science and big data - don't get me wrong - but a lot of what's being said is just plain wrong and doomed to fail.

My main issue is the complete ignorance of history that is shown by people talking up the concepts. They show - often - that they know nothing of the history of work in the area. The most important thing that they miss in doing that is the hard earned lesson that normative approaches to decision making simply don't work unless they are applied collaboratively, with decision makers understanding the models developed. The conceptual frameworks developed for data science and big data at the moment totally ignore human decision making. They assume more data is better for decision making - it's not. Time to take deep breath I think and build a better and reasoned critique of what missing and why it won't work (I'll post here when I've calmed down).

POD (sitting in a "big data" panel session at a major academic conference, fuming)
Just saw a "big data" process model that ignored decision making, and had developing a business case as step 5 of 6. OMG

Here is a link to my ppt slide deck: https://app.box.com/s/yd9i9fh1go1pbbr4cvwzj62i3yboq3mk

A reading list relevant to his Mastering SAP BA presentation soon.

POD

Peter O’Donnell, a lecturer in Monash University’s Decision Support Systems laboratory, along with Masters student Yasmine AL Ahmadi, is running a project aimed at improving medical reporting. The idea is to apply BI design techniques to reports containing medical test results. If you can spare 5 to 10 minutes you can help. Click the link below to find out more about the project.

If you participate in the study you simply use a browser to access a Web-based system that will take you through the “experiment”. After answering some some simple demographic questions (you remain anonymous), and then you will be shown a series of reports showing some medical test results. You will be asked answer three simple questions about each report. It should take no longer than 5 to 10 minutes to complete the task.

We would greatly appreciate it if you could help us conduct this project either by participating in the study yourself, or letting others who you think would like to participate know about the project. For further information about the study please access the following URL:

http://www.bipathology.com

Thank you
Peter and Yasmine 

What Next for Business Intelligence? Big Data Analytics, Hype and All That Jazz

The slide deck:

• 1 per page [pdf]
• 4 per page [pdf]
• 6 per page [pdf]

More to come :-)

- POD

DSS LAB Internship Sharing: BI application prototype development on iPad

The DSS Lab runs an internship programme for coursework students to get a taste of what it's like to be an active participant in the research group. Jason Lu recently completed a project where he developed a proof of concept iOS interface for a BI reporting tool, something that we hope to be able to use to do research on mobile and touch interfaces for BI. - Rob

In recent years, mobile BI has become a frequent topic of discussion among BI vendors and users, especially after the success of the iPhone and iPad since 2008. Smart phones and tablet PCs are performing an essential role in managers' daily lives, however, BI users and vendors face challenges with using and developing mobile BI applications.

Over the last 15 weeks, I have worked as a research intern in the DSS Lab to develop an iPad application prototype for exploring the dimensional structures of BI reports. In this prototype, I adopted an approach that presented the user interface using Apple's WebView controller, as well as Javascript and HTML. Users select report generation criteria in the user interface and the criteria will be sent to a web server that, in turn, uses SQL to create a query command  for execution on a data warehouse. Finally, the information for the report is sent back to the client system on the iPad to display the report.

One of the issues for the BI software vendors is the need to connect to a server to retrieve data every time the users attempts to generate a report. This creates a problem when using the application without a network connection, such as when flying. One area of further research may be to try an alternative that stores temporary report data locally in the iPad. However, this may present a security issue if the user loses the device.

Some other areas of future research might also include investigating user interface principles for touch such as pre-defined gestures for things like drill down and roll up.

To conclude, this internship experience offered me a great opportunity to study, in depth, some of the advantages and disadvantages of mobile BI applications on multi-touch mobile devices.  In my opinion, with the tablet device, users can interact with reports more intuitively, but they face potential security and accessibility issues along with the benefits provided by the portable device.

Jason

Help needed with a study

I'm running a web based study at the moment. I'm asking people to look at a random series of charts - created using chart styles found typically in current BI systems. Got 5 minutes to help out? Then visit http://vishnu.infotech.monash.edu.au/BIcharts

I'll post a summary of the results here in a week or two.

Peter

Design versus Engineering

Stephen Few recently wrote a post after having attended an SAP BI presentation here in Melbourne a couple of weeks ago (unfortunately I missed it, and the chance to meet him, although a couple of colleagues from the DSS Lab did manage to catch up with him). The central thrust of his post centred on the issue of an engineering versus a design perspective on the problem of providing decision support.

I reckon he's spot on and his argument echoes a line of thought that I've been toying with for quite some time.

For myself, I've been grappling with this concept of design versus engineering as I've practised and taught IT-based design techniques like conceptual data modelling. What's fundamentally different between the two perspectives? Both engineering and design express themselves through the properties of a technical artefact. They also have a symbiotic relationship with each other - one can't do design without engineering, or vice versa. However, intuitively there is a fundamental difference of perspective, and it's taken me a couple of years of reading and thinking to put my finger on it.

The 'aha!' moment came a couple of weeks ago as I was working on a paper trying to more rigorously define the notion of Web 2.0. One of the problems with that idea is that every IT industry niche has jumped on the bandwagon so that now we have BI 2.0, ERP 2.0, even Library 2.0, without really identifying what it's fundamentally about.

My thesis is that Web 2.0 is not about the technology itself - AJAX, RoR etc. etc - because you can achieve the same Web 2.0-style outcomes using different technologies like HTML5 etc. Instead, Web 2.0 is a design concept, perhaps even a design movement. This led me, in a circuitous way, to read a paper by Lynne Markus and Mark Silver called A Foundation for the Study of IT Effects: A New Look at DeSanctis and Poole's Concepts of Structural Features and Spirit.¹ In this paper, they critique an earlier theory of studying IT artefacts based on the idea that what's really important is not so much the features of a given piece of technology, but rather the relationship between that technology and its users.

For example, they draw on the concept of affordance. This relates to the things that an object or environment offers an actor, but it's specific to each actor. While a chair might be something that offers me the affordance of sitting, to a child it is an object that can be climbed on, crawled under, and so on. In other words, to understand a technological artefact, we have to look, not so much at the artefact itself, but how the properties of the artefact and the properties of the user interact.

And this was that aha! insight: the difference between design and engineering is that design is concerned with the space between the user and the artefact; engineers, on the other hand, are primarily concerned with the artefact itself. Both express themselves through the properties of the artefact itself, but design treats that as secondary to what the artefact does for the user, what it means, and how it impacts on the environment.

The problem with vendors in BI, and really most of the IT industry in general, is that we're all focussed on the technical artefact. As a result, the design of how that artefact is used, what it allows users to do, and the impact that it has on the decision support environment is largely ignored. As Stephen Few points out, old-school, large-scale BI vendors are so focussed on their software's bells and whistles that they almost completely ignore the fundamentals of helping people make more effective, better quality decisions.

Following from Markus and Silver's paper, I'd argue that design of IT artefacts, including BI software, needs to consider two key concepts: affordance, as described above, and what they call 'symbollic expressions.' The latter refers to the implicit values, principles and ideas communicated by the technology to the user. Too often in BI, the values and principles that are communicated by tools like Crystal Xcelsius, and lots of dashboard implementations, is form over substance. Dazzling your audience with 3D exploding pie charts and speed-dials is given more importance than imparting a proper understanding of the relevant information.

It's time that we had a design company in Business Intelligence, rather than the plethora of engineering companies we now have. Think Apple versus IBM. Love or hate them, there is no denying that Apple's focus is on the experience of using their products within an ecosystem of iTunes, App Store and iDevice, rather than on the technical attributes of the devices themselves. From a technical standpoint, Apple's devices are not significantly better (and they're often worse) than their competitor's products. But it's the design that goes into the experience of using the device that makes the fundamental difference. IBM, on the other hand, is the classic engineering company focussed primarily on the technical artefact itself.

We need a vendor to do for BI what Apple did for music players, mobile phones and tablet computing. We need a vendor that understands tactical and strategic decision making in organisations and who is willing to have a go at changing and improving that process rather than bolting on yet another suite of 'smoke and mirrors' to their already bloated software.

--
¹ Markus, M. L. and M. S. Silver (2008) "A Foundation for the Study of IT Effects: A New Look at DeSanctis and Poole's Concepts of Structural Features and Spirit," Journal of the Association for Information Systems (9) 10/11, pp. 609-632.

Andrew McLean at Mastering SAP

How cool is this - video's of the keynotes presented at last year's Mastering SAP conference. This talk is from Andrew McLean - from the Monash DSS Lab.

1000_Andrew McLean from The Eventful Group on Vimeo.

Time flies

Just a quick update. Been a hard couple of semesters for us all keeping us busy, making it hard to get our research done, and even harder to get time to blog. However, we have had a bit of a reboot as a group and are now all busy pursuing our research agendas over the Australian summer break and hope to get a bit of time to blog in the next few weeks and months, and hopefully do a better job of blogging more regularly when teaching starts again at the end of the first quarter next year. There have been some significant organisational changes for us, we wound up our Centre and reconstituted our "Lab". Our hope is this will make us more agile - like we once were - one concrete change in the short term has been the establishment of our lab web site at http://dsslab.infotech.monash.edu.au. Nice place holder site there at the moment but that site will grow. It will also be the base we use for web applications and experiments so stay tuned for posts featuring calls for participation that involve using applications hosted at that URL. It will also be the URL to visit to get copies of reports and working papers.

POD

Herding Web 2.0 Cats: Contributions to the Platform

This is the third of three posts looking at the details of a functional framework for Web 2.0 / social media. The earlier posts are:

1. An introductory post is here
2. A look at contributions to the content of social media platforms is here.
3. A look at contributions to the social networks on the platforms is here.

The final category in our social media framework deals with contributions to the social media platform itself. One of the key ideas in O'Reilly's explanation of Web 2.0 is the adoption of lightweight, flexible development methodologies that rely on the active involvement of the community of users to build the platform. Intuitively this makes sense: the usage of a particular social media platform shifts and changes over time as community members use the technology for a variety of real-world purposes. Users stretch and pull the features of a particular platform: think of the conventions that have built up on Twitter: the use of the @user reply format or integration of Twitpic, TinyURL and other similar tools - none of which featured in the original design spec for the Twitter platform.

Platforms that don't adapt and change go into decline. MySpace, for example, is burning a US$580 million hole in Rupert Murchoch's pocket in part because News Corp didn't understand the need to allow the platform to evolve and change in a free and flexible manner. Decreased usage of that platform can be attributed to an increased emphasis on advertising (leading to a crappy user experience) and a glacial bureaucratic process for the implementation of design changes.

Involving the community in the development of a social media platform is an example of utilising the wisdom of the crowds, again a key component of O'Reilly's explanation of Web 2.0. A social media platform is more than just the site itself, but also includes the 'ecosystem' of related applications and support sites that allow the community to use the site for a variety of purposes. This means that there are two different ways in which the community of users of a social media platform can contribute to the platform itself:

1. Contributing to the platform design
2. Contributing to the ecosystem of applications around the platform.

Clearly the former is more common than the latter, and takes the form of both explicit and implicit feedback. Often, the developers of a platform will directly engage with the community to find out how they want to use the platform, looking for ideas for new features. Frequently, though, the feedback takes the form of a change of usage pattern. As users try to use the platform in unanticipated ways, developers respond by either making it easier to use existing features (eg. embedding @replies into the structure of the Twitter platform) or introducing brand new features (eg. the ability to geotag photographs on Flickr).

Very occasionally, a user of a platform will develop a 3rd party application to introduce some feature not directly supported by the platform, or to allow a customised means of interacting with the platform. The plethora of Twitter applications for devices like the iPhone is an example of this, as well as the image, video and link applications that extend the functionality of the basic platform: hardly anyone uses the web interface as their primary means of using Twitter. Sometimes the 3rd party applications end up being integrated into the platform itself (see the Twitter search tool which started as an independent web app), but even if that doesn't happen, the platform is extended and supported by this 3rd party ecosystem, leading to a wider range of uses and appealing to a larger group of users. The easier it is to develop a 3rd party application (through technologies like XML, SOAP or even plain old HTML APIs) the richer this ecosystem will be.

Herding Web 2.0 Cats: Contributing to the Social Network

This is the second of three posts looking at the details of a functional framework for Web 2.0 / social media. An introductory post is here, and the first substantive post on the framework can be found here.

Geez, you sure can tell when semester hits - blog posts here come to a grinding halt! What was supposed to be a short interlude of a couple of days has turned into a couple of weeks.

In the first post on the framework, we looked at three kinds of content contributions members of a social media community can make to a social media platform. This post addresses the second category of contributions: those to the social network itself.

The social network is a critical part of what makes Web 2.0 different - in fact, I think that the social nature of Web 2.0 is the thing that makes Web 2.0 fundamentally different to what came before. Social phenomena are notoriously hard to understand. Just look at the competing paradigms in sociology and social science research, all wrestling with the complexity of explaining non-rational human behaviour.

But for our purposes I think we can conceive of three different kinds of contributions to a social network supported by a social media platform:

1. Creating new social networks or groupings
2. Administering these networks
3. General participation in the social network

Each of these three kinds of activities can be formalised to a greater or lesser extent and either explicitly built into the social media platform, or occur in a much more organic way. Indeed, even when formal mechanisms are in place to establish groups or networks, informal groups also tend to form. For example, Flickr.com's groups or Reddit's sub-reddits provide explicit, formal groups to which community members may belong. But the use of contact or friends lists, or simply engaging with other users on the site can lead to informal groupings that can socialise and collaborate. Networks or groups can be short-lived, forming around a specific event, or more permanent.

Regardless of the purpose of the grouping, at some stage the network has to be initiated which can happen in several ways. Many sites support mechanisms for reflecting real-world social networks such as families and friends through the use of contact lists and groupings. These also support intra-platform groupings as they form as well. Social networks will typically have various formal and informal norms and rules (with occasional discrepancies between the two). A group might form based on the initiative of one, or a small group of members, or the platform itself may encourage a grouping, such as with Facebook's country, city or school-based networks. In the former case, the initiators of a social network may require a comparatively high profile to encourage other users to connect with the group.

Social networks also rely on governance of the group to enforce the rules and norms. In some cases, these tasks fall to the group as a whole, in others there are one or more members designated as 'moderators' or something similar. At first, it may be that the members who started the group perform the role of group moderator, but over time, other members and the group as a whole can take on the tasks. A healthy social network, to a certain extent, will be self-governing, but from time-to-time issues arise where if there is disagreement on how the rules should be applied, or discrepancy between what the group as a whole expects and what is actually undertaken by the group moderators, the group itself may devolve into factions or simply go out of existence. Group governance, therefore, is an important part of ensuring that a social network remains healthy and functional. The health and functionality of a social media platform itself is a function of the health and functionality of the various networks that it supports.

Finally, there are the acts of socialisation of the community members themselves. Typically these socialisation acts will be in the form of the primary and secondary contributions to content I wrote about in the last post. But they also include private messages (like Twitter's direct message feature, or Flickr's mail system). These activities collectively make up the social fabric of the individual social networks as well as the broader community on a social media platform, all of which are further shaped by the technical design of the platform itself.

Interestingly, while the nature of the technology (ie. its design and features) shapes the nature of the social networks on the platform, the community also helps to shape the technology (for the academically minded, this is an example of Gidden's duality of structure and Orlikowksi's adaptation of this idea to technology). This idea, of the platform itself being shaped by the community will be the topic of my next post, which will hopefully not as long coming as this post was!

Herding Web 2.0 Cats: Contributions to Content

This is the first of three posts looking at the details of a functional framework for Web 2.0 / social media introduced in the previous post.

Tim O'Reilly's coining of the term 'Web 2.0' was based on an observation of the rise of a new breed of web company, based on a model of collaboration and socialisation rather than the traditional model of publisher/consumer. As I mentioned in the last post, O'Reilly talks about two different kinds of contributions that users of social media make to a web site that they didn't under the old model: contributions to the data set and contributions to the platform (he didn't use those exact words, though). We also added a third to that list: contribution to the social network. This post breaks down the first of these types of contributions to start filling out the details of our social media framework.

Contributions by users to the content of a site are perhaps the most obvious of the three different contributions, and we think think that there are three different types (how's that for symmetry, eh?): primary contributions, secondary contributions and passive contributions.

A primary contribution is typically the main purpose for which a social media platform is developed. For example, a photograph uploaded to Flickr, a 'tweet' on Twitter, or a video uploaded to YouTube would be considered primary contributions. A primary contribution tends to stand alone - that is, the contribution is worth the community's attention in and of itself. The primary contribution is often a 'package' of content. So, for example, a user doesn't simply upload a photograph to Flickr, they also attach metadata such as a title, tags, a description and so on.

A primary contribution - Flickr photo page, consisting of photo and metadata.

A secondary contribution, on the other hand, is not a standalone contribution, but is submitted as a response to a primary or other secondary contribution. The classic case is a text-based comment, but on Flickr would also include actions like adding to the photographer's list of tags on a photo page or adding the photo to a list of favourites. On a site like Reddit.com, a secondary contribution might be an up or downvote for a comment or link, while on YouTube it might be a rating of a video, or even a video-in-reply. Secondary contributions represent, on an individual level, a reaction to something on the site, and in an aggregate sense indicate the community's perception of that thing. Secondary contributions allow for dialog between users of a platform, thereby allowing communities and social networks to form.

Secondary contributions in the context of the primary contribution above.

Passive contributions are examples of O'Reilly's idea of "harnessing collective intelligence". They come about not as an explicit contribution on the part of a user, but rather as a consequence of their use of the platform, whether that be a primary or secondary contribution or simply through clicking around the site. Passive contributions tend to be contributions to aggregate data derived through some algorithm implemented on the platform: it may be as simple as a view count of a photograph or video, or could be as advanced as Google's PageRank algorithm based on web author's linking behaviour or Amazon's recommendation system based on purchasing patterns. Through the tracking of user actions and the application of algorithms, the platform can add value to the user experience, adding to the appeal of the platform. Think of the predictive search in Google, or the use of tag clouds to handle the problem of building taxonomies.

Contributing content to a social media platform is one of the key ways in which users establish their position within an online community and add value to the platform itself, and is the most obvious difference in the way Web 2.0 works compared to the old publisher/consumer model of Web 1.0. Instead of relying on a large database of proprietary information (the Web 1.0 mantra of "content is king!"), Web 2.0 harnesses the concept of "human computing" to build content that in many cases is more compelling than what you find with Web 1.0. With the incorporation of social elements to the online experience, the next most obvious difference is the formation of social networks supported or enhanced by social media. Contributions to social networks will be the topic of my next post.

Herding Web 2.0 Cats

The BI vendor marketing departments are scrambling to get on the BI 2.0 bandwagon, partly as a means of doing something different with their products and partly because they've seen the success of the Web 2.0 juggernaut and want some of that action. The efforts to date have ranged from the fairly ordinary (let's bolt on a comments feature to our reporting tool, but bury it on a separate screen!) to the potentially good (I'm watching the 12Sprints stuff with interest).

The problem with BI 2.0 is kind of the same problem that Web 2.0 has, though: it's easily dismissed as nothing more than a marketing term, and each commentator has their own take on what it means. At least for Web 2.0 there's Tim O'Reilly's fairly reasonable outline of what he meant when he coined the phrase. For BI 2.0, though, the term means pretty much whatever the marketing department wants it to mean: your tool supports decision automation? BI 2.0 baby! Got a poorly thought out commenting feature? You bet that's 2.0!

POD and I have been watching this go on for a while now with interest and occasional amusement. Clearly the BI 2.0 term is inspired by the Web 2.0 equivalent, but we're not sure that everyone who lays claim to the term actually gets it. We reckon that the idea of BI influenced by the kinds of things that have been happening on the web can be a good thing, but to show that, we need to do some research on it. Before that can begin, though, we need to be clear in our own minds about what Web 2.0 is, and how those Web 2.0 features might apply in a BI setting. Our problem is that no-one, apart from O'Reilly, has really done a good job of saying exactly what Web 2.0 is, so we chose that as our starting point - this post (and the next few) is a way of testing the water to see if what we've come up with in that regard is reasonable.

To break this all into consumable chunks, I'll set this out over several posts, but start with a bit of an overview of what we've come up with - the detail will come later. What we decided to do was to work up to a coherent statement of what Web 2.0 is from the ground up. Rather than trying to cover every commentator's pet definition, we thought it would be more realistic to come up with a functional definition of the term. In other words, forget formal definitions - Web 2.0 is best described by what people actually do with it.

O'Reilly's original outline talks about lots of characteristics of Web 2.0 firms like "harnessing collective intelligence", having a "light" approach to development and so on. At it's core, though, O'Reilly's characteristics boil down to the shift from a consumer model to a collaboration model: the community is not a group of passive consumers, the community fundamentally contributes to the Web 2.0 site. O'Reilly talks explicitly about two kinds of contributions the community makes: contribution of content and contribution to the design of the platform. We took that as our starting point and built up a functional framework for Web 2.0.

In addition to O'Reilly's two types of contributions, though, we added a third. The key difference between Web 1.0 and Web 2.0 is the social nature of the latter (we reckon the term social media is a much better descriptor). Beyond the technology platform and the content hosted on that platform, we reckon that the social network itself is something that people contribute to as well. So, at the highest level, we reckon Web 2.0 can be described functionally with the following categories of use:

1. Contributions to content (see this post)


2. Contributions to the social network


3. Contributions to the platform

Where are we headed with this? I'll flesh out each of these three in three separate posts, and perhaps wrap up with a fourth that summarises it all. But our end game really is to take this framework and use it to look at BI: If BI 2.0 is treated as a social media platform, what features might it have? How might it work? And ultimately, does the idea of applying social media concepts to BI hold any water?

Vendor innovation and the iPad

Steve Remington and I have been working, along with our other colleagues, on a project looking at innovation in the BI software industry. We have been using press releases as a surrogate 'history' of BI firms. We think that they record - along with lots of sales and marketing related information - important events in the development of products. We have been looking at the major vendors, trying to determine if there has been any change in the rate and nature of product innovations since they have been taken over by larger general IT firms. The data is messy and complex and the answer doesn't appear at this stage to be a simple yes or no ... the work continues.

However, we are coming to the conclusion that we haven't seen a lot of really major innovation for the BI community generally - really ground breaking stuff. When you look at the major advances in the technology we use they are either really old (for example, what we now call in-memory OLAP dates back to the late 60s!), or developed elsewhere (the web).

In my shtick on interfaces - the talk I gave in a variety of forms at a number of venues last year - I begin by lamenting that in the 80s (in the 'good old days') the best interfaces were found in EIS tools and in games. Both classes of software really pushed the envelope of what was possible with the limited graphics resources of the computers and PCs available at the time. EIS tools like Commander, Pilot and Holos came with their own GUI system. Macs also featured heavily as a platform at the time because, of course, they have a GUI built in that was easier to develop for than building your own. Then of course Windows came along, and now the web, and BI tools just take advantage of the standard graphics tools built into those underlying platforms. They no longer represent - there are few exceptions, but not many - the leading edge of interface design.

This is a pity I think. Games software has continued to push the envelope, games are still written to take advantage of specialist graphic and interaction technologies and have helped foster the development of new interface techniques (think the Wii).

So, to the innovation of the moment ... the Apple iPad. I like it, I'm wondering why Apple, or somebody, hadn't produced it a few years earlier, but still its a nice device and looks like it will sell well and evolve into a serious category of product. I think it has the potential to be an excellent platform for BI software. Love to see some well design software that allows me to pinch and gesture my way around a multi-dimensional database.

The major games houses have been keen to develop versions for the prototypes they have been given access to, and there are press releases galore from them heralding Apple's product and announcing support for the platform and specific titles and release dates. That's an industry that's willing to take risks and continues to innovate and exploit new technologies. Sadly, so far from the BI software vendors ... I've seen nothing. No doubt there will be, eventually, a few half hearted announcements of the availability of Apps that run on the platform but I doubt there will be enough risk taking or innovation to get me excited ... I guess I can always hope that I'm wrong, but I don't think I will be.

POD

Business Intelligence, 1923 style ...

Continuing on the theme from the last post - a press release from C3 Business Solutions (a Melbourne-based BI consultancy) that includes some statements from me has been in the "wild" for some time now. Thought it was worth reposting here:

http://www.abhishek-tiwari.com/2009/10/is-business-intelligence-too-sexy.html

The vendors do a great job of making everything they do seem exciting and new - that's their job, they have to sell software and that's a good way to do it I think. It plays into the general IT culture. However, there is a real danger we will ignore the lessons of the past if we focus uncritically on the new. I was reminded of this recently when I an amazing article - it was written in 1923 (before the great crash). It was published in the Harvard Business Review and it described the creation of a statistics department (providing information for management) at Eastman Kodak. Abstracting a little (the article referred to draftspeople - report designers, and statisticians - data managers), and substitute the name 'statistics department' for 'business intelligence competency centre' and the advice given could apply to a large company today. It really is a stunning article. Here is a quick summary of a selection of the main points in the paper. They might not all work in your organisation, but they represent some excellent ideas for organisation your BI efforts.

• The head of the BI department must report directly to the CEO. Not to Finance, Marketing or Information Technology. They need to have a direct line to the executive decision making processes; this is the only effective way for them to be able to develop processes and systems that will support the strategy of the organisation.
• If there are already BI groups in the organisation, perhaps groups located out in the business units (creating what today we would call data marts), don’t fight against them, and don’t replicate their efforts. They already will have a good understanding of their functional area and a good relationship with their clients. Allow the central team to (slowly) develop standards, and processes that those groups use. The central team can provide quality control, and also help to minimise areas of overlap and inconsistency. The central team can source data from those groups to allow for more “enterprise” views to be created.
• The BI team should be organised along the same functional lines as the business. This will allow members of to develop a deeper understanding of ‘their’ area of the business and to develop relationships with the people in that area.
• Have equal numbers of people in the BI team devoted to the design of processes of systems to source, transform and store data, and to design the reporting systems that users will actually interact with.

As you read the, remember the paper was written in 1923! The important lesson, I think, is to remember people have been attempting to provide managers with information they need for a long time. Quite a lot is understood about how and how not to develop BI systems them – yet I see a lot of the same mistakes being made. Teams that develop and implement systems that are successful, more often than not, include some people with experience. We should try hard to learn from the past – experience really does make a difference.

POD

Do we really build systems to improve decision making?

Just wrapping my mind around a new direction for a research project. Some of you know, I've been around the traps in the last little bit doing my schtick on interfaces. Of course everybody says the interface is important but I don't think we do a lot about it.Main message of my talk is that we don't devote enough effort or resources to the design of the interface. We just use the "orthodox" tables and charts provided by the vendors in our data displays (and the same for the navigation between views of data) - and many of them are not very good.

We just did a study where we can two different groups of people the same data - but one group got the data in "orthodox" BI charts, complete with 3D effects or gradient shading, not over the top but typical of BI systems today - and the other group an equivalent set of charts - very plain in their design (think the design principles of Tufte or Few). As we hypothesised, when asked questions about the data that required accurate analysis - the group that got the plain charts did much better than the group that got the sexier charts. Neat to see that this study and article that Trevor Clarke wrote for Computerworld after a chat with me has stirred up some debate.

Most BI systems provide users with a slicer dicer style interface in the hope that they will explore the information our systems give them access to - finding the "number" that solves their problem - just like in the demo's the vendors give. Sadly for that dream, in another study, we showed that a lot of people can't use pivot tables.

I think we have a problem. A serious one. Nigel Pendse thinks (based on his global survey) that the mean usage rate of a BI system is around 7 to 8%. That seems a little low to me - but its been a while since we collected that kind of data, and with web technologies (esp. in the last couple of years) meaning that more people have access to BI systems, the proportion using them might have fallen - so while 7-8% seems low, his data collection is good, I've got to think that he must be close to right. Even if he's not, the best take up rates we have seen in our case study work aren't very good.

In my presentations and writing I've been arguing that one main reason for this low usage rate is the interface we provide. I think that most BI systems have very similar interfaces (with few exceptions, the character of the main offerings from the BI vendors is identical). Those interfaces suit a few people, but not many. We need a major re-focus on the role and place of the interface if we are going to get better rates of usage of BI systems.

Now, there are a couple of underlying assumptions to those assertions. The first is that its desirable for lots of people to have access to a BI system, and to use that in their daily work. That may not be true. There have been studies that show that using decision support systems actually decreases decision performance - so the people using BI systems might actually be doing their organisations and themselves harm. I'm happy for the moment to believe in the value of BI systems, and leave the research on that one to other people.

The second assumption that I have been thinking about - is that people use BI system to help make decisions. What if that's not true? What if people use these systems for some other purpose. It could be (as would be predicted by cognitive artefacts like the confirmation bias) that our systems are to build a case to convince ourselves and others that a decision already made is correct?

If that's the case, no wonder we are putting 3d funnel charts in our systems - in response to user demand. The aim of the display of data would be to impress people, show them how clever we are, how the data supports our strategy (better obscure the data a bit just in-case it doesn't). Im exaggerating the case a little (of course), but it is interesting to think about - and not to hard I think to design a research instrument that can be used to go out and find out what BI systems are actually being used for: decision making or decision justification?

POD

:-( back to marking exam papers.

Timo Elliott's demonstration dashboard

Neat little demo put together by Timo Elliott (@timoelliott) from Business Objects using their Xcelcius product as a - think I'm inventing words here - mashupable object. He plonked the dashboard he made, on his blog with a number of controls that allow you - YouTube style - to share via email, twitter or cut and past as an object in html. This kind of functionality will get even easier when HTML 5 gains traction, but as Timo shows its do-able right now. Doesn't matter that it's a dashboard, the idea could work with any report or report part. I really like the ability to click to expand to full-screen view - just link clicking on a web page section in iPhone's Safari, a visual drill-down (or up).

So with a click on Timo's blog and a paste, here is Timo's demo Dashboard (his blog has some context that's fun to read too). To start with our blogger hosted blog isn't coping with the width of the element so well, but Timo kindly set a new code block that does fit - Thanks Timo.

An experiment on Twitter

For a long time a few of us, and not just at Monash, have wondered how a real time text feed - perhaps in RSS format - might be applied to BI systems. Now that social networking sites like Facebook/Twitter/Friendfeed exist and are becoming widely used the idea has a bit more traction with people we talk too - nothing like concrete experience to help people understand what you are talking about.

I've just set up a Twitter account that I'm going to use to develop a prototype system to demonstrate how a "feed" of text updates might be useful in a BI context. This feed (@monashbiindex) will contain updates and observations on data collected as part of out BI Index project.

Right now, all it will report on is a single number - the number of on-line job ads placed in Australia for positions related to business intelligence and data warehousing. I've been collecting this number (off and on) since 2005. It's quite interesting, there are weekly wobbles (up on Thursday and Friday) and significant seasonal variations too (up before and after the end of the financial year - way down in January). The analysis we are doing will soon expand (just like any dimensional data set) to include more detail like locations, industry sectors, job categories and the like. Later we might extend it to other countries but right now we'll stick to Australia.

So, I've got a nice little automated app, that will at 9:00 Am each morning go to seek.com.au, do a search and grab from the resulting page the total number of jobs. It records that number in a database - along with the data and time. Then a 'reporting' app fires up and does a simple day by day comparison of the number to the previous days and posts a tweet. Much better than the Excel macro's I've been messing with since 2005! Once I'm happy with how that's working, I'll extend the range of topics tweeted on to include a wider range of temporal tweets (end of week, end of month, end of season summaries), link the tweet to reports (no 3D donut charts I promise) and start to build agent style data monitors that look for exceptions, to demonstrate how a twitter style feed might be used for reporting the results of data mining algorithms.

POD

P.S. Oh, later the "Index" will include more measures of health than job ads. We are planning a regular survey of the "industry" and also a stock market index - something again I started a long time ago.

P.P.S The BI Job index is based on the number of jobs that match the search terms "Business Intelligence" or "Data Warehous" that are listed on www.seek.com.au. The index is expressed on points based on a ratio of the number of jobs to the number on the day the index started 23/10/2005. On that day there were 349 jobs - 100 index points.