Monday, August 4, 2008

The Pope, The Atheist and an Irishman called Dave

After looking at a few data sets, you’ll to appreciate the limitations imposed by only having two dimensions in the great data-analytic sandpit. We have seen how being trapped in a two-dimensional world can cause the weak-willed and those lacking moral fibre to yield to Categorical Sin (1 | 2).

Fortunately help is at hand and there are a number of ways to bring extra dimensions into what would appear to be a two dimensional space. For example, the bubble plot allows you to encode a third dimension using the size of the markers in a scatter plot. Add some color–coding of the markers and all of a sudden you’ve four dimensions. Use different makers (e.g. crosses, circles and squares) and very soon you’ll be able to carve out your very own little piece of hyperspace.

Plotting your data is of course A Good Thing. It’s a first step that must be taken before proceeding with statistical analysis. Also creating a nice plot with lots of colors and shapes and all that stuff is quite fun and can make even the pathologically ungifted feel seriously clever. Attractive to the opposite sex even.

Analysis is relatively easy when you’ve only got two variables. You can calculate a correlation coefficient or use linear regression to fit the dependent variable to the independent variable. If the plot suggests the relationship between the variables is not linear you can fit a curve. The plot may suggest that the things (let’s assume they’re chemical structures) for which you’ve calculated the variables are falling into two clusters. In that case you’re probably better off just trying to understand whether there is a structural basis for that clustering.

Life gets a lot more difficult when your plot has extra dimensions squeezed in. How should you analyze that oh-so-sexy bubble plot with the color-coded elliptical bubbles? Well you might respond piously with the tried and tested, “Well, a picture tells a thousand words, you know”. The problem is that data-analytic capability just hasn’t kept up with the graphics. So just how interesting is that bubble plot? Don’t ask us, we just write The Crapshoot. So you might leap onto your high horse and mention that humans can see patterns that computers just can’t. True enough, and that last comment tells us that it’s now time for The Pope and The Atheist to join us.

So what does this all have to do with Popes and Atheists? And who is this Dave chap?

Patience, loyal readers, all will be revealed. The late Dave Allen was a fine comedian and we will try to do justice to one of his contributions despite having heard it only once 20 years ago. The Master would have made this last 10 minutes and would have surely included his abbreviated index finger but we will be briefer. We would welcome feedback from any of readers who may be more familiar with the joke than we are.

The action starts with The Pope and The Atheist in conversation. His Holiness has just returned from the Influencing Skills course that the Lean Six Sigma folk have specially customised to the Vatican specifications.

His Holiness says to The Atheist, “My Son, You are like a blind man, wearing a blindfold, in a totally dark room, looking for a black cat that isn’t there”

“That is true, Your Holiness”, says The Atheist, “but you too are like a blind man, wearing a blindfold, in a totally dark room, looking for a black cat that isn’t there. Only difference is you’ve found it”

Meow!

0 comments: