Bad BI & Lying Charts

Today statisticians do more than simply explain statistical significance.  More and more we explain the “practical” significance simply.

Read any job posting for a “Statistician” and you will see something like this:

  • Excellent communication skills and are able to communicate technical material to non-technical audiences simply and clearly.

Statisticians must now answer the “why” and “how” questions. We must align statistical logic with practical application. We must bridge the divide between statistician and content expert, and do so while maintaining statistical integrity.  Otherwise, statistics become lies.

Example of a Bad DashboardExample 1 “Interpreting the Picture – The BI Dashboard”

Here is an example of a Business Intelligence Dashboard found online using the search term “dashboards”.  It is a typical IT Dashboard commonly used for corporate decision-making.

Dashboards communicate a lot of information is a small space. However, they can also mislead if they are not constructed properly.  I will focus on one error that could cost an organization a great deal of money and time wasted.

The lower left bar chart, “Average Ranking by Top Performance Categories” compares Males and Females average ranking on “Flexibility”, “Performance”, and “Trustworthy”.  The blue bar represents Females, the brown bar represents Males.  According to the size of the bars, the differences are dramatic. On a scale of Flexibility, Females are almost half as flexible as Males. On the “Trustworthy” scale Males are 4 times more Trustworthy than Females.  On Performance, Males and Females are roughly the same, with Females being slightly better.

I use this dashboard a lot in a presentations.  People instantly understand it. Almost everyone agrees with it.  And some say they would act decisively on it. Would you?

Let’s take a closer look:

The Y Axis (vertical) begins at 2.45 and ends at 2.59. On the “Trustworthy” scale, Females average score is roughly 2.48. The Males average “Trustworthy” score is 2.60. The difference between the average Male “Trustworthy” ranking and the average Females “Trustworthy” ranking is 0.12.  Yet, the size of the bar would suggest a significant difference between genders.

The problem is the Y Axis scale. If we were to begin the scale at 0 the bars would essentially be the same height.  Therefore, the difference of 0.12 would not likely be noticeable. And entirely different management decisions would be made.

However, before you scrap the idea of creating a fantastic HR program to make women more “Trustworthy” you should know there is a test to determine if the difference of 0.12 is significant.

My next blog will introduce the statistical test used to calculate the difference between two averages.

Advertisements