Continued from the previous blog (Bad BI & Lying Charts – https://garfieldfisher.wordpress.com/2013/10/31/bi-dashbaords-need-analytics)
The bar chart, “Average Ranking by Top Performance Categories” compares Males and Females on the following characteristics: “Flexibility”, “Performance”, and “Trustworthy”. The Female bar is blue and the Male bar is brown.
When you look at the relative size of the bars, the difference between genders is dramatic. On the “Trustworthy” scale, Males appear 4 times more Trustworthy than Females. Yet, the calculated difference between the two is only 0.12.
- Are Male employees really four times more trustworthy than Females?
- Is a difference of 0.12 significant?
- What decisions would you make based on this chart?
There’s a Test for That
It’s okay to answer “not sure” to each question. And it’s equally okay to want to know. To find out, a statistician would use a t-test. In this example, a t-test would determine if Male employees (average = 2.60) are significantly more Trustworthy than Female employees (average = 2.48), as the bar chart would suggest.
There are many tutorials online that teach t-test. For instance: StatsCast: What is a t-test. (http://youtu.be/0Pd3dc1GcHc) is very good. And without the original data to perform the t-test, I can only suggest that the bar chart is misleading…well very misleading. In fact, it may even be a lie.
t-test defined: “In simple terms, the t-test compares the actual difference between two means in relation to the variation in the data” (http://www.britannica.com/EBchecked/topic/569907/Students-t-test).