Thursday, October 8, 2009

1,000 Words of Nonesense is Still Nonesense

I really love the good infographics that illustrate the relationship between complex data. A picture really is worth a thousand words. But like good written journalism, good infographic journalism requires some understanding of the subject matter, and an effort to remove bias. Most of my training in critical thinking focused on looking for loaded words in an article; but didn't cover images in much detail, so this is an area that may be ripe for abuse (through malice or incompetence).

Fast Company's article on CEO Pay infographics (the graphics were actually commissioned and ranked by Good Magazine) really illustrates this point. The graphics are very attractive, and do a great job conveying a point. But they don't actually make any sense - they are like a well written article written by someone who has no understanding of the subject.

For example, the first graphic, which was judged best by Good Magazine, and which I agree is the most compelling and information rich image, make no rational point that I can discover. It compares CEO pay with the number of minimum wage employees it could support. Why not compare CEO pay with the number of oranges it could buy? I think it would be just as meaningful. I doubt that anyone really believes that a company could exchange its CEO for a whole pile of minimum wage employees and get the same results, so why compare them?

To me, the only metric that makes any sense for measuring CEO pay, is to ask "would the company be better off if it didn't spend the money?" For example, you wouldn't compare the most expensive machine in a factory to the least expensive machine, and you certainly wouldn't wonder how many of the inexpensive machines you could buy for the price of the expensive one. The only consideration you would have when purchasing a $10M machine is whether the business will benefit by at least (and hopefully more) than $10M.

The second graphic compares the CEO pay to their company's profits. This gets closer to the mark, but of course, since the CEO's actions may take time to benefit the company, you have to consider their pay and the company benefit over time - one year is not nearly long enough. Moreover, you still don't know how the company would have done without paying the CEO, or more practically, by spending half as much on the CEO. If spending an extra $1.00 on the CEO makes the company more valuable by $1.10, this is a good deal, and you should pay as much as you can, until the effect wears off.

Finally, the last graphic is nice for promoting the idea that CEOs are back-room, cigar smoking, old cronies, but it doesn't do a good job of presenting the data - it completely wastes the horizontal axis. The graphic purports to compare age with salary, but it just slaps an age label next to the salary amount - there is no graphical comparison between those two numbers. If, for example, the artist had made the horizontal axis indicate age (the vertical already indicates salary), then we could have looked for a trend or a grouping. As it is, I have to read all the numbers (which are inconveniently written sideways) to determine even the simplest concepts, like does the oldest person make the most money, and does the youngest person make the least? From the data, it appears that there is no relationship between age and salary, although the graphic doesn't highlight this fact either.

I really like good infographics, because they have tremendous power to make complex relationships obvious. I really hate bad infographics, because they give nonexistent relationships tremendous credibility. If we are going to have a contest for the best, we should be judging them on their ability to make the truth apparent, not to make misinformation more appealing. Good magazine is making infographics worse by honoring bad images. Fast Company is making itself look like a business magazine that doesn't understand business when it highlights these as good examples of how to understand CEO pay.