Monday, September 10, 2007

Dr. Deming and the Petraeus Report


General David Petraeus testified before Congress today on the progress of freedom in Iraq. In his testimony the officer used graphical slides to show data over time. As he spoke a number of items struck me. First he mentioned "refining data". Having seen the Bush administration wipe out nearly 2 million uninsured Americans using data refinements, I wondered what calculations has been changed, when and why. Did any of his charts show data operationally defined in different ways over time?

The second thing he said was U.S. intelligence agencies called his data "the best in Iraq". That could be a very low bar, given the poor governmental infrastructure. One could be the best, but also be a very low performer.

Conservative columnist Jay Ambrose recently cited Dr. Deming's "techniques of measurement" in a recent column on the poor quality of Chinese products. Yet, Dr. Deming pioneered and taught for decades robust statistical analytical methods for leaders. I saw none of his tools applied to General Petraeus' data. From a cursory review, what the military leader cited as a trend, Dr. Deming would call common cause variation.

I suggest the Pentagon have a respected statistical follower of Dr. Deming analyze the data. Then have such a person report to Capital Hill. This would give the American people a chance to gain knowledge, instead of hearing someone say what the President wants to hear. George Bush gave us "unprecedented disaster" regarding Katrina while David Petraeus ended his testimony characterizing Iraq as "the most complex and challenging environment imaginable". It's a shame our leaders consistently fail at complex and challenging tasks.