PQ Systems - Quality eLine
>> In this issue:
 

Real-time charting means no waiting for process feedback

Reader survey

Quality Quiz: With a video!

Data in everyday life

Six Sigma and more

Bytes and pieces

FYI: Current releases

 
>> Be social:
 

Visit our Quality Blog and Twitterfollow us on Twitter.

 
>> Sign up:
 

Just type in your friend's e-mail below to have them receive Quality eLine:

 
>> Software:
 

CHARTrunner

GAGEpack

SQCpack

CHARTrunner-m

DOEpack

Quality Gamebox

 
 

Six Sigma and more: David Schwinn
Still managing by the numbers in Atlanta and elsewhere

In early July, Governor Nathan Deal of Georgia spoke at a news conference to discuss the findings of the special investigation into alleged cheating on test scores in the Atlanta Public School system. I wrote about this issue in December, 2010, but since then, new information has come to light. Since I also think it is a pervasive problem in many of our organizations and institutions, and that the people who read this column are uniquely able to help managers understand the damage caused by this approach to management, I’m reprinting the December column with the updated information and a renewed sense of urgency. From December:

-----------------------------------------------------------------------------------------------------------

We’ve done it again! The federal government has set goals for our schools with neither plans nor resources to achieve them. And surprise, surprise—someone cheated to achieve the monetarily rewarded goals.

Excessive numbers of wrong-to-right answers were found on the Criterion-Referenced Competency Tests (CRCT) taken in 58 Atlanta schools. This test provides a key measure of whether schools are meeting standards (read “goals”) that are part of the federally mandated No Child Left Behind Act.

If the standards are met, schools can earn additional money, which principals can spend for almost anything they wish (Alan Judd and Bill Rankin, “Feds Probe Atlanta Schools,” The Atlanta Journal-Constitution). Some of the Atlanta schools may have cheated to get the money they need to improve their schools. I wonder if any other schools had similar misdeeds. Duh!

-----------------------------------------------------------------------------------------------------------

The investigation and other observations about this dilemma that have come to light since December:

  • Although some bloggers talk about a few bad apples causing the entire problem, the report cited by Deal revealed cheating in nearly half of the Atlanta District’s schools (Dorie Turner, “Atlanta Schools Extend Contract of Interim Chief.” 7 July 2011, accessed 17 July 2011). Gregory Cizek, a testing expert at the University of North Carolina, earlier stated the magnitude of the problem, saying that it “suggests a pattern of unethical behavior on the part of either kids or educators.” This pattern is perhaps related to the $2,000 cash bonuses offered to educators who originally met achievement goals and are now on the “severe concern” (for cheating) list.” It may alternately be the result of being required to offer extra tutoring and allowing parents to transfer their children to higher performing schools if they fail under No Child Left Behind. (Shaila Dewan, “Georgia Schools Inquiry Finds Signs of Cheating.” The New York Times. 12 February 2010). Although I am sure the carrots and sticks here are well intended, we all know carrots and sticks don’t always yield the intended results.
  • Dewan also noticed that, at a national level, we do not have a good sense of how much cheating is occurring. Cizek noticed that many states do not monitor erasure rates to check for potential cheating. Dewan also noticed that Ben Scafidi, the Director of the Center for an Educated Georgia, was concerned that the state did not flag enough schools, and that the districts were being asked to investigate themselves.

-----------------------------------------------------------------------------------------------------------

You may be tired of my continual reminders of Myron Tribus’ Perversity Principle (Myron Tribus and Yoshikazu Tsuda, “The Quality Imperative in the New Economic Era,” Quality First, (Alexandria, VA: National Institute for Engineering Management & Systems, 1992), but the violations of it are everywhere. One more time:

If you try to improve the performance of a system of people and machines by setting numerical goals and targets for their performance, the system will defeat you and you will pay a price where you did not expect it.

I guess no one in the federal government expected the Atlanta schools to cheat.

W. Edwards Deming of course, also advocated against goals and standards using slightly different language. In his 14 Points for Management, Deming advised managers to:

  • eliminate slogans, exhortations, and targets for the work force;
  • eliminate numerical quotas for the workforce and numerical goals for management

(Out of the Crisis, Cambridge MA: Massachusetts Institute of Technology, Center for Advanced Engineering Study, 1986). He further explained, in The New Economics (Cambridge, MA: Massachusetts Institute of Technology, Center for Advanced Engineering Study, 1994), how managers might pay a price for such goal setting.

Anybody can achieve almost any goal by:

  • Redefinition of terms;
  • Distortion and faking;
  • Running up costs.

Although Deming has provided several examples of how violation of these admonitions costs us, I want to add a few of my own here to encourage us all to think before we unconsciously set numerical goals for the people over whom we have influence.

My wife, Carole, and I ran into goal achievement through redefinition of goals, interestingly enough, in a regional attempt to reduce the dropout rate in Pennsylvania schools in the late 1980s. As the schools in the region came together to reduce dropout rates, they found that they all measured dropout rates using entirely different metrics based on the incentives and goals that existed in their own school districts. Needless to say, coming to a common set of metrics was impossible, and common progress was very difficult.

I first remember uncovering distortion and faking before I even heard of Deming or Tribus. As part of my engineering degree, I had to complete a project to predict the net production of production equipment at the General Motors manufacturing plant where I worked. My research yielded a procedure that required that such systems begin with the work standards (goal) established by the Industrial Engineering Department at each plant to each piece of equipment. Following the procedure, I started there and gathered the additional productivity data I thought I needed to complete my project.

When I attempted to verify my prediction system, it failed. As I investigated the reason for the failure, I found that the work standards were not scientifically developed as I had assumed, but were negotiated. Because the production workers, who need to be studied to establish the standards, did not trust either management or the engineers, they faked the production rates. Those rates varied so much from the rates calculated by the engineers based on standardized data, standards were negotiated, and therefore, nearly useless.

In my early days at Ford, I learned to run up costs as we ripped off reject tags so we could ship those parts in order to make our production goals. We made our goals and simply passed the costs along to our customers, who might pass their costs along to the next customer, or to our warranty folks who were in an entirely different area of the company…not our problem. I think Ford stopped doing that in the 80s. I hope so.

All this might lead you to think that Deming, Tribus, and I are advocating for the elimination of goals, objectives, standards, or whatever you want to call them. Although Deming may not have admitted it publicly, I think we all understood the value of properly developed goals and standards. Tribus, other colleagues, and I certainly had long conversations about it. My conclusion, based on those conversations, our visit to Deming Prize winning firms in Japan, the theory underlying participative management and employee involvement, and the advice of my old boss at General Motors, Jim Croghan, is that thoughtful goal setting involves:

  • Understanding your current performance using analytic statistical studies;
  • Co-developing the goals with the people doing the work and the other relevant stakeholders, while developing a plan to achieve the goals and planning for adequate resources to assure success.

Please forgive the length of my diatribe. I think this is an important issue and one that gets botched by well-meaning managers every day in every way possible. As always, I welcome your comments and questions. I’m at support@pqsystems.com.

 
PQ Systems  |  Proof of quality.


PQ Systems, Inc.  |  210 B East Spring Valley Road, Dayton, OH 45458  |  800-777-3020

Copyright 2011 PQ Systems, Inc. All rights reserved.