Not engraved in stone:
Recalculating control limits
Like washing a car on Saturday morning, process improvement efforts can bring dramatic, sometimes immediate, improvements in healthcare settings by identifying waste or tracking inefficiencies. The real trick, though, is to sustain improvement over time and continue to improve processes incrementally, not reverting to old practices once improvement is apparent.
Data collection and analysis using control charts provides a key to this continuous improvement, and re-calculating limits on these control charts may offer a clear basis for process analysis. Recognizing the moment when control limits can be re-calculated is based not on arbitrary gesture or whim, but on understanding when a process has changed sufficiently to merit changing its limits.
The general rule for changing control limits involves identifying trends that may indicate a changed process. Chart interpretation is a matter of asking key questions and recognizing patterns. “Changing control limits means you really understand the process that’s being monitored,” says Matt Savage, technical support analyst for PQ Systems. While there are specific guidelines for interpreting control charts, the key lies in understanding that “something’s changed.”
A caveat about making changes in control limits must be reiterated, however. Simply eliminating out-of-control points, or arbitrarily changing limits, or taking steps that impede the information coming from the data itself all represent invalid approaches. The underlying principal is that control limits are recalculated only when it is clear that a process has changed.
Traditionally, signs of a changed process include a run of seven points in a row above or below the centerline, or going either up or down. If there has been a dramatic improvement in a process, control charts will indicate this to the informed observer.
An explanation of the meaning of changing control limits is illustrated by examples from healthcare providers. Both Cleveland Clinic and the Veterans’ Administration hospital in Memphis, have experience with recalculation of limits.
At the VA center, for example, the scheduling process for operating rooms was changed when electronic scheduling was instituted. To conform to principles of evidence-based medicine, data is collected relating to pre-op albumin counts for surgical patients. Liaison personnel entered the data before electronic scheduling had been introduced, resulting in a non-compliance rate of 12 percent, according to Anita Garrison, RN, Utilization Management Specialist for the large VA hospital center. When nurses began to enter data electronically and to manage the process, non-compliance dropped dramatically to one or two percent. “This was not expected,” Garrison says, adding that she monitored the change for six months (using p-charts) before changing control limits to reflect a change in process. The following chart reflects the dramatic change that ensued after electronic surgical scheduling had been implemented:
Figure 1: Before and after implementing surgical scheduling to check for missing albumin.
Another chart indicates data for missing albumin records after the control limits had been changed:
Figure 2: Surgical cases that are missing albumin.
In another case, Garrison cites a process change in length of stay for total knee and total hip replacement patients. She collected data on individuals charts for 2006 and January, 2007, realizing that a process that had changed in November demanded recalculation of control limits. There had been a dramatic drop in length of stay for these patients, and a run of data points below control limits reflected this drop. Chart annotation such as that seen in this chart helps to give instant analysis of the change.
Among the data frequently collected and analyzed in healthcare facilities throughout the nation is clean-case infection statistics for surgery patients. Last year, the Memphis VA facility began administering antibiotics prior to surgery in an effort to reduce the incidence of infection. Garrison says there was a definite trend downward in the data since she had begun to collect it in 2000. “With so many points below the mean, it was clear that we needed to change the process,” she says. These points are recognizable in the chart below, where an explanation of the before- and after-performance point is recorded on the chart itself:
Figure 3: Clean case infections in Surgical Service.
Located on a 33-acre campus, the Memphis Veterans Affairs Medical Center is a fully-accredited 254-bed tertiary care facility. It consists of a main campus and two off-site, VA-staffed primary care clinics. Nearly 20,000 veterans receive primary care and mental health services through the Memphis VA facilities. Memphis’ VA Center supports hundreds of active research projects with a total funding averaging $12 million annually. Its Memphis campus houses a state-of-the-art, 5-story patient bed tower that was dedicated in June 2000. The medical center is now in the final phase of a 3-phase seismic corrections building project, slated for completion in 2007.
Experts say that virtually everyone using statistical process control charts will eventually need to decide whether control limits should be recalculated or left alone. Savage suggests that if one is truly in doubt, the best course is generally to leave control limits alone until clarity about the process emerges. Since the purpose of control charts is to support understanding of processes in order to take the right action, though, it is important that the control limits accurately reflect expected behavior of the process. If the control limits no longer represent that behavior, the chart has lost its ability to support action.
Of course, simply recalculating control limits offers no guarantee that the new limits will accurately reflect expected behavior of a process, either. To improve their reliability, some guidelines are in order. Among the questions that will help one make the decision whether or not to recalculate control limits are the following:
- Has the process changed significantly; and is there an assignable cause for that change that can be identified?
- Do you understand the cause for the change in the process?
- Do you have reason to believe that the cause or condition will remain in the process?
- Have you observed the changed process long enough to determine if newly-recalculated limits will appropriately reflect the behavior of the process?
If the control charts are being used as a part of a larger improvement effort—as they should certainly be—causes for changes in processes are generally easy to identify. They may include increases or decreases in staffing, for example, or more frequent sampling of data, a change in equipment or technology, or even a power outage.
The experience of Cleveland Clinic, named as one of the nation’s top three hospitals in U.S. News & World Report’s annual healthcare survey, provides additional insight about recalculating control limits. Located in Cleveland, Ohio, Cleveland Clinic is a not-for-profit, multispecialty academic medical center that integrates clinical and hospital care with research and education. Today, Cleveland Clinic is one of the largest and most respected hospitals in the nation. In addition to its top-three designation, 11 of its specialties were ranked among the top 10 in their areas in the US News survey, and the Clinic’s heart services earned the top award for the twelfth consecutive year. As the 2001 and 2003 Codman Award winner for its effective use of performance measurement to improve quality and safety in health care, the Cleveland Clinic system clearly understands data-driven decision making. Using control charts and other analytical and statistical tools has become routine practice for the 1000-bed healthcare system.
Eric Hixson, Assistant Director for Data, Quality and Patient Safety Institute, points out that knowing when to change control limits depends on “intimate knowledge of the process under observation.” Since the control limit is a visual aid that represents “the voice of the process,” accurate control limits are the first step in preventing two of the most common errors associated with analysis:
- treating common-cause variation as if it were special cause (Type I, Alpha errors), or
- treating special cause variation as though it were common cause (Type II, Beta errors).
“The value of control charts lies, in part, in the ability to examine patterns of process behavior over time,” says Mike Crossen (Program Manager, Quality and Patient Safety Institute). So the danger in changing control limits is that the pattern exhibited by a recent out-of-control run may indeed be special cause; but it may not be desirable or sustainable.
In a simple scenario, a known and intentional change is made to the process (i.e., an experiment); the control limits are reset; and the process is observed and permitted to stabilize under the new conditions. Analysis of the new process distribution, mean, and general behavior will help one to conclude whether the process has been changed by the intervention.
Another scenario suggested by Hixson and Crossen involves an unintentional, but known change to a process. Awareness of this change usually occurs after observing its effects and investigating causes. A change in the staffing level or skill mix, or introduction of a new technology may alter a process directly or indirectly, and the change may be either positive or negative. A decision must be made whether to bring the process back in line with its prior state, or to change control limits to reflect its new behavior.
Yet another scenario illustrates the complexity of the decision to recalculate control limits. This one involves the combination of multiple factors, both known and unknown, that change the performance of a process. An improvement project may be launched that involves a variety of efforts, including increased awareness, focused training, measurement and results reporting, and changes to tools or equipment.
Crossen points out that unlike an experiment, the object is not to differentiate and attribute the individual effects of each component on the process but rather to observe their combined effects on overall performance. In complex processes, there is an increased likelihood that unknown factors can also influence the process. Participants may modify related processes beyond the scope of the improvement project which indirectly influence its performance but go uncommunicated. “But as in the case of an experiment, once the process stabilizes under the new set of conditions, conclusions can be drawn about whether to change control limits to reflect the new state,” he says.
Understanding recalculation of control limits, to summarize the experience of these two large healthcare organizations, depends on knowing that contol limit recalculation is not something to be done casually or without clear insight, but instead depends on “intimate knowledge” of the process.
So, while bringing about a significant improvement to a process may seem as dramatic as Saturday’s carwash, the improvement in this case relies on complex skills and understandings of processes, including knowledge about re-calculating control limits. Without these skills and understandings, you may find that you’ve washed your car using the wrong tools or equipment.
Or that you’re driving it right back on that muddy road you’ve come from.
2007 PQ Systems.
Please direct questions or problems regarding this web site to the Webmaster.