Six simple steps to sound sampling

In a traditional cartoon with two people staring at a now-empty box of chocolates, one of the characters, looking a little sickly, responds to the other with the comment, “I was just sampling for quality.”

Conventional wisdom about sampling insists that enough samples must be taken to assure an accurate picture of the whole. Most would agree with this approach, though determining how much is “enough” may be open to question. Clearly, sampling every single product, whether it is bread baked in a bakery or switches for floor lamps, is expensive and sometimes highly destructive. To assure the quality of wooden matches, for example, one would not want to strike every single match to be sure that it would light.

Determining sample size and sample frequency is key to the accuracy of the sampling process and plays a key role in understanding processes and their improvement. Six steps will assure that the sampling process is one of integrity that will produce consistent results.

Step 1: Determine the question that you want the data to answer.

Collecting data for its own sake, as we know, is a waste of time and money. Instead, you must define exactly what you really want to know from the data. Data related to errors on medical bills, for example:

Errors on medical bills per subgroup of five bills/day for each clerk

Billing clerk    M T W Th F M T W Th F
A 3 4 3 6 9 2 5 3 4 7
B 5 8 4 7 12 6 4 9 3 11
C 3 2 1 3 5 1 2 1 2 7
Total 11 14 8 16 26 9 11 13 9 25

What questions might this data raise? They may include:

Are there differences among daily results?
How does each clerk’s rate vary within the time period?

For the first question, you would want to look at the daily totals for the three clerks, and plot one point per day (Figure 1).

[Figure 1]

For the answer to the second question, you would plot the daily error rate for each clerk, in order to identify patterns in the two-week period (Figure 2 indicates the pattern for clerk B, for example). For each of the three clerks, the error rate appears to go up on Fridays—some more than others.  This corroborates the results of charting performance for all days, all clerks in Figure 1, and thereby generates information that will be useful in advancing improvement efforts. (The next tool to use might be a cause-and-effect diagram, to get to the root cause for greater numbers of errors on Fridays; thus sampling sets the stage for deeper analysis.)

Step 2: Determine the frequency of sampling (how often samples will be taken).

In deciding how often to take samples, a rule of thumb is to sample each time a process is likely to change. Sampling should occur at least as often as the most-frequently-occurring factor in the change process. If a process exhibited regular up-and-down behavior as indicated in the graph below (Figure 2), it would be misleading to sample only at the high points of the process. Instead, sampling should represent not only the peaks, but the valleys and the times between peaks and valleys as well.

[Figure 2]

Factors that represent changes in the process include those related to personnel, equipment, or materials. Frequencies might be expressed as time--daily, weekly, monthly, etc.—or in terms of numbers, such as “every tenth order,” “every fifth invoice,” etc. If changes are infrequent, sampling can reflect this pattern. Samples might be taken only every few months for a process that remains essentially the same.

Step 3: Determine the actual sampling times.

After the frequency of sampling has been determined, it is important to develop consistency in the timing of this sampling. If it is to be done daily, a specific time for sampling should be noted. Preferably, this time should be set as close to any expected changes—a shift change, for example—as possible. If it is to be done monthly, this must also be defined in specific terms: Every third Tuesday at 2 p.m., for example.

Step 4: Select the subgroup size if you are measuring variable data.

The subgroup size is designated as “n.” The terms “subgroup” and “sample” are sometimes used interchangeably, referring to the number of items chosen to be examined at the same time. Choosing subgroup sizes is different for variable data from that of attribute data. Remember, variable data is that which is measured, such as height, weight, time, length, or dollars. In healthcare, time is often the measurement of interest, for example.

When measuring variable data, a subgroup size larger than one is preferable, in order to provide more possibilities for analysis. Sometimes, however, this is not possible (for example, average ICU bed days per month, or cost per member per month). In this case, the subgroup (or sample) is equal to one. If a larger size subgroup can be chosen, the size is usually between three and eight, a range that has been determined to be statistically efficient. Most commonly-used size is five. If more data is needed, the frequency should be increased, rather than the subgroup size.

Step 5: Select the subgroup size if you are measuring attribute data.

Attribute data is that which can be counted, such as number of patients, number of errors per invoice, number of missed appointments, or number of complaints. The subgroup size for attribute data depends on the process to be sampled. The general rule is to gather a large enough sample each time so that all possible characteristics that are to be examined will show up.

Step 6: Share the sampling plan.

Output from each of the decisions made in the above steps should be recorded in a systematic way. A data gathering form (a variation of check sheet) is useful for this process, with columns indicating relevant details that might include which data, how it is collected, how much, how often, where, and who. Rows will indicate various measures and their operational definitions.

Sampling helps to guide the study of a system by providing quantitative data, supporting a reduction  in cost and an improvement in accuracy. Decisions about frequency and size of sampling depend on the specific process itself, but many processes will benefit from the steps outlined here.

If you’re tempted to do 100 percent inspection of chocolates, remember these steps and utilize selective tasting instead.

Or go ahead—buy a box of chocolates for yourself, if you need to test every variety. Just explain your total consumption in terms of 100 percent sampling rather than an over-active appetite.

Adapted from Practical Tools for Healthcare Quality, ed. Sandra K. Murray and O. Byron Murray. Dayton: PQ Systems, Inc., 1997.

Originally published in the April 2010 edition of Quality eLine, our free monthly newletter.

Subscribe now!