MSA: Analysis of data from measurement system

By Jackie Graham, Ph.D.

This article continues our discussion of measurement systems analysis. Previously we looked at setting up a measurement study and initial analysis calculating R&R (repeatability and reproducibility). Last time we looked at an assessment of the variation in appraisers' results. This time we will continue to discuss analysis of data generated from a measurement study.

The same example data will be used to illustrate more analysis techniques. In this study, 5 samples were measured for length, using 4 appraisers. Each sample was measured twice. The results are shown below.

Sample 1

Sample 2

Sample 3

Sample 4

Sample 5

Appraiser 1

9.0

9.0

9.7

10.1

10.0

9.1

9.3

9.3

10.3

10.4

Appraiser 2

9.6

9.1

9.0

10.3

10.5

9.6

10.1

10.2

10.8

10.6

Appraiser 3

9.5

9.7

9.9

10.0

9.8

9.6

10.0

9.9

10.1

9.4

Appraiser 4

9.1

8.2

9.3

9.1

9.5

8.8

9.3

9.7

10.3

9.0

The R&R (repeatability and reproducibility) results are shown below.

Value

% of specification

Equipment variation EV

1.96

65.44

Appraiser variation AV (appraiser)

1.61

53.62

Repeatability and reproducibility R&R

2.54

84.60

The R&R% should be no more than 30%, making the R&R in the example study of 84.6% unacceptable. Last time we started to track down the causes of this unacceptable variation, by completing an average and range chart, see chart 1.

Text Box: Chart 1
Chart 1

Chart 1 was constructed in exactly the same way as any other average and range chart. However, the interpretation is quite different. Last time, we focused on the range chart, although we detected differences in the appraisers’ ranges, none of these was statistically significant. This time we will take a closer look at the average chart.

The average chart is constructed by taking each of the appraisers in turn and averaging the results for the first sample, second sample, etc. Remember, each appraiser measured each sample twice. The average control limits are calculated in the normal way. That is:

Upper control limit = UCL = + A 2 2

Lower control limit = LCL = - A 2 2

A 2 is found in statistical tables and is based on the number of data points in the range calculation, in our example 2.

Note the average range comes from the bottom chart. When conducting a measurement study, we like to see little variation in the results for each sample, and the ranges as small as possible, ideally zero. It follows that when the average range is calculated, we want this to be as small as possible too. What impact does this have on the average chart? In fact, it has considerable impact: if the range is small, then A 2 2 will also be small. This means the control limits will be very narrow. In practice, the narrower the control limits the better the measurement system.

So, when we interpret an average chart in a measurement study, we actually look for a very different picture from that of a normal control chart. We want to see narrow limits. This means that the averages are likely to be outside the limits. In fact, we would like to see as many of the samples outside the limits as possible! This is the complete opposite of the normal interpretation rules for an average chart. An example of an excellent average chart in a measurement study is shown in chart 2.

Text Box: Chart 2
Chart 2

Note the narrowness of the limits in comparison to the averages. This indicates that the variation in the measurement system is small in comparison to the variation in the samples. As long as the samples have been correctly chosen, this is a sign of a good measurement system.

Why is it so important that the samples are correctly chosen?

In the example shown in chart 2, the samples vary from 40 to 160 (120 units) and the specification width is only 20 units. The specification width is small relative to the sample width. When this occurs, although the average chart will look great, when the R&R rate is calculated, it will be poor.

Similarly, if samples are chosen that are very close to one another--say all in a band between 20 and 30 (10 units)--and the specification is 0 to 100 (100 units), the samples will look close and the chart will be difficult to interpret. The selection of appropriate samples in an R&R study cannot be overstated. It is critical to the success of the study, particularly for ease of analysis. Select the samples correctly and the ability to interpret the data increases substantially.

Referring to the example in Chart 1, it can be seen that the control limits on the average chart are wide in comparison to the samples. The variation in the samples is approximately 9 to 10.5 or 1.5 units. The specification width is 3 units wide. The fact that a narrow group of samples was selected does not help this analysis. Even so, the variation in the samples appears to be overwhelmed by the variation in the measurement system, as depicted by the control limits.

Going back to the example shown in chart 2, it is evident from the average chart that the three appraisers have similar results. The shape of each appraiser’s results in the average chart looks identical--an excellent result. These average results can be plotted over the top of each other to emphasize the similarity or difference, depending on the data being studied. See chart 3.

Text Box: Chart 3
Chart 3

Overlaying each appraiser’s results emphasises the similarity in the results--an ideal outcome.

Returning to the example in chart 1, the averages have a somewhat similar shape, but appear to be set at different levels. When they are plotted over each other, chart 4 is the result.

Text Box: Chart 4
Chart 4

Note that the results for appraisers 1 and 2 are somewhat similar. In comparison to other appraisers, appraiser 4 seems low and appears to have a bias. Deciding whether appraiser 4 has a bias that is statistically significant will be the subject of the next article.

All the charts produced in this article were produced using GAGEpack from PQ Systems.

Originally published in the June 2008 edition of Quality eLine, our free monthly newletter.

Subscribe now!