Vol. 10, No. 2

February 2008

PQ Systems
 
Contents

How to choose calibration software

Quality Quiz: With a video!

Data in everyday life

MSA with Jackie Graham

Bytes and pieces

FYI: Current releases

 

Send Quality eLine
to a friend!

Just type in your friend's email below:

 

Sign up
If you received this newsletter from a friend and want your own subscription to Quality eLine, click below.

Subscribe to Quality eLine

 
Software

 

   

Jackie GrahamMSA with Jackie Graham:
Analyzing data from a measurement system

By Jackie Graham, Ph.D.
Managing Director, PQ Systems, Australia

Last time we discussed how to go about setting up a statistical assessment of a measurement system, and the initial analysis of the data, calculating R&R (repeatability and reproducibility). This time we will continue to discuss analysis of data generated from a measurement study.

The same example data will be used to illustrate some more analysis techniques. In this study, 5 samples were measured for length, using 4 testers. Each sample was measured twice. The results are shown below.

 

Sample 1

Sample 2

Sample 3

Sample 4

Sample 5

Tester 1

9.0

9.0

9.7

10.1

10.0

 

9.1

9.3

9.3

10.3

10.4

Tester 2

9.6

9.1

9.0

10.3

10.5

 

9.6

10.1

10.2

10.8

10.6

Tester 3

9.5

9.7

9.9

10.0

9.8

 

9.6

10.0

9.9

10.1

9.4

Tester 4

9.1

8.2

9.3

9.1

9.5

 

8.8

9.3

9.7

10.3

9.0

Last time we discussed R&R (repeatability and reproducibility), the results are shown below.

 

Value

% of specification

Equipment variation EV

1.96

65.44

Appraiser variation AV (tester)

1.61

53.62

Repeatability and reproducibility R&R

2.54

84.60

In this case, the measurement values were compared to the specification of the product being tested. The specification for the product is 8 to 11, making a specification width of 3. We learned last time that the R&R% should be no more than 30%, making the R&R in the example study of 84.6% unacceptable.

So, how do we start to track down the issues? Well, one simple technique we can use is an average and range chart. For this analysis you can complete the control chart by hand or use GAGEpack EZ, SQCpack EZ, or CHARTrunner.

Simply take the data from the table above and calculate the average and range result for each tester and for each sample, as shown in the table that follows.

 

Sample 1

Sample 2

Sample 3

Sample 4

Sample 5

Tester 1

9

9

9.7

10.1

10

 

9.1

9.3

9.3

10.3

10.4

Average

9.05

9.15

9.5

10.2

10.2

Range

0.1

0.3

0.4

0.2

0.4

Tester 2

9.6

9.1

9

10.3

10.5

 

9.6

10.1

10.2

10.8

10.6

Average

9.6

9.6

9.6

10.55

10.55

Range

0

1

1.2

0.5

0.1

Tester 3

9.5

9.7

9.9

10

9.8

 

9.6

10

9.9

10.1

9.4

Average

9.55

9.85

9.9

10.05

9.6

Range

0.1

0.3

0

0.1

0.4

Tester 4

9.1

8.2

9.3

9.1

9.5

 

8.8

9.3

9.7

10.3

9

Average

8.95

8.75

9.5

9.7

9.25

Range

0.3

1.1

0.4

1.2

0.5

Next, place the averages and ranges on to a normal average and range control chart in the following order, tester 1, samples 1 to 5, then tester 2, samples 1 to 5, etc. The average and range chart for this data follows, it was produced using GAGEpack EZ.

This chart is constructed in exactly the same way as any other average and range chart. However, the interpretation is quite different.

Start by examining the range chart. The first range has a value of 0.1. This range represents the difference in values between the two measurements taken by tester 1, of sample 1. This is obviously different than a normal range control chart. So, in this case the range is the difference between the first measurement of the sample and the second measurement of the same sample. Ideally, all the ranges should be zero. In other words, exactly the same value is achieved for both measurements of each sample. If you ever see one of these, you are lucky, they are extremely rare!

The first aspect we look at in the range chart is whether all the points are inside the upper control limit. In this case, it is apparent that all the data points are inside the upper control limit – this is good. Any points outside the control limit in the range chart require urgent investigation, as they indicate the measurement system is unpredictable and prone to large errors, obviously a reason for major concern.

In the example chart, the ranges are all inside the upper control limit, however, the ranges for each tester look different. Note that testers 1 and 3 have all their ranges below the average range. This would tend to indicate that these testers have a lower test-retest error, in other words they appear to be more consistent. Now, we have to be careful that we don’t jump to the wrong conclusion. Although testers 1 and 3 appear to be more consistent than 2 and 4, we have only a small amount of data. It could be that the difference is not statistically significant. How can we tell?

The answer is an analysis called an operator inconsistency chart. This chart tells us if the differences between the testers are statistically significant. The chart takes the average range for each of the testers, and compares them. First we need to calculate the average ranges for each tester, using the data from the previous table.

Tester 1.

Ranges were: 0.1, 0.3, 0.4, 0.2, and 0.4.

Averaging these ranges gives an average range of 0.28.

Tester 2.

Ranges were: 0, 1, 1.2, 0.5, and 0.1.

Averaging these ranges gives an average range of 0.56.

Tester 3.

Ranges were 0.1, 0.3, 0, 0.1, and 0.4.

Averaging these ranges gives an average range of 0.18.

Tester 4.

Ranges were 0.3, 1.1, 0.4, 1.2, and 0.5.

Averaging these ranges gives an average range of 0.7.

We can see differences in the average ranges from 0.18 to 0.7. These average ranges are now plotted onto an operator inconsistency chart, produced using GAGEpack EZ, as shown below.

Note the points are plotted in a similar way to any normal chart and special control limits are calculated. If any of the points lie outside these limits it indicates that the testers have a significant consistency issue relative to each other. If a point lies below the lower limit (these are rare) it indicates that the tester involved is statistically more consistent than the other testers. This would be excellent news, and a great source of improvement! Simply find out why the tester is more consistent, and use that tester’s methodology as the normal test method. If, however, a point lies above the upper limit, it indicates the tester is statistically more inconsistent than the other testers. This requires an individual assessment of the tester to establish why he or she is more inconsistent, and retraining to remove the cause of the inconsistency. This is a common occurrence that is not restricted to inexperienced testers. This may come as a surprise, but it is generally caused by taking short cuts, or making small changes to procedures, that the tester believes improves the test, when in fact it does not.

In the example, all the average ranges are inside the limits indicating that, although differences can be seen, they are small in comparison to the overall variation found in the measurement system.

Inconsistency is the most important aspect of measurement analysis and often the most difficult to improve, particularly when all the testers have a similar inconsistency, as in the example.

Next time we will look at the average chart, and how to establish if there is a bias among testers.

 

Copyright 2008 PQ Systems.
Please direct questions or problems regarding this web site to the Webmaster.