In this section we discuss the following scenario: You have performed the calibration, you have the results on a certificate, and you have compared the results to the tolerance limits. It’s time to pop the big questions: Is it a passed or failed calibration? Is it in or out of tolerance?

Compliance statement – pass or fail

Typically, when you calibrate an instrument you have predefined tolerance limits that the instrument has to meet. Sure, you may perform some calibrations without tolerance limits, but in process industries the tolerance levels are typically set in advance.

Tolerance levels indicate how much the result can differ from the true value. If the errors of the calibration result are within the tolerance limits, it is a passed calibration, and if some of the errors are outside of tolerance limits, it is a failed calibration. This sounds simple, like basic mathematics. How hard can it be?

It is important to remember that it is not enough to just take the error into account; you must also take the total uncertainty of the calibration into account!

Taking the uncertainty into account turns this into a whole different ball game. As discussed in the white paper, there are many sources of uncertainty. Let’s go through some examples next.

Example #1 - reference with too big uncertainty

Let’s say the process transmitter you are about to calibrate has a tolerance level of ±0.5% of its measurement range.

During the calibration you find out that the biggest error is 0.4%, so this sounds like a pass calibration, right?

But what if the calibrator that was used has an uncertainty specification of ±0.2%? Then the 0.4% result could be either a pass or a fail – it is impossible to know which.

Plus, in any calibration you also have uncertainty caused by many other sources, like the standard deviation of the result, repeatability, the calibration process, environmental conditions, etc.

When you estimate the effect of all these uncertainty components, it is even more likely that the calibration was a fail after all, even though it looked like a pass at first.

Example #2 - different cases

Let’s look at a graphical illustration of the next example to make this easier to understand. In the picture below there are four calibration points taken, the diamond shape indicating the actual calibration result. The line above and below the result indicates the total uncertainty for each calibration point. The black horizontal line marks the tolerance limit.

We can interpret the different cases shown above as follows:

• Case 1: This is clearly within the tolerance limits, even when uncertainty is taken into account. So we can state this as a pass.

• Case 4: This is also a clear case. The result is outside of the tolerance limits, even before uncertainty is taken into account, so this is clearly a fail.

• Cases 2 and 3: These are a bit more difficult to judge. It seems that in case 2 the result is within the tolerance limit while in case 3 it is outside, especially if you don’t care about the uncertainty. But taking the uncertainty into account, we can’t really say this with confidence.

There are regulations (for example, ILAC G8:1996 – Guidelines on Assessment and Reporting of Compliance with Specification; EURACHEM / CITAC Guide: Use of uncertainty information in compliance assessment, First Edition 2007) for how to state the compliance of a calibration.

These guides suggest that a result should only be considered a pass when the error plus the uncertainty is less than the tolerance limit.

They also suggest that a result should only be considered a fail when the error with the uncertainty added or subtracted is greater than the tolerance limit.

When the result is closer to the tolerance limit than half of the uncertainty, they suggest it should be called an “undefined” situation, i.e. you should not state the result as a pass or fail.

We have seen many people interpreting the uncertainty and pass/fail decision in many different ways over the years. In practice, uncertainty is often not taken into account in the pass/fail decision-making process, but it is nonetheless very important to be aware of the uncertainty when making a decision.

Example #3 – different uncertainties

Another situation to illustrate is when the total uncertainty is not always the same.

Cases 1 and 2 have about the same measurement result , so without uncertainty we would consider these as the same level measurements.

But when the uncertainty is taken into account, we can see that case 1 is really terrible because the uncertainty is simply too large to be used for this measurement with the given tolerance limits.

Looking at cases 3 and 4 it seems that case 3 is better, but with uncertainty we can see that it is not good enough for a pass statement, while case 4 is.

Again, I want to point out that we need to know the uncertainty before we can judge a measurement result. Without the uncertainty calculation, cases 1 and 2 look similar; with uncertainty taken into account they are very different.

TUR / TAR ratio vs. uncertainty calculation

TUR (test uncertainty ratio) and TAR (test accuracy ratio) are often mentioned in various publications. Some publications even suggest that with a large enough TUR/TAR ratio there is no need to worry about uncertainty estimation / calculation.

A commonly used TAR ratio is 4:1. In short this means that if you want to calibrate a 1% instrument, your test equipment should be four times more accurate, i.e., it should have an accuracy of 0.25% or better.

Some guides/publications also have recommendations for the ratio. Most often the ratio is used as in the above example, i.e., to compare the specifications of the device under test (DUT) and the manufacturer’s specifications of the reference standard.

But in that scenario you only consider the reference standard (test equipment, calibrator) specifications and you neglect all other related uncertainties.

While this may be “good enough” for some calibrations, this system does not take all uncertainty sources into account.

For an accurate result it is highly recommended that you perform the uncertainty evaluation/calculation, taking into account the whole calibration process.

A question we are asked regularly is “How many times more accurate should the calibrator be compared to the device to be calibrated?”. While some suggestions could be given, there isn’t really any correct answer to this question. Instead, you should be aware of the total uncertainty of your calibrations. And of course, it should reflect your needs! 

Summary & the key takeaways from the white paper

To learn more about this subject, please download and read the white paper linked in this post.

Here is a short list of the key takeaways: