Appendix 1 to AMC1 FSTD(A).300  Validation test tolerances

(a)     Background

(1)     The tolerances listed in AMC1 FSTD(A).300 are designed to be a measure of quality of match using flight test data as a reference.

(2)     There are many reasons, however, why a particular test may not fully comply with the prescribed tolerances:

(i)      flight test is subject to many sources of potential error, e.g. instrumentation errors and atmospheric disturbance during data collection;

(ii)     data that exhibit rapid variation or noise may also be difficult to match; or

(iii)     engineering simulator data and other calculated data may exhibit errors due to a variety of potential differences discussed below.

(3)     When applying tolerances to any test, good engineering judgement should be applied. Where a test clearly falls outside the prescribed tolerance(s) for no apparent reason, then it should be judged to have failed.

(4)     When engineering simulator data are used, the basis for their use is that the reference data are produced using the same simulation models as used in the equivalent FSTD; i.e., the two sets of results should be ‘essentially’ similar. The use of flight test-based tolerances may undermine the basis for using engineering simulator data, because an essential match is needed to demonstrate proper implementation of the data package.

(5)     There are, of course, reasons why the results from the two sources can be expected to differ:

(i)      hardware (avionics units and flight controls);

(ii)     iteration rates;

(iii)     execution order;

(iv)     integration methods;

(v)      processor architecture;

(vi)     digital drift:

(A)     interpolation methods;

(B)     data handling differences; or

(C)     auto-test trim tolerances, etc.

(6)     Any differences should, however, be small and the reasons for any differences, other than those listed above, should be clearly explained.

(7)     Historically, engineering simulation data were used only to demonstrate compliance with certain extra modelling features:

(i)      flight test data could not reasonably be made available;

(ii)     data from engineering simulations made up only a small portion of the overall validation data set; or

(iii)     key areas were validated against flight test data.

(8)     The current rapid increase in the use and projected use of engineering simulation data is an important issue because:

(i)      flight test data are often not available due to sound technical reasons;

(ii)     alternative technical solutions are being advanced; and

(iii)     cost is an ever-present issue.

(9)     Guidelines are therefore needed for the application of tolerances to engineering-simulator-generated validation data.

(b)     Non-flight test tolerances

(1)     Where engineering simulator data or other non-flight test data are used as an allowable form of reference validation data for the objective tests listed in the table of validation tests, the match obtained between the reference data and the FSTD results should be very close. It is not possible to define a precise set of tolerances as the reasons for other than an exact match will vary depending upon a number of factors discussed in paragraph (a) of this Appendix.

(2)     As guidance, unless a rationale justifies a significant variation between the reference data and the FSTD results, 20% of the corresponding ‘flight test’ tolerances would be appropriate.

(3)     For this guideline (20% of flight test tolerances) to be applicable, the data provider should supply a well-documented mathematical model and testing procedure that enables an exact replication of their engineering simulation results.