Appendix 1 to AMC 20-6B – Propulsion system reliability assessment

1.         ASSESSMENT PROCESS

To establish, by utilising service experience, whether a particular airframe/engine combination has satisfied the propulsion systems reliability requirements for ETOPS, an engineering assessment will be made by EASA, using all pertinent propulsion system data. To accomplish the assessment, EASA will need world fleet data (where available), and data from various sources (the operator, the engine and aeroplane (S)TC holder) which should be extensive enough and of sufficient maturity to enable EASA to assess with a high level of confidence, using engineering and operational judgement and standard statistical methods where appropriate, that the risk of total power loss from independent causes is sufficiently low. EASA will state whether or not the current propulsion system reliability of a particular airframe/engine combination satisfies the relevant criteria. Included in the statement, if the operation is approved, will be the engine build standard, propulsion system configuration, operating condition and limitations required to qualify the propulsion system as suitable for ETOPS.

Alternatively, where type design approval for Early ETOPS is sought at entry into service, the engineering assessment can be based on substantiation by analysis, test, in-service experience or other means, to show that the propulsion system will minimise failures and malfunctions and will achieve an IFSD rate that is compatible with the specified safety target associated with total loss of thrust.

If an approved engine CMP is maintained by the responsible engine authority and is duly referenced on the engine Type Certificate Data Sheet or STC, then this shall be made available to EASA conducting the aeroplane propulsion system reliability assessment. Such a CMP shall be produced taking into account all the requirements of Chapter II and should be incorporated or referenced in the aeroplane CMP.

2.         RELIABILITY VALIDATION METHODS

There are two extremes in the ETOPS process with respect to maturity; one is the demonstration of stable reliability by the accumulation of in-service experience and the other is by a programme of design, test and analysis, agreed between the (S)TC holders and EASA. The extent to which a propulsion system is a derivative of previous propulsion systems used on an ETOPS approved aeroplane is also a factor of the level of maturity. When considering the acceptability of a propulsion system, maturity should be assessed not only in terms of total fleet hours but also taking account of fleet leader time over a calendar time and the extent to which test data and design experience can be used as an alternative.

a.         Service Experience

There is justification for the view that modern propulsion systems achieve a stable reliability level by 100 000 engine hours for new types and 50 000 engine hours for derivatives. 3 000 to 4 000 engine hours are considered to be the necessary time in service for a specific unit to indicate problem areas.

Normally, the in-service experience will be:

(1)       For new propulsion systems: 100 000 engine hours and 12 months service. Where experience on another aeroplane is applicable, a significant portion of the 100 000 engine hours should normally be obtained on the candidate aeroplane.

On a case-by-case basis, relevant test and design experience, and maximum diversion time requested, could be taken into account when arriving at the in‑service experience required.

(2)       For derivative propulsion systems: 50 000 engine hours and 12 months service. These values may vary according to the degree of commonality. To this end, in determining the derivative status of a propulsion system, consideration should be given to technical criteria referring to the commonality with the previous propulsion system used on an ETOPS approved aeroplane. Prime areas of concern include:

(i)        Turbomachinery;

(ii)       Controls and accessories and control logic;

(iii)      Configuration hardware (piping, cables, etc.);

(iv)      Aeroplane to engine interfaces and interaction:

(A)      Fire;

(B)       Thrust reverser;

(C)       Avionics;

(D)      etc.

The extent to which the in-service experience might be reduced would depend upon the degree of commonality with the previous propulsion system used on an ETOPS approved aeroplane using the above criteria and would be decided on a case-by-case basis.

Also on a case-by-case basis, relevant test and design experience and maximum diversion time requested could be taken into account when arriving at the in-service experience required.

Thus, the required experience to demonstrate propulsion system reliability should be determined by:

(i)        The extent to which previous service experience with a common propulsion system used on an ETOPS approved aeroplane system can be considered;

(ii)       The extent to which compensating factors, such as design similarity and test evidence, can be used.

The two preceding considerations would then determine the amount of service experience needed for a particular propulsion system proposed for ETOPS.

These considerations would be made on a case-by-case basis and would need to provide a demonstrated level of propulsion system reliability in terms of IFSD rate. See paragraph 3 ‘Risk Management and Risk Model’.

(3)       Data required for the assessment

(i)        A list of all engine shutdown events for all causes (excluding normal training events). The list should provide the following for each event:

(A)      date;

(B)       airline;

(C)       aeroplane and engine identification (model and serial number);

(D)      power-unit configuration and modification history;

(E)       engine position;

(F)       symptoms leading up to the event, phase of flight or ground operation;

(G)      weather/environmental conditions and reason for shutdown and any comment regarding engine restart potential;

(ii)       All occurrences where the intended thrust level was not achieved, or where crew action was taken to reduce thrust below the normal level (for whatever reason);

(iii)      Unscheduled engine removals/shop visit rates;

(iv)      Total engine hours and aeroplane cycles;

(v)       All events should be considered to determine their effects on ETOPS operations;

(vi)      Additional data as required;

(vii)     EASA will also consider relevant design and test data.

b.         Early ETOPS

(1)       Acceptable early ETOPS certification plan

Where type design approval for Early ETOPS is sought at the first entry into service, the engineering assessment can be based on substantiation by analysis, test, in‑service experience, CS-E 1040 compliance or other means to show that the propulsion system will minimise failures and malfunctions, and will achieve an IFSD rate that is compatible with the specified safety target associated with catastrophic loss of thrust. An approval plan, defining the early ETOPS reliability validation tests and processes, must be submitted by the applicant to EASA for agreement. This plan must be implemented and completed to the satisfaction of EASA before an ETOPS type design approval will be granted for a propulsion system.

(2)       Propulsion system validation test

The propulsion system for which approval is being sought should be tested in accordance with the following schedule. The propulsion system for this test should be configured with the aeroplane installation nacelle and engine build-up hardware representative of the type certificate standards.

Tests of simulated ETOPS service operation and vibration endurance should consist of 3 000 representative service start-stop cycles (take-off, climb, cruise, descent, approach, landing and thrust reverse), plus three simulated diversions at maximum continuous thrust for the maximum approved diversion time for which ETOPS eligibility is sought. These diversions are to be approximately evenly distributed over the cyclic duration of the test, with the last diversion to be conducted within 100 cycles of the completion of the test.

This test must be run with the high speed and low speed main engine rotors unbalanced to generate at least 90 % of the applicant’s recommended maintenance vibration levels. Additionally, for engines with three main engine rotors, the intermediate speed rotor must be unbalanced to generate at least 90 % of the applicant’s recommended acceptance vibration level. The vibration level shall be defined as the peak level seen during a slow acceleration/deceleration of the engine across the operating speed range. Conduct the vibration survey at periodic intervals throughout the 3000-cycle test. The average value of the peak vibration level observed in the vibration surveys must meet the 90 % minimum requirement. Minor adjustments in the rotor unbalance (up or down) may be necessary as the test progresses in order to meet the required average vibration level requirement. Alternatively, to a method acceptable to EASA, an applicant may modify their test to accommodate a vibration level marginally less than 90 % or greater than 100 % of the vibration level required in lieu of adjusting rotor unbalance as the test progresses.

Each one hertz (60 rpm) bandwidth of the high-speed rotor service start-stop cycle speed range (take-off, climb, cruise, descent, approach, landing and thrust reverse) must be subjected to 3x10⁶ vibration cycles. An applicant may conduct the test in any rotor speed step increment up to 200 rpm as long as the service start-stop cycle speed range is covered. For a 200-rpm step, the corresponding vibration cycle count is to be 10 million cycles. In addition, each one hertz bandwidth of the high‑speed rotor transient operational speed range between flight idle and cruise must be subjected to 3x10⁵ vibration cycles. An applicant may conduct the test in any rotor speed step increment up to 200rpm as long as the transient service speed range is covered. For a 200-rpm step, the corresponding vibration cycle count is to be 1 million cycles.

At the conclusion of the test, the propulsion system must be:

(i)        Visually inspected according to the applicant’s on-wing inspection recommendations and limits.

(ii)       Completely disassembled and the propulsion system hardware must be inspected in accordance with the service limits submitted in compliance with relevant instructions for continued airworthiness. Any potential sources of in-flight shutdown, loss of thrust control, or other power loss encountered during this inspection must be tracked and resolved in accordance with paragraph 5 of this Appendix 1.

3.         RISK MANAGEMENT AND RISK MODEL

Propulsion systems approved for ETOPS must be sufficiently reliable to assure that defined safety targets are achieved.

a.         For ETOPS with a maximum approved diversion time of 180 minutes or less

An early review of information for modern fixed-wing jet-powered aircraft shows that the rate of fatal accidents for all causes is in the order of 0·3 x 10^-6 per flying hour. The reliability of aeroplane types approved for extended-range operation should be such that they achieve at least as good an accident record as equivalent technology equipment. The overall target of 0 ·3 x 10^-6 per flying hour has therefore been chosen as the safety target for ETOPS approvals up to 180 minutes.

When considering safety targets, an accepted practice is to allocate appropriate portions of the total to the various potential contributing factors. By applying this practice to the overall target of 0·3 x 10^-6 per flying hour, in the proportions previously considered appropriate, the probability of a catastrophic accident due to complete loss of thrust from independent causes must be no worse than 0·3 x 10^-8 per flying hour.

Propulsion system related accidents may result from independent cause events but, based on historical evidence, result primarily from events such as uncontained engine failure events, common cause events, engine failure plus crew error events, human error related events and other. The majority of these factors are not specifically exclusive to ETOPS.

Using an expression developed by ICAO (ref. AN-WP/5593 dated 15/2/84), for the calculation of engine in-flight shutdown rate, together with the above safety objective and accident statistics, a relationship between target engine in-flight shutdown rate for all independent causes and maximum diversion time has been derived. This is shown in Figure 1.

In order that type design approval may be granted for extended operation range, it will be necessary to satisfy EASA that after application of the corrective actions identified during the engineering assessment (see Appendix 1, Section 4: ENGINEERING ASSESSMENT. CRITERIA FOR ACCEPTABLE RELIABILITY VALIDATION METHODS), the target engine in-flight shutdown rates will be achieved. This will provide assurance that the probability objective for loss of all thrust due to independent causes will be met.

b.         For ETOPS with a maximum approved diversion time of longer than 180 minutes

The propulsion systems IFSD rate target should be compatible with the objective that the catastrophic loss of thrust from independent causes is no worse than extremely improbable, based on maximum ETOPS flight duration and maximum ETOPS rule time.

For ETOPS with Maximum Approved Diversion Times longer than 180 minutes, to meet this objective; the powerplant installations must comply with the safety objectives of CS 25.1309; the goal should be that the catastrophic loss of thrust from independent causes should be extremely improbable (see AMC 25.1309). The defined target for ETOPS approvals with diversion times of 180 minutes or less, for catastrophic loss of thrust from independent causes, is 0.3x10^-8/hr (see paragraph 3 of this Appendix). This target was based on engine IFSD rates that were higher than can be and are being achieved by modern ETOPS airframes/engines. To achieve the same level of safety for ETOPS approvals beyond 180 minutes as has been achieved for ETOPS approvals of 180 minutes or less, the propulsion system reliability IFSD rate target needs to be set and maintained at a level that is compatible with an Extremely Improbable safety objective
(i.e. 1.0x10^-9/flight hr).

For example, a target overall IFSD rate of 0.01/1000 hr (engine hours) that is maintained would result in the loss of all thrust on two engine aeroplanes being extremely improbable even assuming the longest time envisaged. The risk model formula summarised for a two-engine aeroplane is:

p/flight hour = [2(Cr x{T-t}) x Mr(t)] divided by T

(1)       p is the probability of a dual independent propulsion unit failure on a twin,

(2)       2 is the number of opportunities for an engine failure on a twin (2),

(3)       Cr is cruise IFSD rate (0.5x overall rate), Mr is max continuous IFSD rate (2x overall rate), T is planned max flight duration in hours (departure to planned arrival airport), and t is the diversion or flight time in hours to a safe landing. IFSD rates, based on engine manufacturers’ historical data from the last 10 years of modern large turbofan engines, presented to the JAA/EASA and ARAC ETOPS working groups, have shown cruise IFSD rates to be of the order of 0.5x overall rate, and the max continuous IFSD rate (estimated from engine fleet analysis) to be 2x overall rate. Then, for an IFSD goal of.010/1000EFH overall, the cruise IFSD rate is .005/1000EFH, and the max continuous rate is .020/1000EFH.

(4)       Sample calculation (max flight case scenario): assume T = 20 hour max flight duration, an engine failure after 10 hours, then continued flight time required is t = 10 hours, using the ETOPS IFSD goal of .010/1000EFH or less, results in a probability of p=1 E-9/hour (i.e. meets extremely improbable safety objective from independent causes).

(5)       A relationship between target IFSD rate and diversion times for two-engine aeroplanes is shown in Figure 2.

Figure 2

4.         ENGINEERING ASSESSMENT CRITERIA FOR ACCEPTABLE RELIABILITY VALIDATION METHODS

The following criteria identify some areas to be considered during the engineering assessment required for either reliability validation method.

a.         There are maintenance programmes, engine on-wing health monitoring programmes, and the promptness and completeness in incorporating engine service bulletins, etc. that influence an operator’s ability to maintain a level of reliability. The data and information required will form a basis from which a worldfleet engine shutdown rate will be established, for use in determining whether a particular airframe/engine combination complies with criteria for extended-range operation.

b.         An analysis will be made on a case-by-case basis, of all significant failures, defects and malfunctions experienced in service or during testing, including reliability validation testing, for the particular airframe/engine combination. Significant failures are principally those causing or resulting in in-flight shutdown or flameout of the engine(s), but may also include unusual ground failures and/or unscheduled removal of engines. In making the assessment, consideration should be given to the following:

(1)       The type of propulsion system, previous experience, whether the power-unit is new or a derivative of an existing model, and the operating thrust level to be used after one engine shutdown;

(2)       The trends in the cumulative 12-month rolling average, updated quarterly, of in‑flight shutdown rates versus propulsion system flight hours and cycles;

(3)       The demonstrated effect of corrective modifications, maintenance, etc. on the possible future reliability of the propulsion system;

(4)       Maintenance actions recommended and performance and their effect on propulsion system and APU failure rates;

(5)       The accumulation of operational experience which covers the range of environmental conditions likely to be encountered;

(6)       Intended maximum flight duration and maximum diversion in the ETOPS segment, used in the extended-range operation under consideration.

c.         Engineering judgement will be used in the analysis of paragraph b. above, such that the potential improvement in reliability, following the introduction of corrective actions identified during the analysis, can be quantified.

d.         The resultant predicted reliability level and the criteria developed in accordance with Section 3 (RISK MANAGEMENT AND RISK MODEL) should be used together to determine the maximum diversion time for which the particular airframe/engine combination qualifies.

e.         The type design standard for type approval of the airframe/engine combination, and the engine, for ETOPS will include all modifications and maintenance actions for which full or partial credit is taken by the (S)TC holder and other actions required by EASA to enhance reliability. The schedule for incorporation of type design standard items should normally be established in the Configuration, Maintenance and Procedures (CMP) document, for example in terms of calendar time, hours or cycles.

f.          When third-country (S)TC holders’ and/or third-country operators’ data is evaluated, the respective foreign authorities will be offered to participate in the assessment.

g.         ETOPS reliability tracking board (RTB)’s findings.

Once an assessment has been completed and the RTB has documented its findings, EASA will declare whether or not the particular airframe/engine combination and engine satisfy the relevant considerations of this AMC. Items recommended qualifying the propulsion system, such as maintenance requirements and limitations will be included in the Assessment Report (Chapter II Section 10 of this AMC).

h.         In order to establish that the predicted propulsion system reliability level is achieved and subsequently maintained, the (S) TC holder should submit to EASA an assessment of the reliability of the propulsion system on a quarterly basis. The assessment should concentrate on the ETOPS configured fleet and should include ETOPS-related events from the non-configured fleet of the subject airframe/engine combination and from other combinations utilising a related engine model.

5.         EARLY ETOPS OCCURRENCES REPORTING & TRACKING

a.         The holder of a (supplemental) type certificate of an engine, which has been approved for ETOPS without service experience in accordance with this AMC, should establish a system to address problems and occurrences encountered on the engine that could affect the safety of operations and timely resolution.

b.         The system should contain a means for: the prompt identification of ETOPS related events, the timely notification of the event to EASA, proposing a resolution of the event and obtaining EASA’s approval. The implementation of the problem resolution can be accomplished by way of EASA-approved change(s) to the type design, the manufacturing process, or an operating or maintenance procedure.

c.         The reporting system should be in place for at least the first 100 000 fleet engine hours. The reporting requirement remains in place until the fleet has demonstrated a stable in‑flight shutdown rate in accordance with the targets defined in this Appendix 1.

d.         For the early ETOPS service period, an applicant must define the sources and content of the service data that will be made available to them in support of their occurrence reporting and tracking system. The content of this data should be adequate to evaluate the specific cause of all service incidents reportable under Part 21A.3A(b), in addition to the occurrences that could affect the safety of operations, and should be reported, including:

(1)       in-flight shutdown events and rates;

(2)       inability to control the engine or obtain desired power;

(3)       precautionary thrust reductions (except for normal troubleshooting as allowed in the aircraft flight manual);

(4)       degraded propulsion in-flight start capability;

(5)       un-commanded power changes or surges;

(6)       diversion or turn-back;

(7)       failures or malfunctions of ETOPS significant systems;

(8)       unscheduled engine removals for conditions that could result in one of the reportable items listed above.

6.         CONTINUED AIRWORTHINESS OF TYPE DESIGN

For ETOPS, EASA will periodically review its original findings by means of a Reliability Tracking Board. In addition, EASA document containing the CMP standard will be revised as necessary.

Note: The Reliability Tracking Board will usually comprise specialists from aeroplane and engine disciplines (see also Appendix 2).

Periodic meetings of the ETOPS Reliability Tracking Board are normally frequent at the start of the assessment of a new product. The periodicity is adjusted by EASA upon accumulation of substantial service experience if there is evidence that the reliability of the product is sufficiently stable. The periodic meetings of the board are discontinued once an ETOPS product, or family of products, has been declared mature by EASA.

Note:The overall engine IFSD rate should be viewed as a worldfleet average target figure of engine reliability (representative of the airframe/engine combination being considered) and if exceeded, may not, in itself, trigger action in the form of a change to the ETOPS design standard or a reduction in the ETOPS approval status of the engine. The actual IFSD rate and its causes should be assessed with considerable engineering judgement. For example, a high IFSD rate early after the commencement of the operation may be due to the limited number of hours contributing to the high rate. There may have been only one shutdown. The underlying causes have to be considered carefully. Conversely, a particular single event may warrant corrective action implementation, even though the overall IFSD rate objective is being achieved.

a.         Mature ETOPS products

A family of ETOPS products with a high degree of similarity is considered to be mature ones if:

(1)       The product family has accumulated at least 250 000 flight hours for an aeroplane family or 500 000 operating hours for an engine family;

(2)       The product family has accumulated service experience covering a comprehensive spectrum of operating conditions (e.g. cold, hot, high, and humid);

(3)       Each ETOPS approved model or variant in the family has achieved the reliability objectives for ETOPS and has remained stable at or below the objectives fleet-wide for at least 2 years;

New models or significant design changes may not be considered mature until they have individually satisfied the condition of paragraph 6.a above.

EASA makes the determination of when a product or a product family is considered mature.

b.         Surveillance of mature ETOPS products

The (S)TC holder of an ETOPS product which EASA has found mature, should institute a process to monitor the reliability of the product in accordance with the objectives defined in this Appendix 1. In case of occurrence of an event or series of events or a statistical trend that implies a deviation of the reliability of the ETOPS fleet, or a portion of the ETOPS fleet (e.g. one model or a range of serial numbers), above the limits specified for ETOPS in this AMC, the (S)TC holder should:

(1)       Inform EASA and define a means to restore the reliability through a minor revision of the CMP document, with a compliance schedule to be agreed with EASA if the situation has no immediate safety impact;

(2)       Inform EASA and propose an adhoc follow-up by EASA until the concern has been alleviated or confirmed if the situation requires further assessment;

(3)       Inform EASA and propose the necessary corrective action(s) to be mandated by EASA through an AD if a direct safety concern exists.

In the absence of a specific event or trend requiring action, the (S)TC holder should provide EASA with the basic statistical indicators prescribed in this Appendix 1 on a yearly basis.

c.         Minor Revision of the ETOPS CMP Document

A Minor Revision of the ETOPS CMP document is one that contains only editorial adjustments, configurations, maintenance and procedures equivalent to those already approved by EASA or new reliability improvements which have no immediate impact on the safety of ETOPS flights and which are introduced as a means to control the continued compliance with the reliability objectives of ETOPS.

Minor revisions of the ETOPS CMP document should be approved by authorised signatories personnel of the (S)TC holder under the provisions of its approved Design Organisation Handbook.

7.         DESIGN ORGANISATION APPROVALS

(S)TC holders of products approved for ETOPS should hold a design organisation approval (DOA) conforming to Part 21, with the appropriate terms of approval and privileges. Their approved design organisation handbook (DOH) must contain an appropriate description of the organisation and procedures covering all applicable tasks and responsibilities of Part 21 and this AMC.

[Amdt 20/7]

[Amdt 20/21]