GM1 ORA.FSTD.100 General

ED Decision 2012/007/R

COMPLIANCE MONITORING – ORGANISATIONs OPERATING FSTDs – GENERAL

(a) The concept of compliance monitoring (CM) is a fundamental requirement for organisations operating FSTDs. An effective CM function is vitally important in supporting operation of the devices, in a structured way, to ensure they remain in compliance with the technical standards of CS-FSTD(A) and CS-FSTD(H) and continue to be effective training tools. An effective CM function is also essential to support any level of extended recurrent evaluation period as permitted by ORA.FSTD.225(b).

(b) The following guidance has been developed to provide additional material to help both organisations operating FSTDs and competent authorities in developing effective CM that satisfy the applicable requirements and ensure the highest standards of training are maintained.

(c) Additional GM provide a compliance checklist for organisations operating FSTDs (GM2 ORA.FSTD.100) and guidance detailing the preparation for an evaluation by the competent authority (GM3 ORA.FSTD.100). The compliance checklist should be used by the competent authorities as a standardised checklist for the elements that are expected in the CM function of an organisation operating FSTDs. The organisation should complete as a minimum the second column of the checklist by providing appropriate manual or procedure references for each of the identified elements of the CM function. Additional information can be provided in the third column to aid assessment of the checklist as appropriate. This would then be provided to the competent authority. Use of this checklist should assist in ensuring a consistent approach by the competent authorities and also provide organisations operating FSTDs with additional guidance on all the elements of a CM function that the competent authorities will expect. The guidance is provided to help organisations operating FSTDs to prepare for authority visits.

(d) The documentation of the CM may be electronic, provided the necessary controls can be demonstrated. This should include control of any paper copies that may be downloaded for use by individuals. It is recommended that any such copies are automatically designated as uncontrolled as part of the download process. Whilst electronic signatures on master documents may be accepted, with appropriate protections, a hardcopy master of the CM manual should be provided, with wet-ink signatures to be held by the applicant.

(e) It should be recognised that whatever CM is developed, it will not be effective unless it becomes an integral part of the way in which the organisation works. It includes both the necessary procedures for maintaining compliance with all the applicable requirements and a compliance monitoring programme (CMP) to monitor the execution of these procedures. A successful CM will ensure that the highest training tool is available at all times. If the CM is viewed as an addon to existing processes it will become a burden and it will never be wholly effective. It should also be noted that compliance control or inspection is only a small part of a CM. If the CM is working effectively, inspections such as fly-outs should become routine revealing little beyond day-to-day unserviceabilities. Systematic defects should be captured by the CMP.

(f) The competent authority should be satisfied that the accountable manager is able to adequately provide the required level of resources to properly support the FSTD. Detailed knowledge of FSTD requirement standards are not necessary, only sufficient to understand his/her responsibility for ensuring the FSTD is properly supported. The assessment of the compliance monitoring manager should concentrate on establishing that the nominee has sufficient knowledge and experience of both CM management and FSTD operations to operate a compliance monitoring system (CMS) within an organisation operating FSTDs. This is likely to require experience of working in the compliance monitoring field and sufficient knowledge of FSTDs and the technical standards with which they should comply.

(g) If an organisation operating FSTDs is certified under any international quality standard it should assure that it fully covers the applicable organisation requirements of Part-ORA and the qualification basis.

(h) For small organisations, it is perfectly acceptable to combine the roles of compliance monitoring manager and accountable manager. For other organisations that hold multiple certificates and may cover multiple sites, it is advantageous to have a common CM function with an overall compliance monitoring manager. However, it is essential, particularly where sites may be significantly separated geographically, that there is a nominated representative at each site and possibly for each certificate. These representatives should hold the delegated responsibility of the CM manager for the day-to-day CM role at their site and in their function and have the necessary direct reporting line to the overall CM manager. It will also be necessary to ensure that local representatives are also acceptable to the local competent authority. In many cases the local representatives may perform other functions in addition to this role. This is acceptable provided the necessary independence of any compliance monitoring activity is maintained.

(i) CM, as a whole, begins with the requirements with which the system seeks to comply. These include both the technical standards, in this case the relevant parts of CS-FSTD(A)/(H) plus any other specific standards, for example health and safety regulations, and the compliance monitoring objectives, such as defect rates and rectification intervals and FSTD reliability targets. The CM should define the process by which these standards are made available to those who require them.

(j) The next part of CM is that part which defines the day-to-day procedures or working practices by which the standards will be achieved. These procedures should include as a minimum defect reporting systems, defect rectification processes, tracking mechanisms, preventative maintenance programmes, spares handling, equipment calibration and configuration management of the device. They should include checks to assess the compliance of the performed actions. These procedures and standards should be made readily available to anybody involved in the maintenance and day-to-day operation of the FSTD.

(k) The third part of CM is the method by which the organisation operating an FSTD confirms the device is maintained in compliance with the defined standards and is being operated in accordance with the defined procedures. This is the compliance monitoring programme (CMP) and includes the audit methods, reporting and corrective action procedures and feedback, management reviews and schedules for audits of all aspects of the FSTD operation.

(l) Across all aspects of CM, and most important to it, are the people. CM includes the definition of the responsibilities of all staff and should include a declaration of the minimum levels of resource proposed for the direct support of the FSTD plus the levels of support and managerial staff proposed. The levels of resource can be affected by factors such as local health and safety regulations, existence of weekend and/or night usage of the device(s), etc. CM also includes definition of the skills and experience required for staff and leads to definition of any required training programmes. Training needs cover both technical training and audit training, including QTG running and checking and fly-out techniques for flight crew.

(m) The documentation of CM may be provided in any number of documents provided there are appropriate cross-references in all documents such that the system is fully traceable in both directions from end to end. For all but small organisations at least two documents would be expected:

(1) Firstly, a CM manual containing the policy, terminology, organisational charts and responsibilities, an overview of all processes, within the system, including those for maintaining regulatory compliance such as QTG running and fly-outs (function and subjective testing), CMP including the audit schedule and audit procedures including reporting and corrective action procedures. In addition, the CM manual should include, either directly or by reference, the identification of skills and experience and associated training.

(2) Secondly, a procedures manual containing, as a minimum, software and hardware control procedures, configuration control procedures including, for example, control of training loads, updates to visual models, navigation and instructor operation station (IOS) databases, QTG running and checking procedures, fly-out procedures, maintenance procedures including both defect rectification and preventative maintenance processes. Any standard forms and checklists should also be included.

(n) The CM documentation also includes all records such as technical logs, QTG runs, fly-out reports and maintenance job cards.

(o) For organisations with several certificates, separate and modular procedures manuals with a single CM manual covering all approvals, may be acceptable.

(p) It is important to understand the difference between compliance assurance and compliance control. An effective CM will contain elements of both. Compliance control is normally done by inspection of the product; it provides confirmation at the time of the inspection that the product conforms to a defined standard.

(q) The compliance assurance element is essential to ensure the standard is maintained throughout the periods between product (FSTD) inspections. Within a CMP, the processes are defined that are necessary to provide confidence that the FSTD(s) is/are being supported and maintained to the highest possible standard and in compliance with the relevant requirements. A programme of internal audits is then set in place to confirm that the processes are being followed and are effective. The competent authority would normally oversee a certified organisation by process and system audit, however, in the case of FSTDs, authority oversight includes an inspection element in the form of the recurrent FSTD evaluation.

(r) In addition to the normal process and system audits, the compliance assurance audit schedule should include the schedule for each FSTD for fly-outs and QTG running through the audit year.

(s) The audit procedure should include, at least, the following: statement of scope, planning, initiation of audit, collection of evidence, analysis, reporting of findings, identification and agreement of corrective actions and feedback, including reporting significant findings to the competent authority, where appropriate. The review of published material could include, in addition to the CM and procedures manuals, QTG records, fly-out reports, technical log sheets, maintenance records and configuration control records.

(t) In addition to basic knowledge of FSTD requirements and operation, it is expected that auditors have received training in CM and audit techniques.

(u) The routine fly-outs of the device are a specialised part of the audit programme. It is essential that the pilots tasked with carrying out these fly-outs are adequately experienced. They would be expected to be type rating instructor/examiner (TRI/TRE) qualified on the type, and should have experience of simulator evaluations carried out by the competent authority. The assignment of such pilots can present difficulties, particularly for the independent organisation operating FSTDs not directly associated with an airline. It is vital for the organisation to ensure their users are aware of the importance of the fly-outs as part of the continued qualification of the device and the need to assist in the provision of suitably qualified pilots to carry them out. It is worth noting that simulator users are required to satisfy themselves that the training devices they use are assessed for continued suitability, as part of their own CMP. Involvement in fly-outs assists in meeting this need.

(v) Whilst it is accepted that the number of audits required in an organisation with a single device will be significantly less than those in larger organisations with multiple devices, the CMP should still meet the same criteria, and cover all aspects of the operation within a 12 month period. The independence of the audit personnel should be maintained at all times. The audit programme, whether by full audit or by using a checklist system should still be sufficiently comprehensive to provide the necessary level of confidence that the device is maintained and operated to the highest possible standard. This includes monitoring and review of corrective actions and feedback processes.

(w) The successful use of sub-contractors who play a significant role in the provision of services, such as maintenance or engineering services, to an organisation operating FSTDs is reliant on the sub-contractor operating under the CM of the organisation. All requirements that an organisation is expected to meet are equally applicable to his/her sub-contractor. It is the organisation’s responsibility to ensure that the sub-contractor complies with its CM.

(x) It is essential that a proper understanding of the CM and how it applies to each and every staff member is provided by appropriate training to all, not just those directly involved in operating the CM, such as the accountable manager, the CM manager, representatives and the auditors. The training given to those directly involved in CM should cover the CM, audit techniques and applicable technical standards. CM familiarisation training should be an integral part of any induction training and recurrent training. Update training on technical standards for audit personnel, is also of particular importance.

(y) Any effective CM will include measurement of its effectiveness. The organisation should develop performance measures that can be monitored against objectives. Such measures, often referred to as metrics, should be reviewed by the competent authority as part of its oversight of the CM within the organisation and during recurrent evaluations. In addition they should form part of the data reviewed during scheduled management reviews as part of the CM.

(z) ARINC 433 provides good guidance on FSTD compliance measurement. Metrics should monitor not only individual FSTD performance but, for larger organisations, how each FSTD is performing within the fleet. It is also recommended that metrics data be shared, regularly, with the FSTD manufacturers to allow monitoring for generic problems such as design issues, which may be best addressed with a fleet-wide solution.