AMC2 SKPI Measurement of the effectiveness of safety management (EoSM) at ANSP level

Regulation (EU) 2019/317

The answers to the questionnaire should be used to measure the level of effectiveness in achieving the management objectives defined in this AMC.

For each question, ANSPs should provide their NSA/competent authority with information on the level of effectiveness (or level of implementation) and evidence to justify their answer as indicated below.

The questionnaire, which should be filled in by the ANSPs, is detailed in the Appendix to AMC3 SKPI, GM3 SKPI and GM4 SKPI.

A.      Components, study areas (SAs)

According to Commission Implementing Regulation (EU) 2019/317, the indicator is stated as follows: ‘The minimum level of the effectiveness of safety management to be achieved by air navigation service providers certified to provide air traffic services. This KPI shall be measured by the level of implementation of the following safety management objectives:’

For the sake of coherence in describing the EoSM in this document and the components of the ICAO Safety Management Framework, these safety management objectives are hereinafter referred to in this AMC as ‘components’, and they are as follows:

(a)      safety policy and objectives;

(b)     safety risk management;

(c)      safety assurance;

(d)     safety promotion;

(e)     safety culture.

Each component addresses a set of SAs as follows:

—          Component 1: Safety culture

—                  SA:

—         Development of a positive and proactive organisational culture

—          Component 2: Safety policy and objectives

—                  SAs:

—         Safety policy

—         Safety accountabilities

—         Coordination of emergency response plan

—         Safety management system documentation

—          Component 3: Safety risk management

—                  SA:

—         Risk management process

—          Component 4: Safety assurance

—                  SAs:

—         Safety reporting

—         Safety surveys and audits

—         Safety performance monitoring

—         Management of change

—         Continual improvement of the SMS

—          Component 5: Safety promotion

—                  SAs:

—         Training and education

—         Safety communication

The SAs are further broken down into questions for which the ANSP respondents are expected to choose a level from the predetermined list of maturity levels that best describes the performance of the organisation with respect to the aim of that question. Organisations are reminded that in order to qualify for the chosen maturity level, all requirements as listed in the question must be met. The maturity level of an SA should be assigned considering the minimum maturity level achieved among the questions in that SA. Similarly, the maturity level of a component should be assigned considering the minimum maturity level achieved among the SAs in that component.

B.      Scoring

In order to be able to measure quantitatively the overall effectiveness of safety management of the ANSP, the answers to the questions should be quantified.

The responses provided by the ANSP on their questionnaires are assigned a numerical value from 1 to 4, corresponding to levels A to D. Level E is not assessed and has no value assigned. Each question has the same weight over the final overall score. The numerical value of each question should be added from the questionnaire responses and the final overall EoSM score is calculated as a percentage of the maximum score value possible.

Mathematically, the effectiveness score for an ANSP ‘j’ is calculated as follows:

Where:

S _j       is the effectiveness score of the ANSP

r_kj       is the numeric value of the response of an ANSP to question k

q        is the number of questions for which responses were provided by the ANSP

Q       is the total number of questions in the EoSM questionnaire, i.e. 28

C.      Mechanism for verification

The verification of the ANSP questionnaires by the NSA/competent authority should take place before the questionnaires and their results are submitted to EASA. The verification mechanism is presented in Figure 1.

ANSPs should assign a focal point for the purpose of the verification process.

Figure 1: Representation of the verification mechanism

The competent authority/NSA may allocate the detailed verification task to a qualified entity.