GM6 Safety performance indicator (SPI) on automated safety data recording systems

Regulation (EU) 2019/317

A.      General

The performance indicator on automated safety data recording systems (where implemented) in point 1.2(e) in Section 2 of Annex I is defined as:

‘[…], the use of these systems by the air navigation service providers, as a component of their safety risk management framework, for the purposes of gathering, storing and near-real time analyses of data related to, as a minimum, separation minima infringements and runway incursions.’

Beyond a narrow interpretation of the indicator as supporting a pure binary assessment of the performance, the indicator should be understood as an initiative to foster a proactive approach to safety management, one looking closely at day-to-day performance and including measures other than occurrences to anticipate risk. This is in line with Recommendation 7.1/1 — Data-driven decision-making from the thirteenth ICAO Air Navigation Conference (ANC) to facilitate ‘[…] data-driven decision-making in support of safety intelligence to support safety risk management’.

This guidance material aims to assist Member States, NSAs and ANSPs in using automated safety data recording systems in the implementation of data-driven safety decision-making processes. The monitoring of this indicator during RP3 will provide key information to the forthcoming development of standardised risk-based decision-making policies and best practices for the design and parameterisation of safety-monitoring tools and models.

B.      Digitalisation and moving towards an early-warning capability for ATM

Together with the massive amount of safety-related information that aviation generates today, as well as the increasingly rare accidents and serious incidents from which to learn and mitigate, goes the potential for a fundamental change in the mindset towards a more proactive, meaning-anticipative, collaborative, meaning-sharing, and performance-based approach to safety management. With the impending rise of information technology and overall digitalisation and rising automation of ATM, the pace of data creation can only increase. Obviously, data mining does not replace the technical and operational competencies of the ATM community and while it reduces uncertainty, it does not eliminate it, but it contributes to create safety intelligence. In particular, data helps in identifying and investigating the weak signals that could eventually result in catastrophic events.

Therefore, today, the usage of automated safety data recording systems paves the way towards an early-warning capability for ATM with the aim to:

—                   detect unsafe trends and implement changes that remove these threats before a serious event or worse happens;

—                   react within a particular timescale that depends on the rate of trend progression;

—                   not raise ‘false alarms’, nor lead to disproportionate focus on low-priority issues, or lead to unanticipated side effects; and

—                   reach all those needed to ensure an aviation-system-wide reaction if the problem is generic, or localised reaction if it is a localised issue.

C.      Functional model

The sequence of steps or functions (building upon automated safety data recording systems) that are needed for an early-warning function for ATM are as follows:

—                   monitoring of data sources in ATM in a systematic and coherent way, in particular with respect to the specification of surrogates for accidents and incidents and setting of triggers for identifying adverse events and signals;

—                   filtering, i.e. determining what is a ‘signal’ and what is ‘noise’, using statistical and risk-based criteria for deciding when to further analyse a potential trend or key occurrence;

—                   trend identification to determine the exact nature of the safety issue;

—                   getting sufficient understanding to estimate the risk priority and to prepare for mitigation measures. This should ensure that disproportionate focus does not occur, and that undesirable side effects are not generated. ‘Deconstructing’ the data should rely on a technical-/operational-centred approach to ensure the right balance between a current issue and others that are pending;

—                   developing mitigation measures to deal with the issue and prevent its recurrence and/or propagation;

—                   disseminating and engaging, i.e. letting the right people know;

—                   verifying and confirming that the problem has gone away building upon the never-ending stream of data while paying due attention to the potential ‘Hawthorne effect’, which means the attention paid to an issue may mean it disappears for a time, then resurfaces;

—                   documenting thereby ensuring that the whole process for an identified issue has been recorded so that if it recurs or a similar problem arises, the safety ‘thinking’ and analysis is available for future users/analysts. Documentation at this level also allows deeper ‘learning’ to occur, e.g. across issues. A larger picture may emerge. It would also save time and resources if problems resurface or ‘mutate’ into related problems; and

—                   feeding forward the information from analyses to the risk assessment processes and to designers of future systems.

D.      Fundamental components

Four fundamental components in the usage of automated safety data recording systems in support of the ‘safety risk management’, ‘safety achievement’, ‘safety assurance’, and ‘safety promotion’ elements of the SMS are:

1.       the involvement of data analysts, data scientists, predictive modellers, statisticians and other analytics professionals to structure and analyse growing volumes of data to uncover information including hidden patterns, unknown correlations, etc.;

2.       the interactive visualisation of the structured safety data to support the safety, technical and operational analyses;

3.       the involvement of safety, operational and technical expertise to comprehend the data and prioritise the actions needed to ensure safe ATM operation; and

4.       the gathering of the safety data and information in a just-culture organisational environment.