Discovering hidden knowledge in aviation data

Author: Paula Lopez (INX)

Machine learning is producing outstanding results although we know it is still far from emulating human intelligence. Applying machine learning techniques, including multi-level artificial neural networks (deep learning) to, for example, speech or image recognition has been continuously resulting in improved results (e.g. digital assistants like Apple´s Siri or Amazon´s Echo). In spite of the significant progress achieved so far, there are still some challenges that need to be resolved in order to be applicable in most industries. On one hand, we face a fragmented ecosystem, meaning that there is a gap between the data scientists and the domain experts working in each particular sector. In order to be able to convert data into knowledge, collaboration among both expertises is required. On the other hand, challenges related to data management and data analysis need to be addressed prior to implementing machine learning techniques in most industries. These challenges, just to name a few, include heterogeneous and distributed data sources, data validation, distributed data architectures, data security, scalability, real-time analysis and decision-support or data visualization.

However, we cannot fall into the error of assuming that a machine learning problem can be addressed through a generic standard application of a set of algorithms and techniques. Machine learning problems are highly case-dependent and, therefore, the purpose of the analysis needs to be carefully defined in advance. This is what we (at Innaxis) call Purposeful Knowledge Discovery which also was the title of the keynote speech made by Innaxis President Carlos Alvarez Pereira at the SESAR Innovation Days 2017 in Belgrade. And this is, precisely, the approach we follow at Innaxis in our data science research projects, like SafeClouds.eu: an H2020 project aimed at enhancing aviation safety through the application of data science techniques.

SafeClouds.eu includes a team of 16 partners including data scientists and engineers from several research entities (Innaxis, Tadorea, Fraunhofer, TU Munich, Linköping University, TU Delft and CRIDA) and a group of airlines, ANSPs and safety authorities (Iberia, Air Europa, Vueling, Norwegian, Pegasus, LFV, Eurocontrol, AESA and EASA). This group of airspace stakeholders is the user group of the project, in other words, those defining the questions for which they need data for gaining answers. These questions can be of three types: descriptive (what happened?), predictive (what will happen?) or prescriptive (what to do for what we want to happen). Once the questions are defined (SafeClouds.eu use cases) the team of data scientists and engineers work together and collaborate with users covering the full cycle of data science techniques: data management, data processing architecture, deep analytics, data protection, pseudo- anonymization, advanced visualization and user experience. As previously mentioned, every step has its own challenges as there are no data science standard tools to be transferred automatically from one field to another. Below, we outline just two challenges: fusion of proprietary confidential data and benchmarking among these competing stakeholders.

  • Smart Data Fusion: Simply erasing the flight-identifier parameters would protect the data but not allow fusion of datasets. Many data require protection and cannot be shared (e.g. FDM data and radar tracks), so fusion needs sophisticated techniques coming from cryptography and enabling coding sensitive data in a non-reversible way.
  • Secure Blind Benchmarking: Benchmarking among stakeholders based on data that cannot be shared also requires the application of specific techniques. This includes secure multiparty computation enabling comparison between confidential data without disclosing the data, not even to a trusted third party.

These are just some examples of the challenges the SafeClouds.eu team is facing in the field of aviation safety data analysis. The solutions offered by these techniques make them ideal to be applied to other fields such as fuel consumption but, again, the purpose of the analysis will determine the following necessary steps.

 

Augmented reality and data visualization (in aviation)

Dataset-post

Present-day technology is so powerful that the perception of reality can be easily and realistically modified with IT tools, providing users withan experience beyond “simple” reality. This is achievable by mixing real-world environment elements supplemented and/or augmented by computer-generated inputs. The current post unpacks this topic, focusing specifically on the data visualization aspects. In brief, augmented reality can take two approaches:

  • First, inventing totally new scenarios, in which the user becomes part of a “parallel universe”. Supplementing the real-world environment with an unreal one; either a virtual place (video game) or a different location (i.e. another real location). This is the case of futuristic 90’s and early 2000’s alike head-mounted displays with users’ eyes looking at full screens recreating other places. The ergonomics aspects are usually modest for most of the applications due to the head-mounted displays weight and size.

5dt.jpeg

  • The second, and closer to “data visualization” area is the so called “mediated reality”. The real-world environment enhanced by virtual elements displayed in glasses, windscreens etc. In them, additional information/data is provided. The real challenges is to decide what, how and when to display the information, without requiring users to look away from their usual viewpoints, while providing extra value. The integration and user experience is much more natural and enjoyable than the fully immersive systems.
Research project Augmented Reality - contact-analogue Head-Up Display (10/2011)

Research project Augmented Reality – contact-analogue Head-Up Display (10/2011)

In this context, one of the very early examples of head-up displays can be found precisely in aviation, almost 80 years ago. In 1937, the German ReviC12/A fighter aircraft included a basic reflector sight indicating some basic aircraft magnitudes such as speed and turn rate, to reduce the (visual) workload of pilots in case of extreme maneuvering
659px-Revi_C12-A
Nowadays virtually all modern fighters (F18, F16, Eurofighter) use head-up displays. The most modern versions (F35) do not have head-up displays, and instead include helmet mounted displays, ensuring the proper orientation of the user’s head, for all circumstances.
Hud_on_the_cat
One of the trending topics of augmented reality within aviation is its usage in air traffic control (ATC), particularly in Tower environments. Below are two common approaches:

  • Visual information is enhanced to ease identification and tracking of aircraft. This includes tools similar to head-up displays and/or helmets-displays that enhance the information (providing for instance, aircraft ID, scheduled times, etc). This approach could be extremely useful in low visibility conditions by facilitating the tower ATCOs tasks. It also avoids dividing attention between the primary visual field (the window) and the auxiliary tools (surface radar, strips etc).

Screen Shot 2017-02-23 at 15.05.29maxresdefault

  • The extreme version is a complete virtual control tower, the so called “remote tower”. ATC would have remote control rooms with video-sensors on-site, including augmented reality enhancements. The synthetic augmentation of vision increases the situational awareness at the airport, especially during poor visibility conditions, or blocked line-of-sight areas due to airport geometry. It additionally provides benefits in terms of cost saving (no need to build and maintain control tower facilities) and a more efficient use of human resources (potentially serving multiple airports with low traffic events from a centralised location). Research in this field started in FP6 project “ART” and is now being progressed by SESAR WP6. In fact, Örnsköldsvik/Gideå airport is the first on the world deployment of remote tower, in late 2015, by the Swedish LFV. In US, Fort Collins-Loveland Municipal Airport was the first approved and tested airport with a remote tower in 2016.

Screen Shot 2017-02-23 at 15.08.47
For the air passenger and mobility context, augmented reality and the wide range of solutions providing additional real-time information to passengers is taking off as well. (No pun intended.)
These technological innovations include indoor location tracking, real-time information on boarding gates, real-time updates on flight delays, and information on airport facilities and shops. This is also being expanded to knowing the number and location of available parking spaces to facilitate the passenger experience in the (sometimes not so easy) airport processes. For example, Copenhagen airport, in collaboration with SITA, created the very first augmented reality indoor app in 2012. Now there is an endless list of both airlines and airports with similar apps.
40154332-1-610-sita-augmented-reality-starbucks40154332-5-610-sita-augmented-reality-gate-info
Do you think augmented reality together with innovative data visualization can have a significant impact in future aviation?
What are its challenges and potential benefits?
We’re interested in hearing your thoughts and ideas.

Data acquisition and Visualisation Technologist

Innaxis is seeking a Data acquisition and Visualisation Technologist to join the research and development team for aviation projects. As a member of the team, you join a very interdisciplinary group of researchers, scientists, mathematicians and engineers who work for private companies and public institutions to solve the most challenging problems using the most of their data.

A mixture of creativity and technical skills are required to complement the skill set of a team that has worked in the last 5 years establishing landmarks in terms of network performance analyses across different areas within the aviation sector.

We are looking for a talented individual to complement the existing research threads on data acquisition and visualising large data sets in an innovative way to provide new insights on the performance of complex systems, including the development of interactive dashboards to enable the real time analysis of complex phenomena. Being part of a team entails working with other skilled researchers who are currently focused on knowledge discovery, data mining and big data infrastructures.

Requirements are as follows:
  • Degree in Computer Science or similar (mathematics, physics) with outstanding background and experience in programming.
  • Solid design and conceptual representation skills.
  • Tasteful and artful conception of data visualisation and infographics.
  • An existing portfolio would be very appreciated and very positively evaluated
  • Passion for data visualisation on top of current thinking and trends
  • Location: Either Madrid (Spain) or Linz (Austria)

Technical skills which may be relevant in the evaluation:

  • Web technologies: HTML, CSS3, JavaScript, …
  • Basic knowledge of database technologies and use: MySQL, MongoDB, JQuery
  • Proficiency in Java
  • Experience in any of the following libraries: d3.js, dc.js, Raphaël, …
  • Other visualisation technologies experience is appreciated
  • Any other programming language is a plus: Python, Perl, C, R, Matlab …
  • Hands-on experience with cloud based technologies. Concretely AWS, Amazon S3, Amazon EC2 and EBS

Innaxis offers:

  • Immediate start within a highly qualified and collaborative international team with innovative thinking and working methodology focused on the development of large scale research and innovation projects.
  • Interesting salary as a function of skills, experience and education.
  • Flexibility and good working conditions

Interested candidates should send their detailed CV and relevant information to innovation@innaxis.org.

Connect with us!