ALBERT

Description: Perception sensors such as camera, radar, and lidar have gained considerable popularity in the automotive industry in recent years. In order to reach the next step towards automated driving it is necessary to implement fault diagnosis systems together with suitable mitigation solutions in automotive perception sensors. This is a crucial prerequisite, since the quality of an automated driving function strongly depends on the reliability of the perception data, especially under adverse conditions. This publication presents a systematic review on faults and suitable detection and recovery methods for automotive perception sensors and suggests a corresponding classification schema. A systematic literature analysis has been performed with focus on lidar in order to review the state-of-the-art and identify promising research opportunities. Faults related to adverse weather conditions have been studied the most, but often without providing suitable recovery methods. Issues related to sensor attachment and mechanical damage of the sensor cover were studied very little and provide opportunities for future research. Algorithms, which use the data stream of a single sensor, proofed to be a viable solution for both fault detection and recovery.

Description: Development and validation of reliable environment perception systems for automated driving functions requires the extension of conventional physical test drives with simulations in virtual test environments. In such a virtual test environment, a perception sensor is replaced by a sensor model. A major challenge for state-of-the-art sensor models is to represent the large variety of material properties of the surrounding objects in a realistic manner. Since lidar sensors are considered to play an essential role for upcoming automated vehicles, this paper presents a new lidar modelling approach that takes material properties and corresponding lidar capabilities into account. The considered material property is the incidence angle dependent reflectance of the illuminated material in the infrared spectrum and the considered lidar property its capability to detect a material with a certain reflectance up to a certain range. A new material classification for lidar modelling in the automotive context is suggested, distinguishing between 7 material classes and 23 subclasses. To measure angle dependent reflectance in the infrared spectrum, a new measurement device based on a time of flight camera is introduced and calibrated using Lambertian targets with defined reflectance values at 10 % , 50 % , and 95 % . Reflectance measurements of 9 material subclasses are presented and 488 spectra from the NASA ECOSTRESS library are considered to evaluate the new measurement device. The parametrisation of the lidar capabilities is illustrated by presenting a lidar measurement campaign with a new Infineon lidar prototype and relevant data from 12 common lidar types.

Description: A computationally efficient, open-source feature-tracking algorithm, called ORB, is adopted and tuned for sea ice drift retrieval from Sentinel-1 SAR (Synthetic Aperture Radar) images. The most suitable setting and parameter values have been found using four Sentinel-1 image pairs representative of sea ice conditions between Greenland and Severnaya Zemlya during winter and spring. The performance of the algorithm is compared to two other feature-tracking algorithms, namely SIFT (Scale-Invariant Feature Transform) and SURF (Speeded-Up Robust Features). Having been applied to 43 test image pairs acquired over Fram Strait and the north-east of Greenland, the tuned ORB (Oriented FAST and Rotated BRIEF) algorithm produces the highest number of vectors (177 513, SIFT: 43 260 and SURF: 25 113), while being computationally most efficient (66 s, SIFT: 182 s and SURF: 99 s per image pair using a 2.7 GHz processor with 8 GB memory). For validation purposes, 314 manually drawn vectors have been compared with the closest calculated vectors, and the resulting root mean square error of ice drift is 563 m. All test image pairs show a significantly better performance of the HV (horizontal transmit, vertical receive) channel due to higher informativeness. On average, around four times as many vectors have been found using HV polarization. All software requirements necessary for applying the presented feature-tracking algorithm are open source to ensure a free and easy implementation.

Description: Synthetic Aperture Radar (SAR) data from RADARSAT-2 (RS2) in dual-polarization mode provide additional information for discriminating sea ice and open water compared to single-polarization data. We have developed an automatic algorithm based on dual-polarized RS2 SAR images to distinguish open water (rough and calm) and sea ice. Several technical issues inherent in RS2 data were solved in the pre-processing stage, including thermal noise reduction in HV polarization and correction of angular backscatter dependency in HH polarization. Texture features were explored and used in addition to supervised image classification based on the support vector machines (SVM) approach. The study was conducted in the ice-covered area between Greenland and Franz Josef Land. The algorithm has been trained using 24 RS2 scenes acquired in winter months in 2011 and 2012, and the results were validated against manually derived ice charts of the Norwegian Meteorological Institute. The algorithm was applied on a total of 2705 RS2 scenes obtained from 2013 to 2015, and the validation results showed that the average classification accuracy was 91 ± 4 %.

Description: An open-source sea ice drift algorithm for Sentinel-1 SAR imagery is introduced based on the combination of feature tracking and pattern matching. Feature tracking produces an initial drift estimate and limits the search area for the consecutive pattern matching, which provides small- to medium-scale drift adjustments and normalised cross-correlation values. The algorithm is designed to combine the two approaches in order to benefit from the respective advantages. The considered feature-tracking method allows for an efficient computation of the drift field and the resulting vectors show a high degree of independence in terms of position, length, direction and rotation. The considered pattern-matching method, on the other hand, allows better control over vector positioning and resolution. The preprocessing of the Sentinel-1 data has been adjusted to retrieve a feature distribution that depends less on SAR backscatter peak values. Applying the algorithm with the recommended parameter setting, sea ice drift retrieval with a vector spacing of 4 km on Sentinel-1 images covering 400 km  ×  400 km, takes about 4 min on a standard 2.7 GHz processor with 8 GB memory. The corresponding recommended patch size for the pattern-matching step that defines the final resolution of each drift vector is 34  ×  34 pixels (2.7  ×  2.7 km). To assess the potential performance after finding suitable search restrictions, calculated drift results from 246 Sentinel-1 image pairs have been compared to buoy GPS data, collected in 2015 between 15 January and 22 April and covering an area from 80.5 to 83.5° N and 12 to 27° E. We found a logarithmic normal distribution of the displacement difference with a median at 352.9 m using HV polarisation and 535.7 m using HH polarisation. All software requirements necessary for applying the presented sea ice drift algorithm are open-source to ensure free implementation and easy distribution.