Title: Critical scenario identification for testing of autonomous driving systems

Speaker:  Qunying Song, WASP PhD Student, Department of Computer Science Lund University

When: 18 May at 12.00-13.15

Where: Online

Spoken language: English

Abstract

Testing of autonomous driving systems is a prerequisite to verify the safety and reliability of such systems. Current approaches for testing autonomous driving systems that rely on substantial real-world testing, or collecting real traffic data at scale, are considered both inefficient and ineffective as it is expensive, time-consuming, and may still not cover the rare-occurring traffic situations.
 
During the seminar, I will introduce an approach for testing autonomous driving systems based on critical scenario identification. Specifically, I will go through some tools and a workflow for generating critical test scenarios, and I will also demonstrate the effectiveness of the said approach using two real autonomous driving systems from industry.

References 

1. Ulbrich, S., Menzel, T., Reschka, A., Schuldt, F., Maurer, M.: Defining and substantiating the terms scene, situation, and scenario for automated driving. In: IEEE 18th International Conference on Intelligent Transportation Systems, pp. 982–988 (2015). IEEE
2. Menzel, T., Bagschik, G., Maurer, M.: Scenarios for development, test and validation of automated vehicles. In: IEEE Intelligent Vehicles Symposium (IV), pp. 1821–1827 (2018). IEEE
3. Bagschik, G., Menzel, T., Maurer, M.: Ontology based scene creation for the development of automated vehicles. In: IEEE Intelligent Vehicles Symposium (IV), pp. 1813–1820 (2018). IEEE
4. Scholtes, Maike, Lukas Westhofen, Lara Ruth Turner, Katrin Lotto, Michael Schuldes, Hendrik Weber, Nicolas Wagener et al. "6-layer model for a structured description and categorization of urban traffic and environment." IEEE Access 9 (2021).
5. Kluck, F., Zimmermann, M., Wotawa, F., Nica, M.: Genetic algorithm-based test parameter optimization for ADAS system testing. In: IEEE 19th International Conference on Software Quality, Reliability and Security (QRS), pp. 418–425 (2019). IEEE
6. ISO, ISO/PAS 21448:2019 Road vehicles — Safety of the intended functionality, 2019.
7. Simens, White paper: Scenario-based verification and validation of self-driving vehicles: relevant safety metrics, 2022.
8. Ponn, T., Breitfuß, M., Yu, X., Diermeyer, F.: Identification of challenging highway-scenarios for the safety validation of automated vehicles based on real driving data. In: 15th International Conference on Ecological Vehicles and Renewable Energies (EVER), pp. 1–10 (2020). IEEE
9. Gambi, A., Huynh, T., Fraser, G.: Generating effective test cases for selfdriving cars from police reports. In: Proceedings of the 27th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering, pp. 257–267 (2019)
10. Ding, W., Chen, B., Xu, M., Zhao, D.: Learning to collide: An adaptive safety-critical scenarios generating method. In: IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 2243–2250 (2020). IEEE
11. Klischat, M., Althoff, M.: Generating critical test scenarios for automated vehicles with evolutionary algorithms. In: IEEE Intelligent Vehicles Symposium (IV), pp. 2352–2358 (2019). IEEE
12. Gambi, A., Mueller, M., Fraser, G.: Automatically testing self-driving cars with search-based procedural content generation. In: Proceedings of the 28th ACM SIGSOFT International Symposium on Software Testing and Analysis, pp. 318–328 (2019)
13. Song, Qunying, Kaige Tan, Per Runeson, and Stefan Persson. "An Industrial Workbench for Test Scenario Identification for Autonomous Driving Software." In 2021 IEEE International Conference on Artificial Intelligence Testing (AITest), pp. 81-82. IEEE, 2021.
14. Song, Qunying, Kaige Tan, Per Runeson, and Stefan Persson. "Critical Scenario Identification for Realistic Testing of Autonomous Driving Systems." (2022).