Senior Software Testing Consultant, STA Testing Consulting
Young-jae Choi is a software testing consultant with over 20 years’ experience in the IT industry leading various testing teams. He is currently leading the AI testing training and consulting division at STA Consulting in Korea. Young-jae has participated in various AI testing projects including facial recognition, emotion recognition, and voice recognition.
Young-jae supports the software testing community in a number of roles. He is a member of Korea’s ICT Standardization Committee in the field of software quality evaluation and is also a member of the ISO Software Testing Working Group, contributing as a co-editor for ISO/IEC 29119-16 Testing Machine Learning Systems. He has also participated in the development of the ISTQB CT-AI syllabus as part of the KSTQB & CSTQB AIT team
Testing AI based systems
AI-Based systems are being developed for various applications and are becoming more prevalent and critical to our daily lives. This talk will discuss how testing of AI based systems is different from that of traditional systems and the challenges it introduces. We will look at key concepts from the ISTQB CT-AI syllabus and ISO/IEC 29119-11 to show how quality can be achieved with these systems. We will also consider the range of approaches suitable for the black-box testing of AI-based systems, introducing, in some detail, the metamorphic testing technique. We will discuss how white-box testing can be applied to drive the testing and measure the test coverage of neural networks.
With more AI systems being used and as we get a better understanding of how to test these systems, it is becoming more evident that achieving quality of these systems requires more than just testing of the model itself. The input data, which is used to train these models and highly impacts the performance and reliability of the model must also be tested. In some cases, the framework used to create the models needs to be tested as well. We will briefly introduce the work being done by JWG2 to define common defect types found at each of these test levels and testing types that can be used to identify such defects.