TestRank: Bringing Order into Unlabeled Test Instances for Deep Learning Tasks
Dez 6, 2021
Sprecher:innen
Qiuxia Lai
Sprecher:in · 0 Follower:innen
Yannan Liu
Sprecher:in · 0 Follower:innen
Über
Deep learning (DL) systems are notoriously difficult to test and debug due to the lack of correctness proof and the huge test input space to cover. Given the ubiquitous unlabeled test data and high labeling cost, in this paper, we propose a novel test prioritization technique, namely TestRank, which aims at revealing more model failures with less labeling effort. TestRank brings order into the unlabeled test data according to their likelihood of being a failure, i.e., their failure-revealing capabilities. Different from existing solutions, TestRank leverages both intrinsic and contextual attributes of the unlabeled test data when prioritizing them. To be specific, we first build a similarity graph on both unlabeled test samples and labeled samples (e.g., training or previously labeled test samples). Then, we conduct graph-based semi-supervised learning to extract contextual features from the correctness of similar labeled samples. For a particular test instance, the contextual features extracted with the graph neural network and the intrinsic features obtained with the DL model itself are combined to predict its failure-revealing capability. Finally, TestRank prioritizes unlabeled test inputs in descending order of the above probability value. We evaluate TestRank on three popular image classification datasets, and results show that TestRank significantly outperforms existing test prioritization techniques.Deep learning (DL) systems are notoriously difficult to test and debug due to the lack of correctness proof and the huge test input space to cover. Given the ubiquitous unlabeled test data and high labeling cost, in this paper, we propose a novel test prioritization technique, namely TestRank, which aims at revealing more model failures with less labeling effort. TestRank brings order into the unlabeled test data according to their likelihood of being a failure, i.e., their failure-revealing cap…
Organisator
NeurIPS 2021
Konto · 1,9k Follower:innen
Über NeurIPS 2021
Neural Information Processing Systems (NeurIPS) is a multi-track machine learning and computational neuroscience conference that includes invited talks, demonstrations, symposia and oral and poster presentations of refereed papers. Following the conference, there are workshops which provide a less formal setting.
Gefällt euch das Format? Vertraut auf SlidesLive, um euer nächstes Event festzuhalten!
Professionelle Aufzeichnung und Livestreaming – weltweit.
Freigeben
Empfohlene Videos
Präsentationen, deren Thema, Kategorie oder Sprecher:in ähnlich sind
An Automated System for Detecting Visual Damages of Wind Turbine Blades
Später ansehen
Linh Nguyen, …
Ewigspeicher-Fortschrittswert: 0 = 0.0%
Closing Remarks
Später ansehen
Ewigspeicher-Fortschrittswert: 0 = 0.0%
URLB: Unsupervised Reinforcement Learning Benchmark
Später ansehen
Ewigspeicher-Fortschrittswert: 0 = 0.0%
Gaussian Dropout as an Information Bottleneck Layer
Später ansehen
Melanie Ray, …
Ewigspeicher-Fortschrittswert: 0 = 0.0%
RadGraph: Extracting Clinical Entities and Relations from Radiology Reports
Später ansehen
Saahil Jain, …
Ewigspeicher-Fortschrittswert: 0 = 0.0%
Machine Learning adn PDEs
Später ansehen
Ewigspeicher-Fortschrittswert: 0 = 0.0%