Saving 100x Storage: Prototype Replay for Reconstructing Training Sample Distribution in Class-Incremental Semantic Segmentation
Dec 10, 2023
Speakers
Jinpeng Chen
Speaker · 0 followers
Runmin Cong
Speaker · 0 followers
Yuxuan Luo
Speaker · 0 followers
About
Existing class-incremental semantic segmentation (CISS) methods mainly tackle catastrophic forgetting and background shift, but often overlook another crucial issue. In CISS, each step focuses on different foreground classes, and the training set for a single step only includes images containing pixels of the current foreground classes, excluding images without them. This leads to an overrepresentation of these foreground classes in the single-step training set, causing the classification biased towards these classes. To address this issue, we present STAR, which preserves the main characteristics of each past class by storing a compact prototype and necessary statistical data, and aligns the class distribution of single-step training samples with the complete dataset by replaying these prototypes and repeating background pixels with appropriate frequency. Compared to the previous works that replay raw images, our method saves over 100 times the storage while achieving better performance. Moreover, STAR incorporates an old-class features maintaining (OCFM) loss, keeping old-class features unchanged while preserving sufficient plasticity for learning new classes. Furthermore, a similarity-aware discriminative (SAD) loss is employed to specifically enhance the feature diversity between similar old-new class pairs. Experiments on two public datasets, Pascal VOC 2012 and ADE20K, reveal that our model surpasses all previous state-of-the-art methods.Existing class-incremental semantic segmentation (CISS) methods mainly tackle catastrophic forgetting and background shift, but often overlook another crucial issue. In CISS, each step focuses on different foreground classes, and the training set for a single step only includes images containing pixels of the current foreground classes, excluding images without them. This leads to an overrepresentation of these foreground classes in the single-step training set, causing the classification biased…
Organizer
NeurIPS 2023
Account · 618 followers
Like the format? Trust SlidesLive to capture your next event!
Professional recording and live streaming, delivered globally.
Sharing
Recommended Videos
Presentations on similar topic, category or speaker
Invariant Anomaly Detection under Distribution Shifts: A Causal Perspective
Watch later
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
A sparse null code emerges in deep neural networks
Watch later
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Real-Time Motion Prediction via Heterogeneous Polyline Transformer with Relative Pose Encoding
Watch later
Zhejun Zhang, …
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Deep graph kernel point processes
Watch later
Zheng Dong, …
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Rotating Features for Object Discovery
Watch later
Sindy Lowe, …
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Understanding Opinions Towards Climate Change on Social Media
Watch later
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%