Segment-then-Classify: Few-shot Instance Segmentation for Environmental Remote Sensing

Dec 15, 2023



Instance segmentation is pivotal for environmental sciences and climate change research, facilitating important tasks from land cover classification to glacier monitoring. This paper addresses the prevailing challenges associated with data scarcity when using traditional models like YOLOv8 by introducing a novel, data-efficient workflow for instance segmentation. The proposed Segment-then-Classify (STC) strategy leverages the zero-shot capabilities of the novel Segment Anything Model (SAM) to segment all objects in an image and then uses a simple classifier such as the Vision Transformer (ViT) to identify objects of interest thereafter. Evaluated on the VHR-10 dataset, our approach demonstrated convergence with merely 40 examples per class. YOLOv8 requires 3 times as much data to achieve the STC's peak performance. The highest performing class in the VHR-10 dataset achieved a near-perfect mAP@0.5 of 0.99 using the STC strategy. However, performance varied greatly across other classes due to the SAM model’s occasional inability to recognize all relevant objects, indicating a need for refining the zero-shot segmentation step. The STC workflow therefore holds promise for advancing few-shot learning for instance segmentation in environmental science.


Store presentation

Should this presentation be stored for 1000 years?

How do we store presentations

Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%


Recommended Videos

Presentations on similar topic, category or speaker

Interested in talks like this? Follow NeurIPS 2023