Segment-then-Classify: Few-shot Instance Segmentation for Environmental Remote Sensing
Dec 15, 2023
Speakers
Kelly Caylor
Speaker · 0 followers
Anna Boser
Speaker · 0 followers
About
Instance segmentation is pivotal for environmental sciences and climate change research, facilitating important tasks from land cover classification to glacier monitoring. This paper addresses the prevailing challenges associated with data scarcity when using traditional models like YOLOv8 by introducing a novel, data-efficient workflow for instance segmentation. The proposed Segment-then-Classify (STC) strategy leverages the zero-shot capabilities of the novel Segment Anything Model (SAM) to segment all objects in an image and then uses a simple classifier such as the Vision Transformer (ViT) to identify objects of interest thereafter. Evaluated on the VHR-10 dataset, our approach demonstrated convergence with merely 40 examples per class. YOLOv8 requires 3 times as much data to achieve the STC's peak performance. The highest performing class in the VHR-10 dataset achieved a near-perfect mAP@0.5 of 0.99 using the STC strategy. However, performance varied greatly across other classes due to the SAM model’s occasional inability to recognize all relevant objects, indicating a need for refining the zero-shot segmentation step. The STC workflow therefore holds promise for advancing few-shot learning for instance segmentation in environmental science.Instance segmentation is pivotal for environmental sciences and climate change research, facilitating important tasks from land cover classification to glacier monitoring. This paper addresses the prevailing challenges associated with data scarcity when using traditional models like YOLOv8 by introducing a novel, data-efficient workflow for instance segmentation. The proposed Segment-then-Classify (STC) strategy leverages the zero-shot capabilities of the novel Segment Anything Model (SAM) to se…
Organizer
NeurIPS 2023
Account · 644 followers
Like the format? Trust SlidesLive to capture your next event!
Professional recording and live streaming, delivered globally.
Sharing
Recommended Videos
Presentations on similar topic, category or speaker
Efficient Few-Shot Clinical Task Adaptation with Large Language Model
Watch later
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Fast Bellman Updates for Wasserstein Distributionally Robust MDPs
Watch later
Zhuodong Yu, …
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Moment Matching Denoising Gibbs Sampling
Watch later
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Learning from Label Proportions Bootstrapping Supervised Learners via Belief Propagation
Watch later
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Likelihood Ratio Confidence Sets for Sequential Decision Making
Watch later
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
SimMTM: A Simple Pre-Training Framework for Masked Time-Series Modeling
Watch later
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%