HSVA: Hierarchical Semantic-Visual Adaptation for Zero-Shot Learning
Dec 6, 2021
Speakers
Shiming Chen
Speaker · 0 followers
Guo-Sen Xie
Speaker · 0 followers
Qingmu Peng
Speaker · 0 followers
Baigui Sun
Speaker · 0 followers
About
Zero-shot learning (ZSL) tackles the unseen class recognition problem, transferring semantic knowledge from seen classes to unseen ones. Typically, to guarantee desirable knowledge transfer, a common (latent) space is adopted for associating the visual and semantic domains in ZSL. However, existing common space learning methods align the semantic and visual domains by merely mitigating distribution disagreement through one-step adaptation. This strategy is usually ineffective due to the heterogeneous nature of the feature representations in the two domains, which intrinsically contain both distribution and structure variations. To address this and advance ZSL, we propose a novel hierarchical semantic-visual adaptation (HSVA) framework. Specifically, HSVA aligns the semantic and visual domains by adopting a hierarchical two-step adaptation, i.e., structure adaptation and distribution adaptation. In the structure adaptation step, we take two task-specific encoders to encode the source data (visual domain) and the target data (semantic domain) into a structure-aligned common space. To this end, a supervised adversarial discrepancy (SAD) module is proposed to adversarially minimize the discrepancy between the predictions of two task-specific classifiers, thus making the visual and semantic feature manifolds more closely aligned. In the distribution adaptation step, we directly minimize the Wasserstein distance between the latent multivariate Gaussian distributions to align the visual and semantic distributions using a common encoder. Finally, the structure and distribution adaptation are derived in a unified framework under two partially-aligned variational autoencoders. Extensive experiments on four benchmark datasets demonstrate that HSVA achieves superior performance on both conventional and generalized ZSL. The code is available at <https://github.com/anonymou-ssubmition/HSVA-NeurIPS-21>.Zero-shot learning (ZSL) tackles the unseen class recognition problem, transferring semantic knowledge from seen classes to unseen ones. Typically, to guarantee desirable knowledge transfer, a common (latent) space is adopted for associating the visual and semantic domains in ZSL. However, existing common space learning methods align the semantic and visual domains by merely mitigating distribution disagreement through one-step adaptation. This strategy is usually ineffective due to the heterog…
Organizer
NeurIPS 2021
Account · 1.9k followers
About NeurIPS 2021
Neural Information Processing Systems (NeurIPS) is a multi-track machine learning and computational neuroscience conference that includes invited talks, demonstrations, symposia and oral and poster presentations of refereed papers. Following the conference, there are workshops which provide a less formal setting.
Like the format? Trust SlidesLive to capture your next event!
Professional recording and live streaming, delivered globally.
Sharing
Recommended Videos
Presentations on similar topic, category or speaker
Free-form Grading of Math Assignments: A case study in collaboration with Art of Problem Solving
Watch later
Yuri Burda, …
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Beltrami Flow and Neural Diffusion on Graphs
Watch later
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Estimating the Long-Term Effects of Novel Treatments
Watch later
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
A Family of Cognitively Realistic Parsing Environments for Deep Reinforcement Learning
Watch later
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Cardinality-Regularized Hawkes-Granger Model
Watch later
Tsuyoshi Idé, …
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Affordance Learning from Play for Sample-Efficient Policy Learning
Watch later
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%