ZIN: When and How to Learn Invariance Without Environment Partition?
Dec 6, 2022
Speakers
About
It is commonplace to encounter heterogeneous data, of which some aspects of the data distribution may vary but the underlying causal mechanisms remain constant. When data are divided into distinct environments according to the heterogeneity, recent invariant learning methods have proposed to learn robust and invariant models based on this environment partition. It is hence tempting to utilize the inherent heterogeneity even when environment partition is not provided. Unfortunately, in this work, we show that learning invariant features under this circumstance is fundamentally impossible without further inductive biases or additional information. Then, we propose a framework to jointly learn environment partition and invariant representation, assisted by additional auxiliary information. We derive sufficient and necessary conditions for our framework to provably identify invariant features under a fairly general setting. Experimental results on both synthetic and real world datasets validate our analysis and demonstrate an improved performance of the proposed framework over existing methods. Finally, our results also raise the need of making the role of inductive biases more explicit in future works, when considering learning invariant models without environment partition.It is commonplace to encounter heterogeneous data, of which some aspects of the data distribution may vary but the underlying causal mechanisms remain constant. When data are divided into distinct environments according to the heterogeneity, recent invariant learning methods have proposed to learn robust and invariant models based on this environment partition. It is hence tempting to utilize the inherent heterogeneity even when environment partition is not provided. Unfortunately, in this work,…
Organizer
NeurIPS 2022
Account · 961 followers
Like the format? Trust SlidesLive to capture your next event!
Professional recording and live streaming, delivered globally.
Sharing
Recommended Videos
Presentations on similar topic, category or speaker
zPROBE: Zero Peek Robustness Checks for Federated Learning
Watch later
Zahra Ghodsi, …
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
On Batch Teaching with Sample Complexity Bounded by VCD
Watch later
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Length Generalization in Quantitative Reasoning
Watch later
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Post-Hoc Attribute-Based Explanations for Recommender Systems
Watch later
Sahil Verma, …
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
NAS-Bench-Graph: Benchmarking Graph Neural Architecture Search
Watch later
Yijian Qin, …
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Quantum algorithms for sampling log-concave distributions and estimating normalizing constants
Watch later
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%