Generalized Shape Metrics on Neural Representations
Dec 6, 2021
Speakers
Alex Williams
Řečník · 0 sledujících
Simon Kornblith
Řečník · 1 sledující
Scott Linderman
Řečník · 0 sledujících
About
Understanding the operation of biological and artificial networks remains a difficult and important challenge. To identify general principles of neural computation, researchers are increasingly interested in surveying large collections of networks that are trained on, or biologically adapted to, similar tasks. A standardized set of analysis tools is now needed to identify how network-level covariates—such as architecture, learning algorithm, anatomical brain region, model organism, etc.—systematically impact neural representations (hidden layer activations). Here, we provide a rigorous foundation for these analyses by defining a broad family of metric spaces that quantify representational dissimilarity between networks. Using this framework, we modify existing measures based on canonical correlation analysis and centered kernel alignment to satisfy the triangle inequality, formulate a novel metric that respects the inductive biases in convolutional layers, and identify approximate Euclidean embeddings that enable network representations to be incorporated into essentially any off-the-shelf machine learning method. We demonstrate these methods on large-scale datasets from biology (Allen Institute Brain Observatory) and deep learning (NAS-Bench-101), and identify relationships between neural representations that are interpretable in terms of anatomical hierarchies, network architecture, and model performance.Understanding the operation of biological and artificial networks remains a difficult and important challenge. To identify general principles of neural computation, researchers are increasingly interested in surveying large collections of networks that are trained on, or biologically adapted to, similar tasks. A standardized set of analysis tools is now needed to identify how network-level covariates—such as architecture, learning algorithm, anatomical brain region, model organism, etc.—systemat…
Organizer
NeurIPS 2021
Účet · 1,9k sledujících
About NeurIPS 2021
Neural Information Processing Systems (NeurIPS) is a multi-track machine learning and computational neuroscience conference that includes invited talks, demonstrations, symposia and oral and poster presentations of refereed papers. Following the conference, there are workshops which provide a less formal setting.
Like the format? Trust SlidesLive to capture your next event!
Professional recording and live streaming, delivered globally.
Sharing
Recommended Videos
Presentations on similar topic, category or speaker
Discovering Dynamical Parameters by Interpreting Echo State Networks
Zhlédnout později
Pro uložení prezentace do věčného trezoru hlasovalo 0 diváků, což je 0.0 %
Safe RL Debate
Zhlédnout později
Animesh Garg, …
Pro uložení prezentace do věčného trezoru hlasovalo 0 diváků, což je 0.0 %
Panel I: Human decisions
Zhlédnout později
Pro uložení prezentace do věčného trezoru hlasovalo 0 diváků, což je 0.0 %
L2ight: Enabling On-Chip Learning for Optical Neural Networks via Efficient in-situ Subspace Optimization
Zhlédnout později
Jiaqi Gu, …
Pro uložení prezentace do věčného trezoru hlasovalo 0 diváků, což je 0.0 %
PyHard: a novel tool for generating hardness embeddings to support data-centric analysis
Zhlédnout později
Pedro Paiva, …
Pro uložení prezentace do věčného trezoru hlasovalo 0 diváků, což je 0.0 %
A Higher Purpose: Measuring Electricity Access Using High-Resolution Daytime Satellite Imagery
Zhlédnout později
Zeal Shah, …
Pro uložení prezentace do věčného trezoru hlasovalo 0 diváků, což je 0.0 %