Scalable Rule-Based Representation Learning for Interpretable Classification
Dec 6, 2021
Speakers
About
Rule-based models, e.g., decision trees, are widely used in scenarios demanding high model interpretability for their transparent inner structures and good model expressivity. However, rule-based models are hard to optimize, especially on large data sets, due to their discrete parameters and structures. Ensemble methods and fuzzy/soft rules are commonly used to improve performance, but they sacrifice the model interpretability. To obtain both good scalability and interpretability, we propose a new classifier, named Rule-based Representation Learner (RRL), that automatically learns interpretable non-fuzzy rules for data representation and classification. To train the non-differentiable RRL effectively, we project it to a continuous space and propose a novel training method, called Gradient Grafting, that can directly optimize the discrete model using gradient descent. An improved design of logical activation functions is also devised to increase the scalability of RRL and enable it to discretize the continuous features end-to-end. Exhaustive experiments on nine small and four large data sets show that RRL outperforms the competitive interpretable approaches and can be easily adjusted to obtain a trade-off between classification accuracy and model complexity for different scenarios.Rule-based models, e.g., decision trees, are widely used in scenarios demanding high model interpretability for their transparent inner structures and good model expressivity. However, rule-based models are hard to optimize, especially on large data sets, due to their discrete parameters and structures. Ensemble methods and fuzzy/soft rules are commonly used to improve performance, but they sacrifice the model interpretability. To obtain both good scalability and interpretability, we propose a n…
Organizer
NeurIPS 2021
Account · 1.9k followers
About NeurIPS 2021
Neural Information Processing Systems (NeurIPS) is a multi-track machine learning and computational neuroscience conference that includes invited talks, demonstrations, symposia and oral and poster presentations of refereed papers. Following the conference, there are workshops which provide a less formal setting.
Like the format? Trust SlidesLive to capture your next event!
Professional recording and live streaming, delivered globally.
Sharing
Recommended Videos
Presentations on similar topic, category or speaker
Adversarial Robustness with Semi-Infinite Constrained Learning
Watch later
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Does Thermal data make the detection systems more reliable?
Watch later
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Particle Dynamics for Learning EBMs
Watch later
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Generalization Bounds for (Wasserstein) Robust Optimization
Watch later
Yang An, …
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Identifying Distributional Differences in Convective Evolution Prior to Rapid Intensification in Tropical Cyclones
Watch later
Trey McNeely, …
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Process for Adapting Language Models to Society (PALMS) with Values-Targeted Datasets
Watch later
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%