Bayesian coresets have emerged as a promisingapproach for implementing scalable Bayesianinference. The Bayesian coreset problem in-volves selecting a (weighted) subset of the datasamples, such that the posterior inference us-ing the selected subset closely approximatesthe posterior inference using the full dataset.This manuscript revisits Bayesian coresetsthrough the lens of sparsity constrained opti-mization. Leveraging recent advances in ac-celerated optimization methods, we proposeand analyze a novel algorithm for coreset se-lection. We provide explicit convergence rateguarantees and present an empirical evalua-tion on a variety of benchmark datasets tohighlight our proposed algorithm’s superiorperformance compared to state-of-the-art onspeed and accuracy.