In this paper, we consider the problem of sequential change-point detection where both the change-points and the distributions before and after the change are assumed to be unknown. For this key problem in statistical and sequential learning theory, we derive a variant of the Bayesian Online Change Point Detector proposed by <cit.> which is easier to analyze than the original version while keeping its powerful message-passing algorithm. We provide a non-asymptotic analysis of the false-alarm rate and the detection delay that matches the existing lower-bound. We further provide the first explicit high-probability control of the detection delay for such approach. Experiments on synthetic and real-world data show that this proposal compares favorably with the state-of-art change-point detection strategy, namely the Improved Generalized Likelihood Ratio (Improved GLR) while outperforming the original Bayesian Online Change Point Detection strategy.