Existing contrastive learning methods rely on pairwise sample contrast z_x^⊤ z_x' to learn data representations, but the learned features often lack clear interpretability from a human perspective. Theoretically, it lacks feature identifiability and different initialization may lead to totally different features.In this paper, we study a new method named tri-contrastive learning (triCL) that involves a 3-factor contrast in the form of z_x^⊤ S z_x', where S= diag(s_1,...,s_k) is a learnable diago…