Hypergraphs are important objects to model ternary or higher-order relations of objects, and have a number of applications in analysing many complex datasets occurring in practice. In this work we study a new heat diffusion process in hypergraphs, and employ this process to design a polynomial-time algorithm that approximately finds bipartite components in a hypergraph. We theoretically prove the performance of our proposed algorithm, and compare it against the previous state-of-the-art through extensive experimental analysis. The significance of our work is further demonstrated on several large-scale datasets (Penn Treebank, DBLP, IMDB, and Wikipedia), in which our unsupervised algorithm clearly separates objects of different types (e.g., verbs vs. adverbs, and authors vs. conferences).