Paired feature constraints for latent dirichlet topic models

Abstract

Non Parametric Bayes models, so called family of Latent Dirichlet Allocation (LDA) Topic Models have found application in various aspects of pattern recognition like sentiment analysis, information retrieval, question answering etc. The topics induced by LDA are used for later tasks such as classification, regression(movie ratings), ranking and recommendation. Recently various approaches are suggested to improve the utility of topics induced by LDA using various side-information such as labeled examples and labeled features. Pair-Wise feature constraints such as cannot-link and must-link, represent weak-supervision and are prevalent in domains such as sentiment analysis. Though must-link constraints are relatively easier to incorporate by using dirichlet tree, the cannot-link constraints are harder to incorporate using the dirichlet forest. In this paper we proposed an approach to address this problem using posterior constraints. We introduced additional latent variables for capturing the constraints, and modified the gibbs sampling algorithm to incorporate these constraints. Our method of Posterior Regularization has enabled us to deal with both types of constraints seamlessly in the same optimization framework. We have demonstrated our approach on a product sentiment review data set which is typically used in text analysis.