tailieunhanh - Báo cáo khoa học: "The Infinite Tree"

Historically, unsupervised learning techniques have lacked a principled technique for selecting the number of unseen components. Research into non-parametric priors, such as the Dirichlet process, has enabled instead the use of infinite models, in which the number of hidden categories is not fixed, but can grow with the amount of training data. Here we develop the infinite tree, a new infinite model capable of representing recursive branching structure over an arbitrarily large set of hidden categories. Specifically, we develop three infinite tree models, each of which enforces different independence assumptions, and for each model we define a simple direct. | The Infinite Tree Jenny Rose Finkel Trond Grenager and Christopher D. Manning Computer Science Department Stanford University Stanford CA 94305 jrfinkel grenager manning @ Abstract Historically unsupervised learning techniques have lacked a principled technique for selecting the number of unseen components. Research into non-parametric priors such as the Dirichlet process has enabled instead the use of infinite models in which the number of hidden categories is not fixed but can grow with the amount of training data. Here we develop the infinite tree a new infinite model capable of representing recursive branching structure over an arbitrarily large set of hidden categories. Specifically we develop three infinite tree models each of which enforces different independence assumptions and for each model we define a simple direct assignment sampling inference procedure. We demonstrate the utility of our models by doing unsupervised learning of part-of-speech tags from treebank dependency skeleton structure achieving an accuracy of and by doing unsupervised splitting of part-of-speech tags which increases the accuracy of a generative dependency parser from to . 1 Introduction Model-based unsupervised learning techniques have historically lacked good methods for choosing the number of unseen components. For example k-means or EM clustering require advance specification of the number of mixture components. But the introduction of nonparametric priors such as the Dirichletprocess Ferguson 1973 enabled development of infinite mixture models in which the number of hidden components is not fixed but emerges naturally from the training data Antoniak 1974 . 272 Teh et al. 2006 proposed the hierarchical Dirichlet process HDP as a way of applying the Dirichlet process DP to more complex model forms so as to allow multiple group-specific infinite mixture models to share their mixture components. The closely related infinite hidden Markov model is .