tailieunhanh - Báo cáo khoa học: "Hierarchical Non-Emitting Markov Models"

We describe a simple variant of the interpolated Markov model with non-emitting state transitions and prove that it is strictly more powerful than any Markov model. Empirical results demonstrate that the non-emitting model outperforms the interpolated model on the Brown corpus and on the Wall Street Journal under a wide range of experimental conditions. The nonemitting model is also much less prone to overtraining. The remainder of our article consists of four sections. In section 2, we review the interpolated Markov model and briefly demonstrate that all interpolated models are equivalent to some basic Markov model of the same. | Hierarchical Non-Emitting Markov Models Eric Sven Ristad and Robert G. Thomas Department of Computer Science Princeton University Princeton NJ 08544-2087 ristad rgt @ Abstract We describe a simple variant of the interpolated Markov model with non-emitting state transitions and prove that it is strictly more powerful than any Markov model. Empirical results demonstrate that the non-emitting model outperforms the interpolated model on the Brown corpus and on the Wall Street Journal under a wide range of experimental conditions. The nonemitting model is also much less prone to overtraining. 1 Introduction The Markov model has long been the core technology of statistical language modeling. Many other models have been proposed but none has offered a better combination of predictive performance computational efficiency and ease of implementation. Here we add hierarchical non-emitting state transitions to the Markov model. Although the states in our model remain Markovian the model itself is no longer Markovian because it can represent unbounded dependencies in the state distribution. Consequently the non-emitting Markov model is strictly more powerful than any Markov model including the context model Rissanen 1983 Rissa-nen 1986 the backoff model Cleary and Witten 1984 Katz 1987 and the interpolated Markov model Jelinek and Mercer 1980 MacKay and Peto 1994 . More importantly the non-emitting model consistently outperforms the interpolated Markov model on natural language texts under a wide range of experimental conditions. We believe that the superior performance of the non-emitting model is due to its ability to better model conditional independence. Thus the non-emitting model is better able to represent both conditional independence and long-distance dependence ie. it is simply a better statistical model. The non-emitting model is also nearly cis computationally efficient and easy to implement as the interpolated model. The remainder of our article .