tailieunhanh - Báo cáo khoa học: "Deep Learning for NLP"

Machine learning is everywhere in today’s NLP, but by and large machine learning amounts to numerical optimization of weights for human designed representations and features. The goal of deep learning is to explore how computers can take advantage of data to develop features and representations appropriate for complex interpretation tasks. | Deep Learning for NLP without Magic Richard Socher Yoshua Bengio Christopher D. Manning richard@ bengioy@ manning@ Computer Science Department Stanford University DIRO Universite de Montreal Montreal QC Canada 1 Abtract Machine learning is everywhere in today s NLP but by and large machine learning amounts to numerical optimization of weights for human designed representations and features. The goal of deep learning is to explore how computers can take advantage of data to develop features and representations appropriate for complex interpretation tasks. This tutorial aims to cover the basic motivation ideas models and learning algorithms in deep learning for natural language processing. Recently these methods have been shown to perform very well on various NLP tasks such as language modeling POS tagging named entity recognition sentiment analysis and paraphrase detection among others. The most attractive quality of these techniques is that they can perform well without any external hand-designed resources or time-intensive feature engineering. Despite these advantages many researchers in NLP are not familiar with these methods. Our focus is on insight and understanding using graphical illustrations and simple intuitive derivations. The goal of the tutorial is to make the inner workings of these techniques transparent intuitive and their results interpretable rather than black boxes labeled magic here . The first part of the tutorial presents the basics of neural networks neural word vectors several simple models based on local windows and the math and algorithms of training via backpropagation. In this section applications include language modeling and POS tagging. In the second section we present recursive neural networks which can learn structured tree outputs as well as vector representations for phrases and sentences. We cover both equations as well as applications. We show how training can be achieved by a 5 modified .