Naïve discriminative learning: From Rescorla-Wagner to machine learning in natural language processing

21.06.2017 - 17:00
21.06.2017 - 19:00
Lecturer

Prof. Dr. Stefan Evert

Friedrich-Alexander Universität Erlangen-Nürnberg

Associative learning according to the Rescorla-Wagner (1972) equations has recently attracted renewed interest in the fields of computational linguistics and psycholinguistics under the label Naïve Discriminative Learning (NDL, Baayen et al. 2011). Despite its focus on simple cue-outcome associations, researchers found NDL to be competitive with more sophisticated machine learning approaches and cognitive models in a range of tasks.This talk gives a clear and consistent account of the NDL learning procedure and shows its intimate connection to least-squares regression, single-layer neural networks and linear classification problems (based on Evert & Arppe 2015). While most of these results are not new, they are scattered across decades of research on associative learning, are presented in different notation and often stated without a mathematical derivation. I will also discuss the central role that linear classification methods play for a wide range of applications in natural language processing (NLP).
REFERENCES: Baayen, R. H., Milin, P., Filipovic Durdevic, D., Hendrix, P. and Marelli, M. (2011). An amorphous model for morphological processing in visual comprehension based on naive discriminative learning. Psychological Review, 118, 438-482.Evert, S. and Arppe, A. (2015). Some theoretical and experimental observations on naïve discriminative learning. In Proceedings of the 6th Conference on Quantitative Investigations in Theoretical Linguistics (QITL-6), Tübingen, Germany.Rescorla, R. A. and Wagner, A. R. (1972). A theory of Pavlovian conditioning: Variations in the effectiveness of reinforcement and nonreinforcement. In Black, A. H., and Prokasy, W. F. (Eds.). Classical conditioning II: Current research and theory, 64-99. New York: Appleton-Century-Crofts.