The seminar will survey work on computational models of various aspects of language learning. We will relate these lines of work to linguistic and psychological issues such as the dispute concerning the poverty of the stimulus argument (Pullum and Scholz), connectionists' views (Elman et al. "Rethinking Innateness"), and work on children's sensitivity to distributional tendencies (work by Newport, Mintz, and others).
Purpose: To explore the increasingly important literature on computational models of human language learning. The focus is on simulations of human language acquisition, where the aim is not simply accuracy, but rather similarity to human accuracy (including error), the order of acquisition, etc. Especially interesting is work which aspires to neurological plausibility. The present course aims to explore this work by reading and discussing published papers. Our primary purpose therefore does not include an examination of applications of machine learning in natural language processing, i.e., studies whose primary aim is to improve, e.g., parsing accuracy.
Example: Tim Dorscheidt, Nicola Valchev and Terence van Zoelen Minimal Generalization of Dutch Diminutives, project report on the 2006-2007 course.
Prerequisites The course assumes familiarity with basic concepts of machine learning, but there will be time to review occasionally unfamiliar concepts as well. The course is aimed at students in research masters' programs, which assumes serious motivation and scientific maturity.
Note This course is referred to as Computermodellen voor taalverwerving in the Dutch course catalogue.
|Docent||prof.dr.ir. J. Nerbonne|
|Literature||See literature section below|
|Organization||Seminars with student presentations as well as discussion and (perhaps) presentations by guest researchers. All students are expected to read each paper and participate in discussions, as well as present at least one paper. Instructor will make early presentations, and lead discussions for these sessions, but will not lecture after the first meetings.|
|Place||Tues. 15:30-17, A-weg 30, Rm 104|
|Credit||Credit based on: leading discussion on (1-2) papers; contributions to discussions; an implementation of a psycholinguistic hypothesis with report of results.|
|Information||J.Nerbonne at RuG dot nl|
|Nov 11||Introduction||None||J. Nerbonne [slides]|
|Nov 18||Categorial Grammars||None||C. Coltekin [slides]|
|Nov 25||Poverty of Stimulus||Pullum & Scholtz (2002)||J. Ma [slides]|
|Learning English Auxiliary Inversion||Clark (2006)||S. Duarte|
|Dec 2||Machine Learning & Universal Grammar||Lappin & Shieber (2007)||X. Yao|
|Dec 9||Project Discussion||None|
The following is an extended reading list for the course. We will be reading a selection from these papers. The same list organized by subject is also available as a pdf file.
Albright, A. and Hayes, B. (2002). Modeling English past tense intuitions with minimal generalization. In SIGPHON 6: Proceedings of the Sixth Meeting of the ACL Special Interest Group in Computational Phonology, pages 58-–69. [ bib | .pdf ]
Clark, A., Eyraud, R., and Habrard, A. (2008). A polynomial algorithm for the inference of context free languages. In Proceedings of International Colloquium on Grammatical Inference. Springer. [ bib | .pdf ]
Crystal, D. (1997). The Cambridge Encyclopedia of Language. Cambridge University Press. [ bib ]
Klein, D. and Manning, C. (2004). Corpus-based induction of syntactic structure: Models of dependency and constituency. In Proceedings of the Association for Computational Linguistics (ACL). [ bib | .pdf ]
Saffran, J. R., Aslin, R. N., and Newport, E. L. (1996). Statistical learning by 8-month old infants. Science, 274(5294):1926-1928. [ bib ]
Thompson, S. P. and Newport, E. L. (2007). Statistical learning of syntax: The role of transitional probability. Language Learning and Development, (3):1-42. [ bib ]
Tomasello, M. (2006). Acquiring linguistic constructions. In Kuhn, D. and Siegler, R. S., editors, Handbook of Child Psychology, pages 255-298. Wiley, New York. [ bib ]
Yang, C. D. (1999). A selectionist theory of language development. In Proceedings of 37th Meeting of the Association for Computational Linguistics, pages 429-435. Association for Computational Linguistics. [ bib | .pdf ]
Zettlemoyer, L. S. and Collins, M. (2005). Learning to map sentences to logical form: Structured classification with probabilistic categorial grammars. In Proceedings of the Twenty First Conference on Uncertainty in Artificial Intelligence (UAI-05). [ bib | .pdf ]
This year's project is on learning Categorial Grammars. The aim of the project is to develop an unsupervised Categorial Grammar learner. The details and scope of the project will be defined during the class discussions.