Bayesian inference [Ling 300]

Klinton Bicknell /// Spring 2016

Bayesian techniques provide a rational way of combining probabilistic information, enabling sophisticated analyses of scientific datasets and principled models of action under uncertainty. This course introduces Bayesian techniques from the ground up.

Schedule Piazza forum

Schedule

Week	Date	Topic	Reading	Materials	Assignments
1	Mar 31	Introduction, basic probability theory	DBDA 1
2	Apr 5, 7	R, more probability theory, Bayes' rule	DBDA 4–5		hw1 out Apr 8
3	Apr 12, 14	More R, models for binary outcomes, exact inference	DBDA 6		hw1 due Apr 15
4	Apr 19, 21	Markov chain Monte Carlo (MCMC), JAGS	DBDA 7–8		hw2 out Apr 22
5	Apr 26, 28	Hierarchical models, more MCMC	DBDA 8–10		hw2 due Apr 29
6	May 3, 5	Gaussians, linear models, mixture models	DBDA 15–16		hw3 out May 6; Project proposal due May 6
7	May 10, 12	Linear regression with continuous predictors	DBDA 17–18		hw3 due May 13 (NOW May 17)
8	May 17, 19	Mixed-effects regression	DBDA 19–21	regression code	hw4 out May 20
9	May 24, 26	Logistic regression; Bayesian vs. frequentist inference	DBDA 11		hw4 due May 27 NOW MAY 30
10	May 31	Computational psycholinguistics
	Jun 7				Final project papers due 5pm

Logistics

Course

Time: Tuesdays & Thursdays 9:30–10:50
Location: University Library B182
Textbook: Doing Bayesian Data Analysis, Second Edition (2014) by John Kruschke [DBDA]
Website: kbicknell.github.io/ling300spring2016/

Instructor

Name: Klinton Bicknell
Office hours: Tuesdays 2–3 and by appointment
Office: Linguistics [2016 Sheridan Road] Office 107

Policies

Email: Questions that are not personal should be posted on the Piazza forum (where they can be posted anonymously if desired). To contact the instructor directly, coming to office hours is encouraged. For questions that are personal, students can email the instructor at kbicknell@northwestern.edu.
Description: Hands-on introduction to probability theory, structured probabilistic models, and Bayesian inference, in order to prepare students both to be able to perform sophisticated analyses of data and to construct and evaluate formal computational models instantiating scientific theories, all in the context of the study of language. Students will also learn to use the R language for statistical computing.
Academic integrity: Violations of academic integrity will be referred to the Dean’s office, per WCAS policies. Sanctions can be quite severe, including suspension or permanent expulsion from the university. For details and discussion of how to avoid plagiarism, see the Academic Integrity section of the WCAS undergraduate handbook.

Requirements

Course Grade

70% homeworks (4)
30% final project

Homeworks

There will be four homework assignments throughout the quarter. These assignments will involve a combination of programming exercises and short answer responses. Working together in pairs or small groups when discussing the assignments is encouraged, but each student must code and write up their own homework independently. In addition, students must list on each assignment all students they discussed the assignment with. Homework must be handed in through Canvas.

Final project

Students will complete a final project on a topic related to the course content. There are two possible types of final projects. In one type, the project will involve an in-depth statistical analysis of a dataset using Bayesian techniques. In the other type, the project will involve designing and implementing a probabilistic Bayesian model of a probabilistic inference problem facing some agent (for example, an aspect of language processing or acquisition) and using techniques from the class to determine some of the model's predictions. These projects should be completed in pairs or individually. Students will write short project proposals by the end of week 6, and then a final paper on the project will be due on the first day of finals week.

Keeping up

The syllabus (topics, assignments, due dates) may change. These changes will be announced in class, over email (via Piazza), and on the course website. It is students' responsibility to keep up with them.

Deadlines

All assignments are due at 5pm. For late work turned in between this deadline and 11:59pm the following day (i.e., the first 31 hours after the deadline), I will deduct one percentage point per hour (or partial hour). After the following day, I will give comments and suggestions on work turned in, but you will not receive credit for the assignment. (Of course, if some unusual external circumstance arises which will cause you have trouble meeting a deadline, please contact the instructor as soon as possible.)

AccessibleNU

Any student requesting accommodations related to a disability or other condition is required to register with AccessibleNU (accessiblenu@northwestern.edu; 847-467-5530) and provide professors with an accommodation notification from AccessibleNU, preferably within the first two weeks of class. All information will remain confidential.