This 4-day course will highlight the main causes of the replication crisis, such as questionable research practices and publication bias. Subsequently, it will be elaborated and practiced [this is an applied course] how these causes can be addressed by the evaluation of (informative) hypotheses employing the Bayes factor and a generalization of the AIC. There will be attention for replication research and combining evidence from multiple studies.
The first day of the course will start with an elaboration of null-hypothesis significance testing. Subsequently, it will be elaborated why the omnipresent alpha level of .05 leads to questionable research practices, publication bias, and the replication crisis. This day will be concluded with a hand-on introduction to R and the packages Bain and Goric.
The second day of the course will start with an introduction of informative hypotheses like, for example, H1: mu1 > mu2 > mu3, that is, an ordering of three means. Subsequently, it will be elaborated how the Bayes factor can be used to evaluate null, alternative, and informative hypotheses. There will be attention for Bayesian error probabilities and Bayesian updating. This day will be concluded with examples of replication research and combining evidence from multiple studies in which informative hypotheses and Bayes factor play an important role.
The third day of the course will start with an introduction of information criteria (i.e., AIC and the generalization called the GORIC) and a brief repetition of informative hypotheses. Subsequently, it will be elaborated how the GORIC can be used to evaluate null, alternative, and informative hypotheses. There will be attention for GORIC weights, which have a nice interpretation. This day will, like Day 2, be concluded with examples of replication research and combining evidence from multiple studies in which informative hypotheses play an important role.
The fourth day is a lab-meeting. This day you will analyse data using i) null-hypothesis significance testing and the evaluation of informative hypotheses using ii) Bayes factor and iii) a generalization of the AIC (i.e. GORIC). You will make comparisons between these three approaches to understand their strengths and weaknesses. You will also evaluate a replication study and combine the evidence from multiple data sets using informative hypotheses. This day will be concluded with a discussion.
1) Kuiper, R.M. & Hoijtink, H.J.A. (2010). Comparisons of Means Using Exploratory and Confirmatory Approaches. Psychological Methods, 15 (1), (pp. 69-86).
2) Kuiper, R.M., Buskens, V.W., Raub, W. & Hoijtink, H. (2013). Combining statistical evidence from several studies: A method using bayesian updating and an example from research on trust problems in social and economic exchange. Sociological Methods and Research, 42 (1), (pp. 60-81).
3) Bain Tutorial, which is part of the bain zip file downloadable from https://informative-hypotheses.sites.uu.nl/software/bain/
4) goric/restriktor tutorial: http://www.restriktor.ugent.be/tutorial/example6.html
Prof. Dr. Herbert Hoijtink and Dr. Rebecca Kuiper
Tuition fee for PhD candidates from the Faculty of Social and Behavioural Sciences from Utrecht University will be funded by the Graduate School of Social and Behavioural Sciences.
Irene de Bruijne firstname.lastname@example.org