ABSTRACT
To illustrate how to use this framework to evaluate research validity, we consider four studies that vary in the approach used but have a number of features in common: They all were published in peer-reviewed journals, made a reasonable case for the research questions and methods based on literature and theory, and interpreted the results with appropriate caution given the strengths and weaknesses of the methods used. Thus, questions 17–19 would not produce major differences in the evaluations. However, there were differences in the strength of various aspects of the design and methods as discussed next. We evaluate each of the four studies in turn using global judgments and narratives to rate each of the eight research validity scales described in chapter 37. In the appendixes, we provide a proposal for how one might do numerical ratings of the research validity of the four studies. Appendix A is a Research Validity Evaluation Form that could be used to make and record numerical ratings of the eight dimensions. Appendix B is a completed evaluation form, and appendix C presents side-by-side numerical ratings and comments about the strengths and weaknesses of each study.
