Mastering the IB Psychology curriculum involves becoming proficient in various skills, which include how to plan for exam study, answering Short Answer Questions (SAQs) and evaluating quantitative studies in Extended Response Questions (ERQs). Now that you've got your IB Psychology exam study plan sorted and know how to answer SAQs (read part 1 and part 2 of this blog series), understanding how to evaluate quantitative studies in ERQs is the next step in your learning journey. This blog intends to provide a step-by-step guide of how to do so.
Why We Need to Evaluate Studies
When writing about psychological studies, it is crucial that we evaluate them critically after describing their procedures and findings. There are two key reasons we need to evaluate studies:
- It demonstrates critical thinking, one criterion used in ERQ marking. By evaluating the strengths and limitations of a study, you show that you can think deeply about research methodology.
- It acknowledges that all studies have flaws and may excel in one aspect over another. For instance, studies often prioritise either internal validity (soundness of causal conclusions) or external validity (generalizability to real-world contexts). An experiment may have excellent control of variables to determine cause-and-effect (high internal validity), but use an artificial lab setting that lacks ecological validity (lower external validity). By evaluating studies along these different dimensions, we recognise these compromises in the research process while still appreciating the purpose of the design.
What Is an Evaluation?
An evaluation is a short paragraph that makes around 3-4 points and provides a balanced commentary on the methodology of a study. The goal is not just to praise or critique the study but to offer insight into how the experimental design impacts our interpretation of the findings.
There are five evaluation points which we can use to identify strengths and limitations in quantitative studies, which can be remembered using the acronym “GRAVE”:
G = Generalisability
R = Reliability
A = Applications
V = Validity
E = Ethics
What Is GRAVE?
Generalisability refers to the extent we can apply the findings to real-world contexts beyond the specific study sample. For example, a study conducted on all male university students has limited generalisability across females or other age groups.
Reliability is the consistency of a measure or experiment across time and participants. Using standardized procedures improves reliability. We can also say that a study’s reliability is a strength if its findings replicate those of prior studies investigating the same phenomenon.
Applications refer to how useful the implications of the study are for the wider field. For example, how could a study on long-term memory influence the field of education and the way we learn at school?
Validity concerns how accurately the study measures what it aimed to measure. Important types are internal validity (can we infer causality between the variables) and external validity (can we generalize the findings beyond the study context – i.e., outside of the lab environment).
Ethics evaluates if ethical guidelines were followed, often by considering consent, deception, confidentiality, and potential for harm. Unethical procedures can invalidate the findings.
How to Use GRAVE to Show Critical Thinking
When using GRAVE, the goal is to explain how each evaluation point actually impacts our interpretation of the study's results, rather than just labelling strengths or limitations in isolation.
For example, consider a study investigating adolescents’ video game usage through self-report questionnaires. A poor evaluation might read:
| "This study had limited internal validity due to social desirability bias - participants' tendency to answer questions in a way that makes them look favourable." |
This does technically identify a limitation but fails to analyse its implications. A better alternative would be:
| "This study's internal validity was limited by potential social desirability bias arising from the self-report questionnaires. As excessive video game usage is often viewed as socially undesirable, participants may have underreported their actual gaming time. This could skew results to show lower usage than what actually occurs." |
Notice how the improved version explains exactly why and how social desirability threatens the accuracy of the data, by directly linking it to the questionnaire method and the sensitive topic. This level of critical analysis strengthens the evaluation considerably by elucidating the bias's direction of influence.
Using GRAVE well means digging into the meaningful impacts of strengths and limitations, not just stating them abstractly. This allows you to demonstrate in-depth critical thinking in your analysis of research.
Don’t Neglect the Strengths
While we might assume that to show critical thinking, we must identify as many limitations as possible presented by the study, it is equally important not to neglect the strengths of a study. Recognising the ways a study excels in certain areas also displays critical analysis. For example, highlighting how a study has precisely controlled for extraneous variables indicates that you understand why these controls were necessary to address the specific research question effectively. In some cases, studies produce groundbreaking results mainly because of methodological strengths rather than despite limitations.
An example of how to recognise a significant strength, using a hypothetical study that was investigating the effects of a hallucinogen on the treatment of a disorder, by using low doses of the drug instead of a placebo.
| "A methodological strength of this study was its internal validity, due to the use of low doses of the hallucinogen (active control) rather than an inert placebo in the control group. Since the hallucinogen produces pronounced psychological effects that participants would likely detect, employing a pharmacologically active placebo control helped minimize the likelihood of participants guessing their treatment arm correctly, thus controlling for the placebo effect. Participants in both the treatment and active control groups may have guessed they received an active substance due to the psychological effects of the drug even at low doses, maintaining sufficient uncertainty regarding their assigned condition. The strategic choice of an active control, therefore, helped isolate the impacts of MDMA itself by accounting for general stimulant effects in the comparison group." |
This is quite a lengthy evaluation point – more than you would have time to write in an ERQ under timed conditions – but illustrates how you can show critical thinking by pointing out unique strengths in a study that contribute significantly to the robustness of its findings.
Conclusion
Evaluating studies critically is a cornerstone of scientific analysis and a key skill assessed in IB Psychology. Remember that the goal is to not simply label strengths as advantageous or limitations as problematic; thoughtful evaluation involves digging into the meaningful implications of these points, to help inform our interpretation of the research findings.
Ready to tackle your IB Psychology exams with confidence? This is Part 3 in a blog series providing expert tips to master IB Psychology. You can read the other parts of this series by heading to our blog and access other free resources on Learnmate. You can also connect with experienced tutors at Learnmate to start working towards achieving your potential in IB Psychology. Share this guide with your peers and start your journey to academic success today!
This blog was written by Claudia Z, an Online IB Tutor on Learnmate. Claudia has been studying a Bachelor of Health Science (Premed) since graduating with a 99.7 ATAR and 44 score in IB. She excelled in the IB program and now tutors students in IB Psychology, English A Literature, Chemistry, French, History and Maths.
You can view her profile and, subject to her availability, request Claudia as your tutor here.
