What Is Internal Validity?
Internal validity refers to the degree to which a study's results can be attributed to the independent variable rather than to extraneous or confounding variables. In ABA research, internal validity is the foundation of demonstrating functional relationships. If a study has high internal validity, we can confidently say that the intervention (not something else) caused the observed behavior change.
Threats to internal validity are alternative explanations for the results. When a threat is present, the researcher cannot be sure that the independent variable was responsible for the change. Understanding these threats is critical for designing rigorous studies, evaluating published research, and answering BCBA exam questions.
Major Threats to Internal Validity
Maturation
Maturation refers to changes that occur naturally over time due to biological or developmental processes. These include physical growth, cognitive development, fatigue, hunger, or simply the passage of time. If an intervention takes place over several months, improvements in behavior might be due to natural development rather than the treatment.
Example: A language intervention study lasting six months shows improvement in a 3-year-old's vocabulary. Is the improvement due to the intervention, or would the child's vocabulary have grown naturally during that developmental period?
How designs control for it: Reversal designs demonstrate that behavior changes when the intervention is introduced and changes back when it is removed, making maturation an unlikely explanation. Multiple baseline designs show that baselines remain stable until treatment is specifically applied, which would not occur if maturation alone were responsible.
History
History refers to external events (other than the independent variable) that occur during the study and could affect the dependent variable. These events might be related to the client's life, the setting, or broader environmental changes.
Example: A behavior analyst implements a new classroom management system. At the same time, the school hires a new, more experienced teaching assistant. Did the new system improve behavior, or was it the new assistant?
How designs control for it: Multiple baseline designs are particularly effective against history threats. Because the intervention is introduced at different times across baselines, a coincidental event would need to occur at each of those specific times to explain the results, which is highly unlikely. Reversal designs also help because the behavior should return to baseline levels during withdrawal regardless of external events.
Testing Effects
Testing effects (also called reactivity) occur when the process of measuring or assessing behavior changes the behavior itself. Repeated exposure to an assessment may produce improvement due to practice or familiarity rather than the intervention.
Example: A client takes a social skills assessment before and after an intervention. Improvement on the post-test might partly reflect familiarity with the test format rather than genuine skill improvement.
How designs control for it: In single-subject designs, measurement occurs continuously throughout all phases, so any testing effects should be relatively constant across conditions. If behavior improves only when the intervention is introduced (not gradually throughout all phases), testing effects are an unlikely explanation.
Instrumentation
Instrumentation threats occur when changes in the measurement system (rather than changes in the actual behavior) produce apparent effects. This can include changes in observers, observation procedures, equipment, or scoring criteria.
Example: An observer becomes more skilled at detecting subtle instances of a behavior over time, leading to an apparent increase in behavior that reflects improved detection rather than actual behavior change. Alternatively, observer drift might occur, where the observer gradually applies the operational definition differently over time.
How designs control for it: Regular interobserver agreement checks help detect instrumentation problems. If IOA remains high throughout the study, it is less likely that instrumentation changes are responsible for the observed effects. Consistent training and calibration of observers also reduce this threat.
Attrition
Attrition (also called mortality) occurs when participants drop out of the study. If the participants who leave differ systematically from those who remain, the results may be biased. While attrition is primarily a concern in group designs, it can affect single-subject research when data from incomplete cases are interpreted.
Example: In a multiple baseline across participants design, one participant drops out during the baseline phase. If this participant was the one showing the most challenging behavior, the remaining data may overrepresent successful outcomes.
Regression to the Mean
Regression to the mean occurs when individuals are selected for study based on extreme scores. Because extreme scores often reflect measurement error or unusual circumstances, subsequent measurements tend to be closer to the average, regardless of any intervention.
Example: A student is referred for intervention after a particularly bad week of behavior. The following week, behavior improves, but this might simply reflect a return to the student's typical level of performance rather than an intervention effect.
How designs control for it: Extended baselines that demonstrate a stable pattern of behavior help rule out regression to the mean. If the behavior remains at a consistent (problematic) level throughout an extended baseline and changes only when treatment is introduced, regression is not a viable explanation.
Controlling for Threats in Single-Subject Designs
The strength of single-subject experimental designs lies in their ability to control for multiple threats simultaneously through repeated measurement, stable baselines, and replication of effects. The three key elements of experimental control (prediction, verification, and replication) work together to rule out alternative explanations.
Stable baselines provide prediction. Behavior changes coinciding with intervention introduction provide verification. And repeated demonstrations of the effect (whether through reversal, staggered introduction, or alternation of conditions) provide replication that makes coincidental explanations increasingly implausible.
Exam Preparation Tips
On the BCBA exam, be prepared to identify threats to internal validity in described scenarios, explain how specific research designs control for specific threats, and distinguish between threats that are well-controlled in a given design and those that are not. Understanding these concepts is essential for both the exam and for conducting and evaluating research in practice.