2- CRIMINAL JUSTICE RESEARCH

Page 1 of 2
CRM204(01): INTRODUCTION TO CRIMINAL JUSTICE RESEARCH

Assignment #2: Threats to Internal Validity (worth 20% of final grade)

Instructions: For both studies, only identify the threats to internal validity, do not critique broader methodological
issues and do not simply list all threats and create scenarios for how they might all apply. You should be working with
the information provided as was done in class – marks may be deducted if you list something as a threat when it is not a
threat.
Formatting: Please use either a Word or PDF format for submission (letter sized paper; 1inch margins; 12point Times
New Roman font (or something similar to that); double or single spaced). You do not need a title page, but your name
and student ID should be somewhere on the 1st page). You should not need more than two (single-spaced) pages to
explain the internal validity threats present in both studies. Use the example answers from class as a guide for
how much detail to provide and how to organize/format your answers – FAILURE to follow this formatting
will result in a 2 point deduction. (Do not, however, simply quote information off of the PowerPoint slides). If you
missed classes, get notes from someone who attended and review the PowerPoint slides posted on D2L.
Late Penalties: Late assignments (e.g. assignments submitted via D2L after 11:10am) will be penalized 5% and another
5% each day thereafter.
Group Work: While you may work in groups, do your own work (do not as a group all submit the same answers – identical
answers submitted by multiple students constitute academic misconduct).
Questions?: Make an appointment to see me in my office hours (or email me to set a time outside my office hours).
Please note that I will not read rough drafts or review labels for identified internal validity threats. You should see me if
you don’t understand a study; or if you have specific questions about an internal validity threat/class lecture/the text.

Question 1 [10 points]: A police organization wanted to run a diversion program in order to reduce recidivism among first
time young offenders aged 13 and 14 charged with minor shoplifting offences. The program consisted of attending a weekly
two-hour anti-shoplifting workshop for one month (4 workshops in total). Once the youths completed the program, the charges
would be dismissed. In order to evaluate the effect of this program on recidivism, the police chief hired an independent
researcher to randomly assign youths who were charged, for the first time, with minor shoplifting offences to receive the
program or not. The researcher randomly assigned 150 youths to receive the program (experimental group) and 150 youths
to proceed on to court (control group). However, an officer reviewed the selection to be sure that only those cases that were
in the public interest to be dealt with outside of court were included in the experimental group. This resulted in 110 youths
being in the experimental group and 190 being in the control group.
As the program began, defense counsel and youths themselves heard that this option (diversion) was available to some youths
but not others. They thus began strongly advocating for a diversion program for all first time minor offenders arguing that it
was unfair not to allow broader access. In the end, the court created its own diversion program available for all first time
offenders that Crown attorneys could send youths to instead of proceeding with the charge.
It took one month for everyone in the experimental group to finish the diversion program, but unfortunately 27 youths dropped
out part way through, leaving 83 in the experimental group. A month after the youths in the experimental group completed
the program the researcher hired by the police – but not in any way associated with the police – contacted the youths who
completed the program and asked how many offences they committed in the last month. The same was done with the control
group – a month after their case was disposed of, the researcher contacted the 190 youths from the control group and asked
how many offences they committed in the last two months.
To test the effectiveness of the diversion program on recidivism, the youths’ self-reported offending was used (described above)
along with the number of new charges youths accumulated within six months following the intervention (diversion or court).
In order to keep the groups equivalent in size, the first 83 youths that the researcher found charge data on were used for the
control group.
Self-Reported Offending
Average number of reported offences
New Charges
Average number of charges
Experimental group 2.3
(N=83)
3.6
(N=83)
Control group 5.4
(N=190)
9.0
(N=83)
Looking at the above table, the police chief saw that his program was an immediate success – there were fewer self-reported
offences in the experimental group immediately following the completion of the program. Moreover, six months after the
program – and using a different measure of “offending” (charge data), the results were consistent – fewer charges in the
experimental group compared to the control group. Identify and explain any threats to internal validity with this
experiment.
Page 2 of 2
Question 2 [10 points]: After an unusually large number of fights at a school (for grades 7 and 8) during November and
December the principal decided to implement a violence reduction program. Since she took a research methods class, she
knew the program should be evaluated. She identified all of the youths who were involved in fights over the past two months
(N=127) and asked them to participate. Notes were also sent to the parents/guardians of these children asking whether they
would allow their child “to participate in an anti-aggressiveness program (running during the month of January)”. All of the
parents gave their consent as did the children.
Using a table of random numbers, 127 students from the remainder of the student population were randomly selected to be in
the ‘control group’. Before the program started, youths in both the experimental and control groups were asked to fill out a
survey (pre-test) which asked them how often in the past two months (during Nov/Dec) they had been in a fight with someone
(involving any sort of verbal violence (e.g. threats) or physical violence (e.g. anything from pushing and shoving to needing
stitches and having broken bones)).
When the program began (during January) youths who were not given access to the program heard about it and began getting
upset that they did not have access to anything equivalent. Many of the youths started complaining to their teachers and acting
out but the principal refused to allow them any access resulting in growing discontent among the students. In the end, 72 of
the 127 youths in the experimental group completed the program. At the beginning of April (two months after the program
finished) the 72 youths from the experimental group were congratulated for completing the program and putting themselves
on a path towards lower violence. They were then asked again (post-test) how often, since the program ended (i.e. during
Feb/March), they had been in a fight with someone (involving any sort of verbal violence (e.g. threats) or physical violence
(e.g. anything from pushing and shoving to needing stitches and having broken bones)).
The principal was concerned about the participant loss in the experimental group, so, in order to address that and keep the
groups identical, she selected (using a table of random numbers) 72 of 127 students from control group for the post-test. This
ensured a representative sample from the control group and identical numbers between the two groups. At the beginning of
April, these youths were asked how often, in the past two months (i.e. during Feb/March), they had been in a fight with
someone (involving any sort of verbal violence (e.g. threats) or physical violence (e.g. anything from pushing and shoving to
needing stitches and having broken bones)). Compared to the pre-test, the experimental group saw a significantly large
reduction in the average number of fights reported by youths while the control group’s reported involvement in fights actually
increased in the post-test (see table). Thus, in the second survey (the post-test), the experimental group was actually lower in
violence then the control group (see table). Based on these findings the Principal decided that the program was phenomenally
successful in reducing violence and should be offered to every student in the school. Identify and explain any internal
validity threats that are present.
Pre-test
Average number of reported fights
(Nov/Dec)
Post-test
Average number of reported fights
(Feb/March)
Experimental group 15.8
(N=127)
1.3
(N=72)
Control group 0.0
(N=127)
9.0
(N=72)