Statistics

This assignment is concerned with measures, investigating measures, and understanding measures well. Every construct in your Dissertation, using a
quantitative or mixed methods approach, must be measured. One cannot do statistical analysis on words or ideas, but only on numbers. So a construct, let’s
say Organizational Commitment, must be converted into numbers by being measured. A student working on her quantitative Dissertation asked if every
construct in her Dissertation (or, more specifically, in her regression analysis which included predictor variable constructs and an outcome variable construct)
needs to be measured. The answer is that, yes, every construct in the model needs to be measured, and then the regression analysis will be conducted on the
numbers that come from the measures of the constructs.
Finding and investigating measures of constructs is one of the most time consuming tasks for Dissertation students. It is also a crucial task for a relatively
smooth Dissertation process. It might at first seem like a straightforward matter to find and properly use a measure of a construct, but that could not be
further from the truth. Indeed, if a student working on her or his Dissertation thinks “I just collected all my measures and it was no big deal to do” then that
usually means that the student has done it poorly and that the measures will pose problems and delays for the student later. So being practiced (purpose of
this assignment), thoughtful, and proactive in finding and investigating measures so that you can choose measures that measure your constructs well and
that you can properly and meaningfully use strongly contributes to a relatively efficient timeline for making progress on your Dissertation.
Constructs are only useful for your Dissertation research to the extent that they are measured well and that the measures are consistent with your research
questions. Constructs are measured well to the extent that the measures are reliable and valid. Reliable measures have been shown to be measuring
something of a systematic nature that differentiates between people, without an excessive amount of measurement error. Measures always contain some
amount of error, but it needs to be limited. Valid measures have been shown to actually measure what they claim to measure. A measurement instrument
claiming to measure the Organizational Commitment construct, for example, is of no use if the instrument does not actually measure how committed
employees are to the organizations for which they work. If the measure is full of error (not reliable) or is measuring something else besides what it claims to
measure (not valid) then the measure is of no use. So while there may be lots of measures out there claiming to measure constructs you are interested in for
your research, you must make sure that the measure that you choose is reliable and valid.
You also must make sure that the measure you are considering for a construct is consistent with your research questions. Suppose a student has a research
question which asks “Does degree of organizational commitment influence turnover intention?” Quickly note that in this research question there are two
constructs to be measured, organizational commitment and turnover intention. Let’s say the student is not particularly careful in choosing measures but
feels confident about his organizational commitment measure because he read in a reputable journal that the measure is reliable and valid. The student
collects his data and conducts his data analysis but then runs into a problem because the results are not relevant for answering his research question. It
turns out that the measure of organizational commitment does not provide a score for employees degree of organizational commitment overall but only in
specific domains as measured by subscales. So the measure provides a score for Emotionally Driven Organizational Commitment, Obligation Based
Organizational Commitment, and Personally Embedded Organizational Commitment, but not for Organizational Commitment overall. There are other
measures of organizational commitment that provide a score for organizational commitment overall but none of these measures were chosen by the
student. So now the student has to change his research question to something like “Does degree of emotionally driven organizational commitment influence
turnover intention?” The student also needs to rewrite the introduction chapter of his Dissertation to justify conducting research on the possible influence of
emotionally driven organizational commitment on turnover intention, rewrite some of his method chapter, and rewrite substantial portions of his literature
review and discussion chapters. It would have been much more time efficient for the student to have investigated and chosen measures of constructs that
were consistent with the research question as originally written.
So the purpose of this assignment is for you to investigate a measure of a construct. Investigating measures of constructs – so that you make sure you
choose measures that measure your constructs well and that you can use properly and in a meaningful way – is a highly important task you must engage in
when conducting research, be it for your Dissertation or in some other context.
From the Measures List (see below), choose one measure to investigate. IN MY WORKING WITH STUDENTS, in different contexts I ask them to investigate
measures including for courses. In choosing one measure to investigate for this assignment, DO NOT choose a measure that you have investigated
previously (either as a part of your being in a course with me, with another instructor, engaging in your own research, or whatever the context may have
been). There are plenty of measures to choose from in the Measures List, so choose one measure to investigate that you have never investigated before.
From the Measures List, for the one measure you choose to investigate be sure to find information on each of the following twelve (12) points. Describe the
measure in terms of each of these points. The format for describing the measure should be point by point (1, 2, 3,…11, 12). The twelve points are:
Continuous or categorical measure (continuous measures provide a range of possible scores, categorical measures typically provide two or three categories
in which persons are placed (e.g., low risk, medium risk, high risk))
Reliability of measure (usually in the form of Cronbach’s alpha (or internal consistency of a measure))
Summarize validity of measure (is the measure considered to be valid, not valid, or basically valid but there are concerns or limitations (in which case be sure
to indicate the concerns or limitations))
Total number of items in measure
Response scale(s) of measure (e.g., 5-point Likert scale ranging from Strongly Disagree to Strongly Agree for each item; indicate Yes or No for each item)
Examples of items
Range of possible scores on continuous measure (lowest possible score, highest possible score); categories on categorical measure (e.g., low risk, high risk)
How is measure scored? Any reverse-scored items? If subscales are present, do you only get subscale scores or can you also get a score for the entire
measure? If you can get a score for the entire measure, then is it a total score or an average score? What does a higher score mean (e.g., the higher the score
on the measure the more likely an employee is to leave an organization)? How did you determine how to score measure – did you find formal scoring
Range of possible scores on continuous measure (lowest possible score, highest possible score); categories on categorical measure (e.g., low risk, high risk)
How is measure scored? Any reverse-scored items? If subscales are present, do you only get subscale scores or can you also get a score for the entire
measure? If you can get a score for the entire measure, then is it a total score or an average score? What does a higher score mean (e.g., the higher the score
on the measure the more likely an employee is to leave an organization)? How did you determine how to score measure – did you find formal scoring
directions (indicate source of formal scoring directions) or did you have to attempt to figure scoring out on your own?
Subscales? If so, number of subscales, name of each subscale, reliability of each subscale, summarize validity of each subscale, number of items on each
subscale. How is each subscale scored? Any reverse-scored items on the subscales? Total or average score on each subscale? Range of possible scores on
each subscale? On each subscale what does a higher score mean? Examples of items for each subscale. How did you determine how to score subscales –
did you find formal scoring directions (indicate source of formal scoring directions) or did you have to attempt to figure scoring out on your own?
In total, how many scores do you get from the measure?
How do you obtain the measure (that you actually provide to people to fill out)? Must you purchase the measure from a publisher, or is the measure available
for free given appropriate permission? If the measure must be purchased from a publisher, can you find an equivalent measure (including in its psychometric
properties (reliability, validity, etc.)) that is available for free?
What is your overall evaluation of the measure? Does the measure seem to be at least an adequate measure of the construct? What do you like most about
the measure? What concerns you? Keep this evaluation brief, no more than 4 or 5 sentences.
How do I Investigate a Measure?!
Look at research articles concerned with particular constructs or that may have used a particular measure. A popular source of information about
characteristics of measures is to look in the Method section of research articles, and the descriiption of measures. If you are interested in finding a measure
of Turnover Intention, for example, then search for research articles concerned with Turnover Intention (using the keywords of the construct name, in this
instance Turnover Intention). Look in the Method section of some of these research articles, see how Turnover Intention was measured, and read the
descriiption of the measure. Alternatively, if you already know the name of the measure you want to investigate, then use the name of the measure as the
keywords of your search. Some of the research articles from this search will have used that measure and you can read about the measure’s characteristics in
the Method section of the articles. EBSCO, ProQuest, and Google Scholar are search engines commonly used for this purpose.
A research article on the factor structure of a measure is usually a goldmine of information about the measure. For a particular measure, such a research
article may or may not exist but if you find one you have hit pay dirt! Put the keywords factor and the name of the measure in the search engine, and see what
happens.
Search Dissertations concerned with particular constructs or that may have used a particular measure. ProQuest (the part of ProQuest where you specifically
search for Dissertations) is used for this search. Some Dissertations provide a lot of detail about the measures used, including providing the measures
themselves in Appendices. That gives you access to all of the items of the measure so that you can examine the individual items for determining the content
validity of the measure, and you will see the full response scale or scales of the items. Detailed scoring directions may also be provided in Dissertations.
When looking at Dissertations, be sure to corroborate and confirm what you find by looking at a few Dissertations, at least.
ResearchGate is a popular online platform to connect with and ask questions of the community of researchers. Researchers are very helpful to students on
this platform (so be sure to mention you are a student!). Students often use this platform to ask particular questions about measures and their
characteristics, including questions about how to score measures, reliability and validity of measures, and other measurement matters.
Some web pages are dedicated to particular measures. Do a search on Google using the name of the measure as the keywords, and see what comes up.
Such web pages can provide comprehensive information on a measure (scoring directions, reliability and validity, etc.), as well as the measure itself.
Email one of the authors of a research article that used a particular measure, or the original developer of the measure (who has probably published their own
research), and ask particular questions about the measure. Be sure to mention you are a student. You won’t always get a response, but sometimes you will!
Be sure to look at the bottom of the first page of the research article, or at the very last page of the article, to find out which of the authors to send an email.
Measures List