Standardized Measurement and Assessment

           Standardized Measurement and Assessment

1. Which of the following is not an assumption underlying testing and measurement?

a.  Various approaches to measuring aspects of the same thing can be useful

b.  Error is rarely present in the measurement process

c.  Present-day behavior predicts future behavior

d.  Testing and assessment benefit society

2. Systematic error is associated with:

a.  Reliability

b.  Validity

3. Which of the following is a type of criterion–related validity evidence?

a.  Concurrent evidence

b.  Predictive evidence

c.  Internal consistency

d.  Both a and b are correct answers

4. If a test measures a single construct then:

a.  The items should correlate with the total score

b.  The items should not correlate with the total score

c.  The test should not correlate with other measures of the same construct

d.  There must be a reliable alternative form.

5. Professor X develops a test of emotional intelligence. Which of the following represent convergent and discriminant evidence?

a.  The test correlates highly with another test of emotional intelligence and is

                 uncorrelated with self-efficacy

b.  The test correlates with highly with another test of emotional intelligence and is

            highly correlated with self-efficacy

c.  The test does not correlate with another test of emotional intelligence, but does

                 correlate with self-efficacy

d.  The test does not correlate with other tests of emotional intelligence nor with self-

            efficacy

6. An ordinal scale is used to rank order people, objects, or characteristics.
a. true

b.false

7. Which scale is the simplest form of measurement?
a. Nominal
b. Ordinal
c. Interval
d. Ratio

8.  ______ tests focus on information acquired through the informal learning that goes on in life.
a. Personality
b. Achievement
c. Aptitude
d. Intelligence

9. Let’s say that a test accurately indicates participants’ scores on a future criterion (e.g., the PSAT is used to indicate high-school GPA scores). This test would clearly have which of the following?

a. Face validity

b. Concurrent validity

c. Predictive validity

d. Content validity

10. If a baseball coach calculates batting averages, what scale would be used?

a. Interval scale

b. Ratio scale

c. Nominal scale

d. Ordinal scale

11. According to the text, most of the outcome/dependent variable characteristics and attributes measured in educational research probably exist at the ______________ level of measurement.

a. Nominal

b. Ordinal

c. Interval

d. Ratio

12. Which of the following is most clearly an example of a psychological trait?

a. Anxiety enduring for months or years

b. Anxiety over just seeing a spider

c. Shyness when meeting a stranger for the first time

d. Depression caused by the loss of a ball game

13. All of the following are examples of Intelligence Tests except _________:

a.  Wechsler Scales

b.  Stanford-Binet

c.  Minnesota Multiphasic Personality Inventory(MMPI)

d.  Slosson

14. Reliability is most simply known as which of the following?

a.  Consistency or stability

b.  Appropriateness of interpretations on the basis of test scores

c.  Ways in which people are the same

d.  A rank order of participants on some characteristic

15. An ordinal scale is:
a. The simplest form of measurement
b. A rank-order scale of measurement
c. A scale with equal intervals between adjacent numbers
d. A scale with an absolute zero point

e. A categorical scale

16. Which of the following is not a type of reliability?
a. Test-retest
b. Split-half
c. Content

d. Internal consistency

17. Which of the following statements accurately describes test-retest reliability?

a.  Measure of consistency of test scores over time

b.  Measure of consistency of scores obtained from two equivalent halves of the same test

c.  Measure of consistency with which a test measures a single construct or concept

d.  Measure of degree of agreement between two or more scorers, judges, or raters

18. Which of the following types of reliability refers to the consistency of test scores over time?
a. Equivalent forms reliability
b. Split-half reliability
c. Test-retest reliability
d. Inter-scorer reliability

19. Identify the following term that most closely refers to a judgement of the extent to which scores from a test can be used to infer, or predict, the examinees' performance in some activity:
a. Content reliability
b. Face validity
c. Criterion-related validity 
d. Inference validity

20. Which of the following is the correct order of Stevens’ four levels of measurement?
a. Ordinal, nominal, ratio, interval
b. Nominal, ordinal, interval, ratio
c. Interval, nominal, ordinal, ratio
d. Ratio, interval, nominal, ordinal

21. Which is the process of gathering evidence supporting inferences based test scores?
a. Validation
b. Validity
c. Reliability

d. Prediction

22. When evaluating tests and assessments, “reliability” refers to asking ourselves which of the following questions?

a.  Does it measure what it is supposed to measure?

b.  Are there ways to avoid subjective judgments when measuring something?

c.  Does it give consistent results?

d.  Does it measure multiple constructs?

23. Validity of a test designed to measure a construct such as self-esteem is best described by which of the following?

a.  Scores from the test correlate highly with most intelligence tests

b.  Scores from the test correlate highly with most tests of different constructs

c.  Scores from the test are not correlated with anything

d.  Scores from the test have a relatively strong and positive correlation with other tests of the

same construct (i.e., with other measures of self-esteem) but much lower correlations

with tests of different constructs

24. Which type of reliability refers to the consistency of a group of individuals' scores on two  equivalent forms of a test designed to measure the same characteristic? 

a.  Split-half

b.  Test-retest

c.  Split-forms

d.  Equivalent forms

25. Achievement tests are designed to measure the degree of learning that has taken place after a person has been exposed to a specific learning experience.

a.   True

b.   False

26. _________ refers to how well the particular sample of behaviors used to measure a characteristic reflects the entire domain of behaviors that constitutes that characteristic.
a. Construct validity evidence
b. Criterion-related validity evidence
c. Content validity evidence
d. Face validity evidence

Methods of Data Collection

                                                Methods of Data Collection

1. According to your text, how many points should a rating scale have?

a. Five

b. Four

c. Ten

d. Somewhere from 4 to 11 points

2. What is the problem(s) with this set of response categories to the question “What is your current age?”

1-5

5-10

10-20

20-30

30-40

a. The categories are not mutually exclusive

b. The categories are not exhaustive

c. Both a and b are problems

d. There is no problem with the above set of response categories

3. You should mix methods in a way that provides complementary strengths and nonoverlapping weaknesses. This is known as the fundamental principle of mixed research.

a. True

b. False

4. According to the text, questionnaires can address events and characteristics taking place when?

a. In the past (retrospective questions)

b. In the present (current time questions)

c. In the future (prospective questions)

d. All of the above

5. Which of the following are principles of questionnaire construction?

a. Consider using multiple methods when measuring abstract constructs

b. Use multiple items to measure abstract constructs

c. Avoid double-barreled questions

d. All of the above

e. Only  b and c

6. Which of these is not a method of data collection.
a. Questionnaires
b. Interviews
c. Experiments

d. Observations

7. Secondary/existing data may include which of the following?
a. Official documents
b. Personal documents

c. Archived research data

d. All of the above

8. An item that directs participants to different follow-up questions depending on their response is called a ____________.

a. Response set

b. Probe

c. Semantic differential

d. Contingency question

9. Which of the following terms best describes data that were originally collected at an earlier time by a different person for a different purpose?

a. Primary data

b. Secondary data

c. Experimental data

d. Field notes

10. Researchers use both open-ended and closed-ended questions to collect data. Which of the following statements is true?

a. Open-ended questions directly provide quantitative data based on the researcher’s predetermined response categories

b. Closed-ended questions provide quantitative data in the participant’s own words

c. Open-ended questions provide qualitative data in the participant’s own words

d. Closed-ended questions directly provide qualitative data in the participants’ own words

11. Open-ended questions provide primarily ______ data.

a.  Confirmatory data

b.  Qualitative data

c.  Predictive data

d.  None of the above

12. Which of the following is true concerning observation?

a. It takes less time than self-report approaches

b. It costs less money than self-report approaches

c. It is often not possible to determine exactly why the people behave as they do

d. All of the above

13. Qualitative observation is usually done for exploratory purposes; it is also called ___________ observation.

a. Structured

b. Naturalistic

c. Complete

d. Probed

14. As discussed in chapter 6, when constructing a questionnaire it is important to do each of the following except  ______.

a. Use "leading" or "loaded" questions

b. Use natural language

c. Understand your research participants

d. Pilot your test questionnaire

15. Another name for a Likert Scale is a(n):

a. Interview protocol

b. Event sampling

c. Summated rating scale

d. Ranking

16. Which of the following is not one of the six major methods of data collection that are used by educational researchers?

a.  Observation

b.  Interviews

c.  Questionnaires

d.  Checklists

17. The type of interview in which the specific topics are decided in advance but the sequence and wording can be modified during the interview is called:

a. The interview guide approach

b. The informal conversational interview

c. A closed quantitative interview

d. The standardized open-ended interview

18. Which one of the following is not a major method of data collection:

a. Questionnaires

b. Interviews

c. Secondary data

d. Focus groups

e. All of the above are methods of data collection

19. A question during an interview such as “Why do you feel that way?” is known as a:

a. Probe

b. Filter question

c. Response

d. Pilot

20. A census taker often collects data through which of the following?

a. Standardized tests

b. Interviews

c. Secondary data

d. Observations

21. The researcher has secretly placed him or herself (as a member) in the group that is being studied. This researcher may be which of the following?

a. A complete participant

b. An observer-as-participant

c. A participant-as-observer

d. None of the above

22. Which of the following is not a major method of data collection?
a. Questionnaires
b. Focus groups
c. Correlational method
d. Secondary data

23. Which type of interview allows the questions to emerge from the immediate context or course of things?
a. Interview guide approach
b. Informal conversational interview
c. Closed quantitative interview
d. Standardized open-ended interview

24. When conducting an interview, asking "Anything else?, What do you mean?, Why do you feel that way?," etc, are all forms of:

a. Contingency questions

b. Probes

c. Protocols

d. Response categories

25. When constructing a questionnaire, there are 15 principles to which you should adhere. Which of the following is not one of those principles?

a. Do not use "leading" or "loaded" questions

b. Avoid double-barreled questions

c. Avoid double negatives

d. Avoid using multiple items to measure a single construct

Research Ethics

                                                     Research Ethics

1. Ethics is the set of principles and guidelines that help us to uphold the things we value.
a. True
b. False
 
2. Which of the following is necessary in obtaining informed consent?
a.  A description of the statistical analyses that will be carried out
b.  A description of the purpose of the research
c.  A description of the reliability and validity of test instruments
d.  A list of publications that the researcher has had in the last ten years
 
3. Which of the following need(s) to be obtained when doing research with children?
a.   Informed consent from the parent or guardian
b.  Assent from the child if he or she is capable
c.  Informed consent from the child
d.  Both a and b
 
4. Which of the following is true about the use of deception in research?
a.  It should never be used 
b.  It can be used anytime
c.  If there is deception in a study, the participants may need to be debriefed
d.  The use of deception must be outweighed by other benefits of the study
e.  Both c and d are true
 
5. Which of the following generally cannot be done in qualitative studies conducted in the field?
a.  Getting informed consent
b.  Keeping participants from physical harm
c.  Maintaining consent forms
d.  Having full anonymity rather than just confidentiality 
 

6. What is the primary approach that is used by the IRB to assess the ethical acceptability of a research study?
a.  Utilitarianismb.  Deontologyc.  Ethical skepticism

 
7. Which of the following approaches says that ethical issues should be judged on the basis of some universal code?
a. Deontological
b. Ethical skepticism
c. Utilitarianism
8 Which of the following is not an ethical guideline for conducting research with humans?
a. Getting informed consent of the participant
b. Telling participants they must continue until the study has been completed
c. Keeping participants’ identity anonymous
d. Telling participants they are free to withdraw at any time

9. Which of the three ethics approaches says research ethics should be a matter of the individual's conscience?
a. Deontological approach
b. Ethical skepticism
c. Utilitarianism
d. Ontological skepticism

10. ________ means that the participant's identity, although known to the researcher, is not revealed to anyone outside of the researcher and his or her staff.
a. Anonymity
b. Confidentiality

11. Which of the following is not true?
a. Misrepresenting and creating fraudulent data is dishonest
b. Misrepresenting data is very easy to detect
c. Misrepresenting data can be difficult to detect
d. Breaking confidentiality is not a problem

12. Ideally, the research participant's identity is not known to the researcher. This is called:
a. Anonymity
b. Confidentiality
c. Deception
d. Desensitizing

13. Which of the following approaches taken by people to resolve ethical issues is the primary approach used by the federal government and most professional organizations? 
a. Deontological approach
b. Ethical skepticism
c. Utilitarianism
d. None of the above

14. What is it called when the participants are not revealed to any one but researcher and staff?
a. Confidentiality
b. Anonymity
c. Ethics
d. Discretion

15. Research participants must give what before they can participate in a study?
a. Guidelines
b. A commitment
c. Informed consent
d. Private information

16. There are three basic approaches that people tend to adopt when considering ethical issues in research. Which one of the following is not one of the approaches?
a. Ethical skepticism
b. Deontology
c. Ontology
d. Utilitarianism

17. Identify the term that refers to a poststudy interview in which all aspects of the study are revealed, reasons for the use of deception are given, and the participants’ questions are answered?
a. Desensitizing
b. Debriefing
c. Dehoaxing
d. Deploying
 
18. A set of principles to guide and assist researchers in deciding which goals are most important and in reconciling conflicting values when conducting research is called ____.
a. Research ethics
b. Deontological approach
c. Utilitarianism
d. None of the above

19. IRB is an acronym for which of the following?
a. Internal Review Board
b. Institutional Rating Board
c. Institutional Review Board
d. Internal Request Board 

20. When it is necessary to engage in a good amount of deception to conduct a scientifically valid study, what procedure(s) should a researcher consider following?
a. Debriefing
b. Dehoaxing
c. Desensitizing
d. All of the above should be considered

21. The act of publishing the same data and results in more than one journal or publication refers to which of the following professional issues:
a. Partial publication
b. Duplicate publication
c. Deception
d. Full publication
 
22. Concerning "authorship" in educational research, intellectual ownership is predominantly a function of:
a.       Effort expended
b.      Creative contribution
c.       Professional position
d.      Level of higher education

23. Which term refers to publishing several articles from the data collected in one large study?
a. Duplicate publication
b. Partial publication
c. Triplicate publication
d. None of these
24. Which of the following is a right of each participant according to the AERA?
a. Deception
b. Utilitarianism
c. Freedom to withdraw
d. Participants have no rights

Developing Research Questions and Proposal Preperation

        Developing Research Questions and Proposal                                               Preparation

1. A good qualitative problem statement:
a. Defines the independent and dependent variables

b. Conveys a sense of emerging design

c. Specifies a research hypothesis to be tested
d. Specifies the relationship between variables that the researcher expects to find 
 
2. The “tool” function of theory is to:
a. Summarize existing knowledge
b. Summarize existing hypotheses
c. Suggest new relationships and make new predictions
d. Suggest new theories
 
3. The statement of purpose in a research study should:
a. Identify the design of the study
b. Identify the intent or objective of the study
c. Specify the type of people to be used in the study
d. Describe the study
 
4. Why is the statement “What are the effects of extracurricular activities on cognitive development of school age children” not a good statement of a quantitative research question?
a. Because there is no connection between extracurricular activities and cognitive
development
b. Because there are not enough school age children engaged in extracurricular activities
to conduct the study
c. Because the study would be too difficult to do given all the different extracurricular
activities
d. Because the statement was not specific enough to provide an understanding of the
variables being investigated
 
5. A qualitative research question:
a. Asks a question about some process, or phenomenon to be explored
b. Is generally an open-ended question
c. both a and b are correct
d. None of the above
 
6. According to the text, which of the following orders is the recommended in the flowchart of the development of a research idea?
a. Research topic, research problem, research purpose, research question, hypothesis
b. Research topic, research purpose, research problem, research question, hypothesis
c. Research topic, research problem, research purpose, research question, hypothesis
d. Research topic, hypothesis, research problem, research question, research purpose
 
7. It is essential that you evaluate the quality of internet resources because information obtained via the internet ranges from very poor to very good.
a. True
b. False
 
8.  One step that is not included in planning a research study is:
a.  Identifying a researchable problem 
b.  A review of current research
c.  Statement of the research question 
d.  Conducting a meta-analysis of the research
e.  Developing a research plan
 
9.  Sources of researchable problems can include: 
a.  Researchers’ own experiences as educators
b.  Practical issues that require solutions
c.  Theory and past research 
d.  All of the above
 
10.  A key characteristic of past research that guides researchers in new research questions is that: 
a.  Extensive research conclusively and definitively answers research questions 
b.  Studies typically generate more research questions than they answer
 
11.  Which of the following is a function of theory?
a.  Integrating and summarizing current knowledge
b.  Making predictions
c.  Explaining phenomena
d. All of the above are important functions of theory
 
12.  A review of the literature prior to formulating research questions allows the researcher to do which of the following?
a.  To become familiar with prior research on the phenomenon of interest
b.  To identify potential methodological problems in the research area
c.  To develop a list of pertinent problems relative to the phenomenon of interest 
d.  All of the above
 
13.  Sometimes a comprehensive review of the literature prior to data collection is not recommended by grounded theorists.
a.  True
b.  False
 
14.  What kind of ideas can’t be empirically researched? 
a.  Effectiveness of different methods of instruction
b.  Description of educational practices
c.  Issues of values and morality such as the correctness of having prayer in schools
d.  Factors helpful in predicting future drug use 
 
15.  Which of the following is not a database containing information to be used during the literature review?
a.  ERIC
b.  PsychINFO
c.  SocioFILE 
d.  all of the above are potentially useful data bases
 
16.  Computer database searches can be done:
a.  With a computer with CD-ROM drive
b.  At the library
c.  Online
d.  All of the above
 
17.  The feasibility of a research study should be considered in light of:
a.  Cost and time required to conduct the study
b.  Skills required of the researcher
c.  Potential ethical concerns
d.  All of the above
 
18.  A formal statement of the research question or “purpose of research study” generally ______.
a.  Is made prior to the literature review 
b.  Is made after the literature review
c.  Will help guide the research process 
d.  All of the above
e.  b and c
 
19.  Is the following qualitative research purpose statement “well stated” or “poorly stated”? “The focus of the present study was to explore distressing and nurturing encounters of patients with caregivers and to ascertain the meanings that are engendered by such encounters. The study was conducted on one of the surgical units and the obstetrical/gynecological unit of a 374-bed community hospital.”a. It is a well stated
b. It is poorly stated
 
20.  Which of the following quantitative research questions is superior?
a.  “What is the effect of participation in various extracurricular activities on academic performance?” 
b.  “What effect does playing high school football have on students’ overall grade point average during the football season?”
 
21.  A statement of the quantitative research question should:
a.   Extend the statement of purpose by specifying exactly the question(s) the researcher will
      address
b.   Help the research in selecting appropriate participants, research methods, measures, and
      materials 
c.   Specify the variables of interest 
d.  All of the above
 
22. The research participants are described in detail in which section of the research plan?
a. Introduction
b. Method
c. Data analysis
d. Discussion
 
23.  Research hypotheses are ______.
a.  Formulated prior to a review of the literature 
b.  Statements of predicted relationships between variables
c.  Stated such that they can be confirmed or refuted
d.  b and c
 
24.  Hypotheses in qualitative research studies usually _____.
a.  Are very specific and stated prior to beginning the study
b.  Are often generated as the data are collected, interpreted, and analyzed
c.  Are never used
d. Are always stated after the research study has been completed
 
25.  A research plan _____.
a.  Should be detailed
b.  Should be given to others for review and comments 
c.  Sets out the rationale for a research study
d.  All of the above
 
26.  The Method section of the research plan typically specifies
a.  The research participants 
b.  The results of prior studies that address the phenomena of interest 
c.  The apparatus, instruments, and materials for the research study
d.  The planned research procedures
e.  a, c and d
 
27.  The Introduction section of the research plan
a.  Gives an overview of prior relevant studies
b.  Contains a statement of the purpose of the study
c.  Concludes with a statement of the research questions and, for quantitative research, it includes
      the research hypothesis 
d.  All of the above
 
28. According to your text, which of the following is not a source of research ideas?
a. Everyday life
b. Practical issues
c. Past research
d. Theory
e. All of the above ARE sources of research ideas