Experimental Design

Question 1

what are factorial designs?

Answer 1

experimental designs with more than one independent variable

Question 2

what does the term “factor” refer to?

Answer 2

each manipulated independent variable

Question 3

what are two-factorial designs?

Answer 3

experiments with two independent variables

Question 4

what are experiments with two independent variables called?

Answer 4

two-factorial designs

Question 5

what are factorial designs describes by?

Answer 5

described by the number of factors (independent variables) and the number of levels in each factor

Question 6

A two-factorial design with two factors having two levels each is described as a …

Answer 6

2 x 2

Question 7

what do the number of conditions correspond to?

Answer 7

the number of cells in the design

Question 8

what are the different factorial designs?

Answer 8

• within-subject designs
• between-subject designs
• mixed designs

Question 9

what is an interaction in factorial design?

Answer 9

In a factorial design, an interaction is said to occur when the effect of one independent variable
depends on the level of another independent variable.

Question 10

what is a cross-over interaction?

Answer 10

In a cross-over interaction the effect of one variable reverses as a function of the other variable.
Furthermore, there is no main effects

Question 11

what is a survey?

Answer 11

A survey is a structured set of questions or statements given to a group of people to measure their
attitudes, beliefs, values, or tendencies to act.

Question 12

examples of questionaires

Answer 12

• paper
• online
• mailed

Question 13

how can interviews be held?

Answer 13

• Live
• Telephone

Question 14

what can be used for data acquisition for a survey?

Answer 14

• questionnaire
• interview

Question 15

name formats of questions

Answer 15

• open-ended question
• closed question

Question 16

what is an open-ended question?

Answer 16

participants provide narrative information

Question 17

what is a closed question?

Answer 17

Participants should respond to pre-formulated answers.

Question 18

what are the advantages of open-ended questions?

Answer 18

• eliciting a wide range of responses, including some not even conceived of by the researchers
• increase the sense of involvement of the respondents

Question 19

what are the disadvantages of open-ended questions?

Answer 19

• open-ended questions are difficult to score
• take a lot of time to analyse

Question 20

when are open-ended questions useful?

Answer 20

• asking an open-ended question at the end of closed questions
• using open-ended questions in a pilot study to get an idea of potential answers

Question 21

types of closed questions?

Answer 21

• simple answer
• likert scale

Question 22

what is an advantage of closed questions?

Answer 22

• Can be summarized in a quantitative way (e.g. means, standard deviation)

Question 23

what should be considered when using closed questions?

Answer 23

• number of answer options (three, five, seven)
• some items should be formulated favourably and other unfavourably
• sequence of items (sensitive questions towards the end)

Question 24

when should demographic information be added and why?

Answer 24

Should be added at the end (not to bore participants)

Question 25

what are example of demographic information?

Answer 25

age, gender, socioeconomic status, marital status, education

Question 26

what does the type of demographic information depend on?

Answer 26

the research question

Question 27

what should you avoid when wording survey questions? (3)

Answer 27

- ambiguity - double-barreled questions -leading questions

Question 28

what are double-barreled questions?

Answer 28

Question that touches upon more than one issue, yet allows only for one answer.

Question 29

what are leading questions?

Answer 29

A question that is structured in a way that makes it likely to produce an answer desired by the asker.

Question 30

3 types of leading questions

Answer 30

1. Leading questions that are based on assumptions 2. Leading questions based on interconnected statements 3. Leading questions based on coerciveness

Question 31

what is the researcher's goal with surveys?

Answer 31

try to draw conclusions about the population based on the sample

Question 32

which sampling procedures should be used?

Answer 32

One should use probabilistic sampling procedures

Question 33

what does the sample need to be/should not be?

Answer 33

It is crucial that the sample is representative and not biased.

Question 34

what is psychometrics?

Answer 34

Psychometrics is a scientific discipline concerned with the question of how psychological constructs (e.g., intelligence, depression) can be optimally related to observables (e.g., outcomes of psychological tests)

Question 35

general steps of scale construction (8)

Answer 35

1. Determine what you want to measure (e.g. paranormal beliefs) 2. Generate an item pool 3. Content adequacy assessment 4. Questionnaire administration 5. Factor analysis 6. Internal consistency assessment 7. Construct validation 8. Replication

Question 36

determine what you want to measure (4)

Answer 36

- definition of the construct - objective of the scale - population - content of the scale

Question 37

what are the two general approaches to item generation?

Answer 37

- inductive approach - deductive approach

Question 38

what is the inductive approach?

Answer 38

- generate a pool of items first - use statistical methods to generate scales (e.g. factor analysis)

Question 39

when is the inductive approach used?

Answer 39

used when investigating an unfamiliar topic

Question 40

what is the deductive approach?

Answer 40

uses a theoretical definition of a construct which is then used as a guide for the creation of items

Question 41

when is the deductive approach used?

Answer 41

when clear theoretical assumptions about the construct exists

Question 42

what should one be aware of when generating the items?

Answer 42

when generating the items one should be aware of the response format (e.g. Likert scale)

Question 43

what is the goal of the item generation process?

Answer 43

The goal of the item generation process is to generate much more items than are needed for the final scale because one will select the best items based on their psychometric properties.

Question 44

what should you think about when generating items?

Answer 44

- redundancy - writing process - positive and negative wording of items

Question 45

what are bad items (4)?

Answer 45

- Exceptionally lengthy - Unnecessarily wordy - Multiple negatives - Double-barreled items

Question 46

what are good items?(2)

Answer 46

- Unambiguous - Targets the appropriate reading level for the intended sample

Question 47

what is content adequacy assessment?

Answer 47

assesses whether items are really adequate for a specific construct

Question 48

common approach of content adequacy assessment

Answer 48

* categorize or sort items based on their similarity to construct definitions * respondents are presented with construct definitions without titles and are asked to match items with a corresponding definition * can be done by experts or by non-experts * maximizes content validity

Question 49

how big should the sample size at least be?

Answer 49

Sample size should be at least 100 to 150 participants depending on the statistical approach

Question 50

what does the sample size also depend on?

Answer 50

The sample size also depends on the number of items (more items -> larger sample).

Question 51

what is a factor analysis?

Answer 51

Is a statistical method used to describe variability among observed, correlated variables in terms of a potentially lower number of unobserved variables called factors.

Question 52

what is the general goal of the factor analysis?

Answer 52

to determine the structure of a specific construct and investigate which items are most strongly related to a specific scale or subscale

Question 53

what is an exploratory factor analysis used for?

Answer 53

used to reduce the set of observed variables to a smaller, more parsimonious set of variables

Question 54

what is the objective of an exploratory factor analysis?

Answer 54

the objective is to identify those items that most clearly represent the content domain of the underlying construct

Question 55

what does the confirmatory factor analysis assess?

Answer 55

Assesses the quality of the factor structure by statistically testing the significance of the overall model.

Question 56

which items should be eliminated?

Answer 56

Items that do not ‘load’ on specific factors are eliminated.

Question 57

what should items of one scale be?

Answer 57

Items of one scale should be highly correlated

Question 58

what should items of different sales be?

Answer 58

Items of different scales should be less correlated.

Question 59

what should good items have?

Answer 59

large variance

Question 60

what does internal consistency determine?

Answer 60

how well the items of a scale measure the same construct.

Question 61

what is chronbachs alpha?

Answer 61

* Indicates the degree of interrelationship among the items of a scale (how strongly they are correlated. * Items are deleted that impact the reliability of the scale negatively.

Question 62

types of construct validity (2)

Answer 62

- convergent validity - discriminant validity

Question 63

what is convergent validity?

Answer 63

scores on a scales measuring a construct should relate to scores on other tests that are theoretically related to the construct

Question 64

what is discriminant validity?

Answer 64

scales measuring some construct should not be related to scores on other measures that are theoretically unrelated to the construct

Question 65

3 ways of replication

Answer 65

* Replicate the findings in a different sample. * Alternatively one can also split the original sample and test the factor structure in the two halves of the sample. * One can use a confirmatory factor analysis for replication.