What are two main factors of measurements that we are concerned with when conducting a study?
Reliability and validity
What is reliability? What is an example? For qualitative studies how is this measured?
Reliability: About the consistency of a measure. If you measure the same thing with the same measure in the same context, are you going to get the same result each time?
AKA test-retest reliability
For qualitative studies, this is looking at inter-rater reliability. How similarly are researchers interpreting the data and how is this influencing results? The more researchers you have coding the data, the better you can test for inter-rater reliability and the more reliable you can prove your study is.
An example of this is do two researchers observing the same phenomenon record the same data?
DOESN’T MATTER THE CORRECTNESS OF THE VALUES!
sometimes surveys can give different answers each time because you are not testing for the trait you wanted to, so using average or most commonly can help to counteract this.
What is validity? What is an example? What are two types of validity and why are they used?
Validity: About the accuracy of the data (is it actually measuring what you want to measure?)
Ex) Is a survey truly testing for self-esteem and not anxiety
Two types of validity can be used to test that accuracy.
Convergent validity: If you create another construct deeply related to the topic and the results are correlated to the results from the initial construct, then it proves that it is testing what you want it to.
Divergent validity: If you create another construct unrelated (or very slightly relate) to the construct, and there is no correlation between results, then it is also likely measuring what it intends to measure.
EX) If you have a survey for depression, then you have a survey known to measure sadness and happiness, if sadness is positively correlated and happiness is negatively correlated, then this indicates convergent validity. Then if you have a survey known to measure intelligence or colour preferences that are not related, you should have weak or zero correlations in results and this highlights divergent/discriminant validity.
What is external validity?
The extent to which results generalize beyond the specific context and population the study is focusing on. Does it apply in other settings or other individuals? This validity increases with larger sample size since you can be more sure the sample represents the population and thus the results represent the population.
What are some issues with sampling that make it not accurately represent the general population?
-too small a sample, can’t possibly represent all the different variances in the whole population
-not varied enough (only sampling some specific people whether purposeful or not) will impact the results
What is a census?
Gathering data from ALL members of the population.
What is the goal of random sampling? Why?
Ensure that the differences among participants are distributed evenly so that they don’t impact the data as those effects cancel out. DIFFERENT THAN RANDOM ASSIGNMENT. Just know there are different types of random sampling.
IT IS ALMOST NEVER POSSIBLE TO GET A TRULY RANDOM SAMPLE.
What is convenience sampling?
Convenience sampling is when you choose members from the population who are readily available but don’t necessarily represent the diversity of the entire population. Therefore data is likely not generalizable.
What type of sample are we trying to move away from?
Convenience sampling
Is selecting every 100th person from a voters list random or convenience?
It is more random then some but not everyone is necessarily on that list so still convenience.
What is the main issue with sampling that makes random sampling almost impossible?
People can self select out of the sample!
What is a statistic and a parameter and what is random sampling trying to do with these?
Statistic is a numerical characteristic of a value in the sample, so like the sample mean or median.
Parameter is the same thing but for the whole population.
