JC

Question 1

What are the observations?

Answer 1

• personal
• reported

Question 2

explanation

Answer 2

can it already be found?

Question 3

hypotheses

Answer 3

• null + alternative
• testable
• combine question with existing knowledge
• predictions
• experiments to test these predictions

Question 4

experimental design

Answer 4

• iterations and improvements
• does it test at multiple scales?

Question 5

findings

Answer 5

• how are they written up and presented?
• open
• peer reviewed
• conference/preprint
• social media/blog

Question 6

integration

Answer 6

• into existing knowledge
• general explanation/phenomenon

Question 7

headlines

Answer 7

• sensational?
• solved/prevented/cured
• accurate?

Question 8

Observing/questioning tips

Answer 8

• explain why the q matters to understanding the phenomenon
• make observations in different contexts/places; check that the problem/phenomenon is real
• financial feasibility
• does it consult prior literature/theory/experts?

Question 9

Observing/questioning pitfalls

Answer 9

• oversimplifying an existing field/minimising prior work
• not explaining why something is interesting/important
• confirmation bias
• not justifying why doing study matters

Question 10

confirmation bias

Answer 10

• assuming that a phenomenon exists or thinking that you know the answer, then forming a question retrospectively
• only testing predictions to confirm/neutral

Question 11

Hypotheses tips

Answer 11

• a priori
• true alternatives
• plausible
• falsifiable
• not tangential
• not too broad
• actually look at the question of interest

Question 12

true alternatives

Answer 12

attempt to address the same phenomenon

Question 13

Prediction tips

Answer 13

• clear
• measurable given technical, time and financial constraints
• pre-registration

Question 14

Hypotheses pitfalls

Answer 14

• confusing with predictions
• not mutually exclusive
• answer a complementary q
• post-hoc data mining

Question 15

what is the difference between a hypothesis and a prediction

Answer 15

hypotheses need context

Question 16

post-hoc data mining

Answer 16

HARKing

Question 17

prediction pitfalls

Answer 17

• confirmation bias
• unrelated

Question 18

Gathering/analysing data tips

Answer 18

• independent, representative sample
• appropriate study system picked and justified
• display, examine and present at all stages in a clear way
• display raw data
• appropriate stats test

Question 19

Gathering/analysing data pitfalls

Answer 19

• p and R values
• multiple tests without adjusting p value (p-hacking)
• pseudoreplication, autocorrelation
• confounding effects

Question 20

p-hacking

Answer 20

• missing data analysis techniques
• results in false significance

Question 21

give some examples of confounding effects

Answer 21

• contamination
• seasonal effects
• observer influence

Question 22

Sharing/writing up findings tips

Answer 22

• measured, objective language in conclusions
• communicate clearly and fairly
• simplify without exaggerating
• discuss level of confidence in results
• limit speculation/extrapolation
• discuss possible alternative interpretations
• discuss limitations
• suggest further work
• colour-blind friendly graphics

Question 23

Sharing/writing up findings pitfalls

Answer 23

• prematurely assertive language

Question 24

prematurely assertive language

Answer 24

• “conclusively shows”
• “beyond doubt”
• “proves”
• “confirms”