Evidence-based software engineering

Question 1

Evidence-based Software Engineering

Answer 1

• Concerned with determining what works, when and where in terms of software engineering technologies
• Uses empirical research as a means to create evidence
• Ongoing effort, applicable EVERYWHERE in SENG.

Question 2

Directions for creating evidence

Answer 2

• Technology evaluation (effort, time, performance usability)
• Find strength and limitations of technologies
• Find preferred areas of applications of technologies
• Comparison between technologies
• Project management (plan, monitor, control)
• Quality prediction, evaluation, improvement

Question 3

Goal Definition

Answer 3

Object of interest, purpose, focus, point of view, context

• Analysis perspective-based reading (object of interest)
• In order to evaluate (purpose)
• With respect to defect detection (focus)
• From the point of quality assurance (point of view)
• For environment XYZ (context)

Specify the context as carefully as possible

Specify research hypothesis and discuss the theory/sources from which it is derived

Describe research that is similar to the current research and how current work relates to it

Question 4

Validity

Answer 4

Results are valid if they are representative for the population to which we would like to generalize.

Question 5

Internal Validity

Answer 5

• Degree to which conclusions can be drawn about the casual effect of independent variable(s) on the dependent variable(s)
• Depends mainly upon the empirical design used
• Maximised at the expense of external validity

Question 6

Construct Validity

Answer 6

• Degree to which the independent and dependent variables accurately measure the concepts they are intended to measure.

Question 7

External Validity

Answer 7

• Degree to which the results of the research can be generalized to the population under study and other research settings.
• Usually achieved at the expense of internal validity.
• Potential risks:
  
  • Characteristics of the sample: is the sample representative of the population under study
  • Hawthorne effect: the effect on subjects knowing they are in an experiment
  • Ability to scale up: are the empirical tasks, performed, measures taken, the environment used representative of the ‘real world’ or the organization.

Question 8

Qualitative Research

Answer 8

Concerned with the study of ‘objects’ in their natural setting.

• Involves making interpretations that are based upon explanations
• Accepts that there may well be different interpretations of a phenomenon
• Concerned with discovering the causes of effects/behavior.
• Ex. Why different inspection groups perform differently?

Question 9

Quantitative Research

Answer 9

Mainly seeks to quantify a relationship, or to compare two or more groups

• Employed to test the effect of some intervention
• Lends itself to statistical analysis
• Ex. Determine whether introducing new inspection method can increase the number of faults found during testing?

Question 10

Participants

Answer 10

• Many empirical studies in SE involve people, usually refered to as participants.
• Recruiting participants:
  
  • Involves ethical issues-recruiting
  • Should not put people under pressure to participate.
  • As far as possible they should be guaranteed anonymity
  • Should aim to obtain a representative sample from the domain of interest or a surrogate domain.
  • Students are often used as participants.
  • Care needed about how representative they are.