software dev team members
developer (writes implementation)
tester (writes specification)
and manager
4 observations
- specifications must be explicit
- independent dev and testing
- resources are finite
- specifications evolve over time
testing checks
consistency between specification and implementation
classification of testing approaches
manual/automated vs black box/ white box
advantages of automated testing
- find bugs more quickly
- no need to write tests
- no need to maintain tests if software changes
advantages of manual testing
- efficient test suite
- potential better coverage (not guarantee)
advantages of black box testing
- can work with codes that cannot be modified
- does not need to analyze or study codes
- code can be any format
advantages of white box testing
- efficient test suite
- potential better coverage
disadvantages of automated testing
- hard to do
- impossible for entire systems
- impossible without specification
pre/post conditions
- predicates assumed to hold before/after a function executes
- most useful when they are executable
- need not be precise (may become more complex than the code, but is useful even if they don’t cover every situations)
common problems of a test suite
- too few test: may miss bugs
- too many tests: costly to run, bloat, and redundancy, harder to maintain.
2 approaches to test suite validation
- code coverage
- mutation analysis (or mutation testing)
code coverage
- metric to quantify the extent to which a program code is tested
- measured in % (100% is rare in practice)
type of code coverage
- function coverage
- statement coverage
- branch coverage
- and many others
mutation analysis
- test variation (mutants) of the program
- the mutant can be equivalent to the original
frame conditions
the assumption about everything else outside of the pre/post condition
mutation analysis adequates if
mutant fails but the original code passes
problem with mutation analysis
if mutant is equivalent to the original
