conventional testing
code first, test second
enforced by waterfall, V-model
conventional testing problems
static checking tools
used to find
* using undefined variables
* unreachable/dead code
* resource leaks
* infinite loops
white box testing
uses knowledge of the code
black box testing
focuses on inputs and outputs
only good if adequately covers the input
test-driven development
test are made before any development and code is written to make the tests pass
only write new code if an automated test has failed
dijkstra
program testing can be used to show the presense of defects but never their absence
generating tests to cover every possible input is not feasible
tdd flow
unit testing tools
property based testing
originally made for Haskell
specify rules that should always hold about a system and the test framework automatically generates inputs to verify those properties
property based testing benefits
ai systems: drift
bias that built the model is misaligned with current data
bias
systematic difference in treatment of certain objects people or groups in comparison to others
automation bias
propensity for humans to favour suggestions from automated decision-making systems and to ignore contradictory information made without automation even if it is correct
human-congnitive bias
bias that occurs when humans are processing and interpreting information and influences judgement and decision making
confirmation bias
type of human cognitive bias that favours predicitons of ai systems that confirm pre-existing beliefs or hypothesis
data bias
data properties that if unaddressed lead to ai systems that perform better or worse for different groups
statistical bias
type of consistent numerical offset in an estimate relative to the true underlying value inherent to most estimates