1
Q
Test techniques classification
A
- Black-box test techniques
- White-box test techniques
- Experience-based test techniques
2
Q
Black-box test techniques
(specification-based techniques)
A
- based on an analysis of the specified behavior of the test object without reference to its internal structure.
- the test cases are independent of how the software is implemented
- black-box testing does not provide a measure of actual code coverage.
3
Q
White-box test techniques
(structure-based techniques)
A
- based on an analysis of the test object’s internal structure and processing
- can only be created after the design or implementation of the test object
4
Q
Experience-based test techniques
A
use the knowledge and experience of testers for the design and implementation of test cases. The effectiveness of these techniques depends heavily on the tester’s skills. Experience-based test techniques can detect defects that may be missed using the black- box and white-box test techniques.
5
Q
Black-Box Test Techniques
A
- Equivalence Partitioning
- Boundary Value Analysis
- Decision Table Testing
- State Transition Testing
- Use Case Analysis
6
Q
Equivalence Partitioning Assumption
A
if a test case, that tests one value from an equivalence partition, detects a defect, this defect should also be detected by test cases that test any other value from the same partition => one test is sufficient
7
Q
Data elements related to the test object
A
- inputs
- outputs
- configuration items
- internal values
- time-related values
- interface parameter
8
Q
Valid vs invalid values
A
- definition varies among teams and organizations
- valid values - those that should be processed by the test object or as those for which the specification defines their processing.
- Invalid values - those that should be ignored or rejected by the test object or as those for which no processing is defined in the test object specification.
9
Q
Coverage (EP)
A
the number of partitions exercised by at least one test case, divided by the total number of identified partitions, and is expressed as a percentage.
10
Q
Each Choice coverage
A
- test objects include multiple sets of partitions (multiple inputs)
- exercise each partition from each set of partitions at least once.
- does not take into account combinations of partitions
11
Q
Boundary Value Analysis Assumptions
A
- can only be used for ordered partitions.
- if two elements belong to the same partition, all elements between them must also belong to that partition.
12
Q
Boundary Value Analysis Motivation
A
- developers are more likely to make errors with these boundary values.
- Typical defects found by BVA are located where implemented boundaries are misplaced to positions above or below their intended positions or are omitted altogether.
13
Q
2-value BVA
A
- for each boundary value there are two coverage items: this boundary value and its closest neighbor belonging to the adjacent partition.
- overage is measured as the number of boundary values that were exercised, divided by the total number of identified boundary values, and is expressed as a percentage.
14
Q
3-value BVA
A
- for each boundary value there are three coverage items: this boundary value and both its neighbors.
- some of the coverage items may not be boundary values
15
Q
3-value BVA is more rigorous than 2-value BVA
A
- may detect defects overlooked by 2-value BVA
- , if the decision “if (x ≤ 10) …” is incorrectly implemented as “if (x = 10) …”, no test data derived from the 2-value BVA (x = 10, x = 11) can detect the defect. However, x = 9, derived from the 3-value BVA, is likely to detect it.
16
Q
Limited-entry decision tables
A
all the values of the conditions and actions (except for irrelevant or infeasible ones; see below) are shown as Boolean values (true or false)
17
Q
Extended-entry decision tables
A
some or all the conditions and actions may also take on multiple values (e.g., ranges of numbers, equivalence partitions, discrete values)
18
Q
Decision table
A
- full decision table has enough columns to cover every combination of conditions
- “N/A” means that the condition is infeasible for a given rule.
- The table can be simplified by deleting columns containing infeasible combinations of conditions. The table can also be minimized by merging columns, in which some conditions do not affect the outcome
19
Q
Decision table testing coverage
A
- coverage items are the columns containing feasible combinations of conditions
- the number of exercised columns, divided by the total number of feasible columns, and is expressed as a percentage.
20
Q
The strength of decision table testing
A
- provides a systematic approach to identify all the combinations of conditions, some of which might otherwise be overlooked
- helps to find any gaps or contradictions in the requirements
21
Q
A state transition diagram
A
models the behavior of a system by showing its possible states and valid state transitions
22
Q
state table
A
- model equivalent to a state transition diagram
- rows represent states, and its columns represent events
- Table entries (cells) represent transitions, and contain the target state, as well as the resulting actions, if defined.
- In contrast to the state transition diagram, the state table explicitly shows invalid transitions, which are represented by empty cells.
23
Q
A test case based on a state transition diagram
A
- a sequence of events, which results in a sequence of state changes (and actions, if needed)
- one test case may, and usually will, cover several transitions between states.
24
Q
Coverage criteria for state transition testing
A
- all states coverage
- valid transitions coverage
- all transitions coverage
25
all states coverage
1. coverage items are the states
2. Coverage is measured as the number of visited states divided by the total number of states, and is expressed as a percentage.
26
valid transitions coverage
1. the coverage items are single valid transitions
2. Coverage is measured as the number of exercised valid transitions divided by the total number of valid transitions, and is expressed as a percentage.
3. most often used
4. guarantees full all states coverage
27
all transitions coverage
1. all the transitions shown in a state table
2. Testing only one invalid transition in a single test case helps to avoid fault masking, i.e., a situation in which one defect prevents the detection of another
3. Coverage is measured as the number of valid and invalid transitions exercised or attempted to be covered by executed test cases, divided by the total number of valid and invalid transitions, and is expressed as a percentage
28
White-Box Test Techniques
1. Statement testing
2. Branch testing
29
The Value of White-box Testing
1. the entire software implementation is taken into account during testing => defect detection even when the software specification is vague, outdated or incomplete.
2. white box testing may not detect the resulting defects of omission
3. White-box coverage measures provide an objective measurement of coverage and provide the necessary information to allow additional tests to be generated to increase this coverage, and subsequently increase confidence in the code.
30
Statement Testing and Statement Coverage
1. the coverage items are executable statements
2. Coverage is measured as the number of statements exercised by the test cases divided by the total number of executable statements in the code
3. SC = min no. of paths required for covering all the nodes
4. SC = no. of else + 1
31
Branch Testing and Branch Coverage
32
Experience-based Test Techniques
1. Error guessing
2. Exploratory testing
3. Checklist-based testing
33
Statement Testing and Statement Coverage Problems
1. exercising a statement with a test case will not detect defects in all cases
2. may not detect defects that are data dependent
3. 100% statement coverage does not ensure that all the decision logic has been tested as, for instance, it may not exercise all the branches in the code
34
A branch in the code
1. a transfer of control between two nodes in the control flow graph, which shows the possible sequences in which source code statements are executed in the test object.
2. Each transfer of control can be either unconditional (i.e., straight-line code) or conditional (i.e., a decision outcome)
35
Branch Testing and Branch Coverage
1. coverage items are branches
2. the number of branches exercised by the test cases divided by the total number of branches
3. Branch coverage subsumes statement coverage
36
Branch Testing and Branch Coverage Problem
exercising a branch with a test case will not detect defects in all cases. For example, it may not detect defects requiring the execution of a specific path in a code.
37
Error Guessing
anticipate the occurrence of errors, defects, and failures, based on the tester’s knowledge, including:
1. How the application has worked in the past
2. The types of errors the developers tend to make and the types of defects that result from these errors
3. The types of failures that have occurred in other, similar applications
38
Fault attacks
1. a methodical approach to the implementation of error guessing.
2. requires the tester to create or acquire a list of possible errors, defects and failures, and to design tests that will identify defects associated with the errors, expose the defects, or cause the failures.
39
Errors, defects and failures may be related to
1. input (correct input not accepted)
2. output (wrong format)
3. logic
4. computation (incorrect operand, wrong computation)
5. interface (incompatible type)
6. data
40
Checklist-Based Testing
1. a tester designs, implements, and executes tests to cover test conditions from a checklist
2. It should be possible to check each item separately and directly
3. can be created to support various test types, including functional and non-functional testing
4. should be regularly updated
5. an provide guidelines and some degree of consistency for the testing.
6. less repeatability
41
Checklist should not contain
1. items that can be checked automatically
2. items better suited as entry/exit criteria
3. items that are too general
42
Exploratory Testing
1. tests are simultaneously designed, executed, and evaluated while the tester learns about the test object.
2. can incorporate the use of other test techniques
3. typically time boxes - range between 30 to 120 minutes
43
Exploratory testing is useful when
1. there are few or inadequate specifications or there is significant time pressure on the testing
2. also useful to complement other more formal test techniques
3. will be more effective if the tester is experienced, has domain knowledge and has a high degree of essential skills, like analytical skills, curiosity and creativeness
44
Session-based exploratory testing
1. exploratory testing is conducted within a defined time-box
2. uses a test charter containing test objectives to guide the testing
3. he test session is usually followed by a debriefing that involves a discussion between the tester and stakeholders interested in the test results of the test session.
4. Coverage items are identified and exercised during the test session.
45
Collaboration-based Test Approaches
focus also on defect avoidance by collaboration and communication.
1. Collaborative User Story Writing
2. Acceptance Criteria
3. Acceptance Test-driven Development (ATDD)
46
User story
a feature that will be valuable to either a user or purchaser of a system or software.
“As a [role], I want [goal to be accomplished], so that I can [resulting business value for the role]”, followed by the acceptance criteria.
47
3Cs of a user story
1. Card – the medium describing a user story
2. Conversation – explains how the software will be used (can be documented or verbal)
3. Confirmation – the acceptance criteria
48
Good user stories should be:
INVEST
1. Independent
2. Negotiable
3. Valuable
4. Estimable
5. Small
6. Testable
49
Acceptance Criteria
the conditions that an implementation of the user story must meet to be accepted by stakeholders.
50
Acceptance criteria are used to:
1. Define the scope of the user story
2. Reach consensus among the stakeholders
3. Describe both positive and negative scenarios
4. Serve as a basis for the user story acceptance testing
5. Allow accurate planning and estimation
51
Most common formats of acceptance criteria
1. Scenario-oriented (e.g., Given/When/Then format used in BDD)
2. Rule-oriented (e.g., bullet point verification list, or tabulated form of input-output mapping)
52
Acceptance Test-driven Development (ATDD)
1. Test cases are created prior to implementing the user story
2. test cases are created by team members with different perspectives, e.g., customers, developers, and testers
53
Steps of ATDD
1. The first step is a specification workshop where the user story and (if not yet defined) its acceptance criteria are analyzed, discussed, and written by the team members
2. create the test cases (ased on the acceptance criteria)
54
Test cases in ATDD
1. first test cases are positive, confirming the correct behavior without exceptions or error conditions
2. negative testing
3. non-functional
55
An integer field shall contain values between and including 1 to 15. By applying EP which of the following is a valid collection of equivalence classes for the given scenario
1. Less than 1 (invalid)
2. 1-15 (valid)
3. more than 15
56
Boundary values are calculated as (Boundary values of analysis)
LB, LB-1, RB, RB+1
NB: use tables - helpful
57
Use cases
1. Describe the interaction between actor and system
2. Helpful for deriving test cases for the integration testing, system testing and acceptance testing
58
A Path (white box)
Executable path which reaches the end of the code starting from start point
59
Decision Coverage (rules of thumb)
1. Un-nested if => 2
2. Nested = no. of if + 1
60
Statement Coverage
1. Un-nested: 2 if else are there, 1 if only ifs
2. Nested: no. of else + 1
61
Scenarios when only experience based techniques are applicable
1. specifications are poorly defined
2. time pressure
3. testing team is not formally trained in the testing fundamentals
62
Test charter (exploratory testing)
contains test objectives
ASY ISTQB
flashcards
Decks in class (8)
# Cards
