What is Software Testing?
Software Testing is a way to assess the quality of a software and to reduce the chances of failure during operation.
How many activities are in Software Testing and what are they?
Software Testing constitutes of 6 activities:
1. Planning
2. Analysing
3. Designing
4. Implementing tests
5. Reporting test progress and results
6. Evaluating the quality of test object
What are the typical objectives of testing?
- Evaluate work products such as requirements, user stories, design and code.
- Verify all requirements have been fulfilled.
- Validate whether the test object is complete AND works as expected by user.
- Build confidence in the quality of the test object.
- Prevent defects.
- Find the failures and defects.
- Provide information to stakeholders to allow them to make informed decisions (can be of the quality of the test object).
- Reduce the risk of unacceptable software quality.
- Ensure the test object complies with regulatory standards
What is the difference between Testing vs Debugging?
- Testing identifies errors and defects in the software.
- Debugging is the DEVELOPMENT ACTIVITY that finds, analyses, and fix those defects found in testing.
Why is testing NECESSARY?
- People make mistakes and the bugs could be anywhere. From requirements & design specification to code and documentation.
- And bugs lead to defective code. Once defective code is executed, failures occur. If the failures are seen/experienced by the stakeholders, dissatisfaction with the system quality may occur.
What are the impact of software failures? (Hint: 3 risks)
- Business risks
- Damaged reputation for quality
- High or unpredictable maintenance cost
- Unexpected delays in release of product
- Lack of confidence in system
- Lawsuits - Environmental risk
- Pollution & waste - Risks to people societies & states:
- Lost jobs, lives, rights, missions, wars
What is the difference between Error vs Defect vs Failure? (Hint: Domino effect)
- Error: People make mistakes and these leads to errors.
- Defect: The errors results in a defect in the software.
- Failure: A failure is the effect of a defect when executing a program.
What is the difference between Validation and Verification?
(Hint: Did we implement the system right? and Did we implement the right system?)
Validation: Confirmation by examination that the requirements have been fulfilled. Did we implement the right system?
Verification: Confirmation by examination that the specified requirements have been fulfilled. Did we implement the system right?
Can we test exhaustively? State why it is impossible or possible.
No, it is impossible to perform complete testing or exhaustive testing.
Because:
1. Input that needs to be tested for a domain may be too large.
2. The design issues may be too complex to completely test
3. It may not be possible to create all possible execution environments of the system.
What are the 7 Testing principles?
- Testing shows the presence of defects, not their absence.
- Exhaustive testing is impossible.
- Early testing saves time and money.
- Defects cluster together.
- Beware of the pesticide paradox.
- Testing is context dependent.
- Absence of errors fallacy.
Explain the principle: Testing shows presence of defects, not their absence
- Testing can show that the defects are present, but cannot prove that there are no defects.
(Even though testing reduces the probability of undiscovered defects in the software, even if there is no defects found, testing is not a proof of correctness)
Explain the principle: Exhaustive testing is impossible
Testing everything (including the combinations of input and preconditions) is not feasible.
Explain the principle: Early testing saves time and money (Shift-left)
To find defects as early as possible, both static and dynamic test activities should be started as early as possible in the SDLC. It helps reduce or eliminate costly changes.
Explain the principle: Defects cluster together
A small number of modules usually contains most of the defects discovered during pre-release testing, and usually is responsible for most of the operational failures.
Explain the principle: Beware of the pesticide paradox
If the same tests are repeated over and over again, eventually the tests will no longer find any new defects in the software.
& To detect new defects, existing tests may need changing, so new tests may need to be written.
Explain the principle: Testing is context dependent
Testing is done differently in different contexts. No one fix-all method for testing software.
Explain the principle: Absence-of-errors- is a fallacy
(Fallacy: Mistaken-belief)
Some people believe that testers can run all possible tests and find all possible defects, but principles 2 and 1, tells us that it is impossible.
& that even though fixing all defects it does not ensure that the software fulfil the user’s needs and expectations.
What does a test process consist of? (Hint: 7 activities)
- Test planning
- Test monitoring and control
- Test analysis
- Test design
- Test implementation
- Test execution
- Test completion
What do you know about “Test planning”?
Test planning involves the activities that define the objectives of testing & the approach for meeting the test objectives
(Specifying suitable test techniques, tasks, and formulation a test schedule for deadline)
What do you know about “Test Monitoring and Control”?
Testing monitoring involves the comparison of actual progress against the test plan. Using any test monitoring metrics defined in the test plan.
& Test control involves taking actions necessary to meet the objectives set in the test plan.
What do you know about “Test Analysis”? (What to test?)
Test analysis determines “what to test” in terms of measurable coverage criteria.
The test basis is analysed to identify testable features and define test conditions.
Major activities:
1. Analysing the test basis
2. Identify features to be tested
3. Defining and prioritising test conditions for each feature
4. Evaluate the test basis and test items to identify defects of various types.
What do you know about “Test Design”? (How to test?)
During test design, the test conditions are elaborated into high-level test cases, and other testware.
Major activities:
1. Designing and prioritising test cases
2. Designing the test environment and identifying any required infrastructure and tools.
3. Identifying necessary test data to support test conditions and test cases.
What do you know about “Test Implementation”? (Do we have everything in place to run the tests?)
During test implementation, the testware necessary for test execution is created/completed.
& sequencing the test cases into test procedures.
What do you know about “Test Execution”?
During test execution, tests are conducted and according to the test execution schedule.
- Recording the IDs and versions of the test item(s)
