1
Q
Static Testing
What does static testing rely on?
A
- It relies on the manual examination of work products (F.E: Reviews)
- Or tool-driven evaluation of the code or other work products (F.E: Static Analysis)
2
Q
Static Testing
For both types of static testing, does it need to actually execute the code or work product being tested when the testers assess the code?
A
No, it does not actually execute the code or work product being tested.
3
Q
Static Testing
When is “Static Analysis” applied?
A
Static analysis:
1. Applied to any work product with a formal structure (code or models)
- Applied when evaluating work products written in natural language like requirements.
4
Q
Static Testing
When is “Review” applied?
A
Review:
1. Applied to any work product that the participants know how to read and understand.
Example: Manual
5
Q
Static Testing Basics
Reviews and Tools:
Tell me things you know about reviews and tools for static testing.
Hint: 4 points
1. The formality of reviews?
2. What types of tests can tools perform?
3. When is it typically used?
4. What other areas can it be used on?
A
- Reviews can range from informal to very formal
- Tools can perform some types of static tests (Ex: Checking for coding standards/compliance)
- Typically used for requirements and design
- Can and should also be used in code, database schemas, documentation, tests, and written work product.
6
Q
Static Testing Basics
Use models and prototypes:
Why use diagrams?
A
A diagram of a complex system can often reveal design problems hidden in text/words.
Note: An ugly diagram often means a lot of bugs
7
Q
Static Testing Bascis:
Creation of test cases and data is a form of static testing:
Why is it so?
A
- Test analysis and design based on requirements and design specifications is a form of structured review.
- Test analysis and design often reveal problems, thus preventing bugs
8
Q
Static Testing Basics: Static Tools
Static Analysis: What do you know about static analysis?
Hint: 3 points
1. What can it check?
2. What does it check?
3. What can it measure?
A
- It can check for problematic wording: using spelling/ grammar checkers
- It checks for dangerous programming constructs , using tools like J-Test, Safer C, Lint..
- It can measure the complexity of code: complexity analysis
9
Q
Static Testing Basics: Static Tools
System simulations: What are the 3 system simulations?
A
- General purpose system simulator: Simulate basic system components like queue and resources.
- Performance modelling / operations research tools: Predict systems under load
- Spreadsheets: Simulate system behaviour like performance, and also functionality.
10
Q
Work products that can be examined by Static Testing:
Almost any work product can be examined using static testing (reviews/static analysis), what do they include?
A
- Specifications (Business requirements, functional requirements, security requirements) and also Architecture and design specifications.
- Epics, user stories, and acceptance criteria
- Code
- Testware, including test plans, test cases, test procedures, and automated test scripts
- User guides
- Web pages
- Contracts, project plans, schedules, and budgets
- Models, such as activity diagrams
11
Q
What are the benefits of static testing?
A
- Shorter schedules (due to early / efficient bug removal), reducing development cost and time, and also reducing testing cost and time
- Improved quality of product (reduces downstream costs, both in development and after releases)
- Detecting and correcting defects more efficiently
- Prevent defects in design or coding by uncovering inconsistencies, ambiguities, contradictions, omissions, inaccuracies, and redundancies in requirements.
- Increase development productivity ( Improved design, more maintainable code)
- Improve communication between team members in the course of participating in reviews.
12
Q
What are the similarities and differences between static and dynamic testing?
A
Similarities:
- They both seek to identify defects.
- Work best when a broad cross-section of stakeholders are involved.
- Save the company time and money.
Differences:
1. Each technique can find different types of defects more efficiently than the other.
2. Static techniques find defects rather than failures (Like chapter 1: Mistake, Defects, Failures)
3. Static- Test object is not being executed and Dynamic- test object is being executed.
13
Q
Typical defects found and fixed through static testing (easier and cheaper to):
A
- Requirement defect
- Design defect
- Coding defect
- Maintainability defects
- Deviations from standards
- Incorrect interface specifications
- Security vulnerabilities
- Gaps or inaccuracies in test basis traceability or coverage
14
Q
Static Test
3 questions:
1. What is the definition of static tests?
2. What is the objective of static tests?
3. What is the basic idea of static tests?
A
- Definition: Analysis of a test object via intensive observation (mostly documents).
- Objective: Detection of faults or defects in an artefact.
- Basic idea: Defect prevention
15
Q
Reviews:
- What does it observe and how?
- How many people?
- What are used to test and evaluate the work products?
- What software work product can be reviewed?
A
- Observation of the test object and manually
- Manual testing done by one or more people
- Human analysis and thinking capabilities are used to test and evaluate work products
- Any software work product can be reviewed (requirements, specifications, design specification, code, testware)
16
Q
Review Process
- What is the formality of the review process?
- What are the characteristics of an informal review?
- What are the characteristics of a formal review?
- How is a formality of a review process defined?
- What is the focus on the review process?
A
- Reviews may range from informal to formal.
- Informal reviews are characterised by not following a defined process & not having a formal documented output.
- Formal reviews are characterised by team participation, documented results of the review, and documented procedures for conducting the review.
- The formality is related to factors such as the SDLC, maturity of the development process, complexity of work product to be reviews, requirements, and any legal regulatory requirements.
- The focus depends on the agreed objectives of the review.
17
Q
The phase of the review process
Explain the “Planning” phase in the Review process
A
- You define the scope: The purpose of the review, documents or part of documents to reviews, and the quality characteristics to be evaluated.
- You estimate the effort and timeframe.
- Identify review characteristics such as the review type, roles, activities, and checklists.
- Selecting the people to participate in the review and assigning roles.
- Defining the exit and entry criteria for more formal review types.
18
Q
The phase of the review process
Explain the “Initiate Review” phase in the Review process
A
- Distributing the work product and other material such as issue log forms, checklists, and related work products.
- Explain the scope, objective, process, roles, and work products to the participants in the review process.
- Answering any questions that participants may have about the review process.
19
Q
The phase of the review process
Explain the “Individual Review” phase in the Review process.
AKA Individual preparation
A
- Review all part of the work product.
- Noting down the potential defects, recommendations, and questions.
20
Q
The phase of the review process
Explain the “Issue communication and analysis” phase in the Review process
A
- Communicating identified potential defects.
- Analysing potential defects, assigning ownership, and status to them.
- Evaluate and document quality characteristics.
- Evaluate the review findings against the exit criteria, to make a review decision. (Reject as not done or accept it that it is done)
21
Q
The phase of the review process
Explain the “Fixing and reporting” phase in the Review process
A
- Creating defect reports for findings that require changes.
- Fixing defects found in the work product reviewed.
- Communicating defects to the appropriate person or team.
- Recording updated status of defects.
- Gathering metrics
- Checking that the exit criteria are met.
- Accept the work product when the exit criteria is reached.
22
Q
Formal Review
What are the roles in a formal review?
A
- Author
- Manager
- Facilitator (moderator)
- Review Leader
- Reviewers (inspectors)
- Scribe (recorder)
23
Q
Roles and responsibilities in a: Formal Review
What is the R&R for the “Author” in a formal review?
A
- Creates the work product that is under review.
- Fixes defects in the work product under review.
24
Q
Roles and responsibilities in a: Formal Review
What is the R&R for the “Manager” in a formal review?
A
- Responsible for review planning (chooses the test objects and documents to be used).
- Decides on the execution of reviews.
- Assigns staff, budget, and time.
- Monitors ongoing cost-effectiveness.
- Executes control decisions in the event of inadequate outcomes.
25
Roles and responsibilities in a: Formal Review
What is the R&R for the "Facilitator" in a formal review?
1. Ensures effective running of review meetings.
2. Mediates, if necessary, between the various points of view.
Note: The facilitators are only involved in the meetings only during the whole review process.
26
Roles and responsibilities in a: Formal Review
What is the R&R for the "Review Leader" in a formal review?
1. Takes overall responsibility for the review.
2. Decides who will be involved and organises when and where it will take place.
It is like a team leader in the project, they understand technically what is going on and ensure everything is executed.
27
Roles and responsibilities in a: Formal Review
What is the R&R for the "Reviewers" in a formal review?
1. May be subject matter experts or individuals with specific technical or business backgrounds.
2. Identify potential defects in the work product under review.
28
Roles and responsibilities in a: Formal Review
What is the R&R for the "Scribe" in a formal review?
1. Collates potential defects found during the individual review activity.
2. Records new potential defects, open points, and discussions from the review meeting.
29
There are 2 different types of Reviews, what are they?
1. Management Reviews:
Analyse the project plan and the development project itself.
2. Product Reviews:
Review products or partial products which are being created during a development process. There are a few types of product reviews:
- Informal Review
- Walkthrough
- Technical Review
- Inspection
30
Informal Review (Pair Review)
Answer some questions about the Informal Review:
1. What is the main purpose?
2. What other additional purposes does informal review have?
3. What is it not based on?
4. Will the results be documented?
5. How useful is it?
6. What is optional?
7. Where is it commonly used in?
1. Main purpose is to detect potential defects.
2. Other purposes include generating new ideas or solutions, and quickly solving minor problems.
3. It is not based on a formal documented process.
4. The results may be documented.
5. Varies in usefulness depending on the reviewers.
6. Use of checklists is optional.
7. Commonly used in agile development.
31
Walkthrough
Answer some questions about the Walkthrough process:
1. What are the main purposes?
2. What are other possible purposes?
3. Who is it typically led by?
4. Who has a mandatory role in the walkthrough?
5. What is optional?
6. What are the different type of formalities practiced?
1. The purposes include:
i) Find defects
ii) Improve software product
iii) Consider alternative implementation
iv) Evaluate conformance to standards and specifications.
2. Other potential purposes include:
i) Exchanging ideas about techniques and style variations.
ii) Training of participants.
iii) Reaching a consensus.
3. It is typically led by the author of the work product.
4. Scribe is mandatory.
5. The use of checklists is optional
6. May vary in practices from informal to very formal.
32
Technical Review
Answer some questions about the Technical Review:
1. What are the main purposes?
2. What are other purposes?
3. What is required before the review meeting?
4. What is optional?
5. What role is mandatory?
6. The usage of what is optional?
7. What is typically produced during a technical review?
1. Gain consensus, detecting potential defects.
2. Evaluate quality and build confidence in the work product, generating new ideas, motivating and enabling authors to improve future work products.
3. Individual preparation before the review meeting is required.
4. Review meeting is optional, ideally led by a trained facilitator.
5. Scribe is mandatory, ideally not the author.
6. Use of checklist is also optional.
7. Potential defect logs and review reports are typically produced.
33
Inspection
Answer some questions about the Inspection:
1. What are the main purposes?
2. What are possible further purposes?
3. What does it follow?
4.
1. Main purpose:
- Detecting potential defects
- Evaluating quality and building confidence in the work product
- Prevent future and similar problems through author learning and root cause analysis.
2. Other possible purposes:
- Motivating and enabling authors to improve further work products and achieving consensus.
3. Follows a defined process with formal documented outputs.
4. Uses clearly defined roles, which is mandatory. May also include a dedicated reader.
5. Individual preparation before the review meeting is required.
6. Specified entry and exit criteria.
7. Scribe is mandatory.
8. Review meeting is led by a trained facilitator.
9. Author cannot acts as the review leader, reader, or scribe.
10. Potential defect logs and review report are produced.
11. Metrics are collected and used to improve the software development process.
34
Applying Review Techniques:
The effectiveness of the techniques may differ depending on the type of review used.
What are the different review techniques? (Hint: There is 5 techniques)
1. Ad hoc
2. Checklist-based
3. Scenarios and dry runs
4. Role-based
5. Perspective-based
35
Review Techniques:
Answer some questions about the "Ad hoc" reviewing technique:
1. It is a commonly used technique? and why?
2. Are reviewers provided guidance?
3. What do reviewers do?
4. What is it highly dependent on?
5. What is the drawback of this technique?
1. It is a commonly used technique because it needs only little preparation.
2. Reviewers are provided with little to no guidance on how this task should be performed.
3. Reviewers identify and document issues as they encounter them.
4. Ad hoc reviewing technique is highly dependent on the reviewers skills.
5. May lead to many duplicate issues being reported by different reviewers.
36
Review Techniques:
Answer some questions about the "Checklist-based" reviewing technique:
(Hint: "Systematic")
1. What is the checklist-based technique?
2. What does the checklist consist?
3. What is the main guideline for checklist-based technique?
4. What is the advantage of this technique?
1. It is a systematic technique, whereby the reviewers detect issues based on checklists that are given at review initiation.
2. The checklist consists of a set of questions based on potential defects.
3. Checklists should be specific to the type of work product that is being reviewed & also maintained regularly.
4. The advantage of checklist-based is a systematic coverage of typical defect types.
37
Scenarios and dry runs:
Answer some questions about the "Scenarios and dry runs" reviewing techniques:
1. What are reviewers provided with?
2. How does the scenarios help reviewers?
1. Reviewers are provided with structured guidelines on how to read through the work product.
2. The scenarios provide reviewers with better guidelines on how to identify specific defect types.
38
Role-based:
What is role-based reviewing technique?
1. A technique in which the reviewers evaluate the work product from the perspective of individual stakeholder roles.
39
Perspective-based:
Answer some questions about the "Perspective-based" reviewing techniques:
1. What is perspective-based?
2. What are the typical viewpoints?
3. Why do you use different stakeholder viewpoints?
4. What does this technique require stakeholders to do?
5. What is expected to be used?
1. It is similar to a role-based review, reviewers take on different stakeholder viewpoints in individual reviewing.
2. Viewpoints include end user, marketing, designer, tester or operations.
3. Using different stakeholder viewpoints leads to more depth in individual reviewing and less duplication of issues between reviewers.
4. Requires the reviewers to attempt to use the work product to generate the product they would derive from it.
5. Checklists are expected to be used.
6. Most effective general technique for reviewing requirements and technical work products.
40
Success factors for review: Business side:
1. Each review has clear objectives that is defined during review planning , and used as measurable exit criteria.
2. Suitable review types are applied which are needed to achieve the objectives.
3. Management supports the review process.
-
41
Success factors for review: People side:
1. The right people are involved in the process.
2. Participants dedicate enough time and attention to details.
3. Reviews are conducted in small chunks so that reviewers do not lose concentration.
4. Defects found are acknowledged, appreciated, and handled objectively.
-
Software Testing
flashcards
Decks in class (3)
# Cards
