1
Q
Testing and Development Activities
A
- At the beginning of the project, there is a release planning period.
- This is followed by a sequence of iterations. At the beginning of each iteration, there is an iteration planning period.
- Once iteration scope is established, the selected user stories are developed, integrated with the system, and tested.
- These iterations are highly dynamic, with development, integration, and testing activities taking place throughout each iteration, and with considerable parallelism and overlap.
- Testing activities occur throughout the iteration, not as a final activity.
2
Q
Testers, developers, and business stakeholders all have a role in testing, Explain why?
A
- Developers perform unit tests as they develop features from the user stories.
- Testers then test those features.
- Business stakeholders also test the stories during implementation.
- Business stakeholders might use written test cases, but they also might simply experiment with and use the feature in order to provide fast feedback to the development team.
3
Q
Testing and Development Activities: The best practice is that no feature is considered done until when?
A
- It has been integrated and tested with the system
- To address defects remaining from the previous iteration at the beginning of the next iteration, as part of the backlog for that iteration (referred to as “fix bugs first”).
4
Q
Testing and Development Activities: When risk-based testing is used as one of the test strategies, what happens?
A
- High-level risk analysis occurs during release planning, with testers often driving that analysis
- The specific quality risks associated with each iteration are identified and assessed in iteration planning
- This risk analysis can influence the sequence of development as well as the priority and depth of testing for the features.
- It also influences the estimation of the test effort required for each feature.
5
Q
Testing and Development Activities: Extreme Programming:
A
- Pairing is used.
- Pairing can involve testers working together in twos to test a feature
- Pairing can also involve a tester working collaboratively with a developer to develop and test a feature.
5
Q
Testing and Development Activities: Extreme Programming:
A
- Pairing is used.
- Pairing can involve testers working together in twos to test a feature
- Pairing can also involve a tester working collaboratively with a developer to develop and test a feature.
6
Q
Testing and Development Activities: Test automation at all levels of testing occurs in many Agile teams, Why?:
A
- Because of the heavy use of test automation, a higher percentage of the manual testing on Agile projects tends to be done using experience-based and defect-based techniques such as software attacks, exploratory testing, and error guessing.
7
Q
Testing and Development Activities: Why do Agile teams tend to favor testers with a strong technical and test automation background?
A
*While developers will focus on creating unit tests, testers should focus on creating automated integration, system, and system integration tests
8
Q
Testing and Development Activities: Why is lightweight work product documentation is favored in Agile projects?
A
One core Agile principle is that change may occur throughout the project.
9
Q
Testing and Development Activities: Changes to existing features have testing implications, especially regression testing implications. What is the one way to manage it?:
A
- The use of automated testing is one way of managing the amount of test effort associated with change.
- However, it’s important that the rate of change not exceed the project team’s ability to deal with the risks associated with those changes.
10
Q
Project Work Products: Project work products of immediate interest to Agile testers typically fall into three categories:
A
- Business-oriented work products that describe what is needed (e.g., requirements specifications) and how to use it (e.g., user documentation)
- Development work products that describe how the system is built (e.g., database entity-relationship diagrams), that actually implement the system (e.g., code), or that evaluate individual pieces of code (e.g., automated unit tests)
- Test work products that describe how the system is tested (e.g., test strategies and plans), that actually test the system (e.g., manual and automated tests), or that present test results (e.g., test dashboards as discussed in Section 2.2.1)
11
Q
Project Work Products: Typical business-oriented work products on Agile projects include?
A
- User stories and acceptance criteria.
- User stories are the Agile form of requirements specifications, and should explain how the system should behave with respect to a single, coherent feature or function.
- A user story should define a feature small enough to be completed in a single iteration.
12
Q
Project Work Products: Define Epic?
A
- Larger collections of related features, or a collection of sub-features that make up a single complex feature.
- Epics may include user stories for different development teams.
- Each epic and its user stories should have associated acceptance criteria.
13
Q
Project Work Products: Which approach is referred to as test first or test-driven development?
A
- Tests might be created after the development of code. In some cases, though, developers create tests incrementally, before each portion of the code is written, in order to provide a way of verifying, once that portion of code is written, whether it works as expected.
14
Q
Project Work Products: Typical tester work products on Agile projects include?
A
- Automated tests, as well as documents such as test plans, quality risk catalogs, manual tests, defect reports, and test results logs.
- Testers will also produce test metrics from defect reports and test results logs
15
Q
Project Work Products: Which work products may be prepared to satisfy auditors, regulations, and other requirements?
A
- Teams transform user stories and acceptance criteria into more formal requirements specifications. Vertical and horizontal traceability reports.
15
Q
Project Work Products: Which work products may be prepared to satisfy auditors, regulations, and other requirements?
A
- Teams transform user stories and acceptance criteria into more formal requirements specifications. Vertical and horizontal traceability reports.
16
Q
Test Levels: What are Test Levels?
A
- Are test activities that are logically related, often by the maturity or completeness of the item under test.
17
Q
Test Levels: In sequential lifecycle models, the test levels are often defined such that the exit criteria of one level are part of the entry criteria for the next level. In some iterative models, this rule does not apply. Why?
A
- Test levels overlap. Requirement specification, design specification, and development activities may overlap with test levels.
18
Q
Test Levels: During an iteration, any given user story will typically progress sequentially through the following test activities:
A
- Unit testing, typically done by the developer
- Feature acceptance testing, which is sometimes broken into two activities:
- Feature verification testing, which is often automated, may be done by developers or testers, and involves testing against the user story’s acceptance criteria
- Feature validation testing, which is usually manual and can involve developers, testers, and business stakeholders working collaboratively to determine whether the feature is fit for use, to improve visibility of the progress made, and to receive real feedback from the business stakeholders.
19
Q
Test Levels: Usually via a continuous integration framework, there is often a parallel process of regression testing occurring throughout the iteration, what does it involve?
A
- This involves re-running the automated unit tests and feature verification tests from the current iteration and previous iterations,
20
Q
Test Levels: In some Agile projects, there may be a system test level, when does it start?
A
- Starts once the first user story is ready for such testing. This can involve executing functional tests, as well as non-functional tests for performance, reliability, usability, and other relevant test types.
21
Q
Test Levels: Internal alpha tests and external beta tests may occur when?
A
- Either at the close of each iteration, after the completion of each iteration, or after a series of iterations.
22
Q
Test Levels: User acceptance tests, operational acceptance tests, regulatory acceptance tests, and contract acceptance tests also may occur when?
A
- Either at the close of each iteration, after the completion of each iteration, or after a series of iterations.
23
Testing and Configuration Management: Which frameworks allow the continuous integration of new software with the system?
* Developers use tools for static analysis, unit testing, and code coverage. Developers continuously check the code and unit tests into a configuration management system, using automated build and test frameworks
24
Testing and Configuration Management: In some cases, due to the duration of the functional tests when are functional tests ran?
* The functional tests are separated from the unit tests and run less frequently. For example, unit tests may be run each time new software is checked in, while the longer functional tests are run only every few days.
25
Testing and Configuration Management: What is the One goal of the automated tests?
* Is to confirm that the build is functioning and installable.
* If any automated test fails, the team should fix the underlying defect in time for the next code check-in.
* This approach helps reduce expensive and inefficient cycles of “build-install-fail-rebuild-reinstall” that can occur in many traditional projects, since changes that break the build or cause software to fail to install are detected quickly.
26
Testing and Configuration Management: Over-reliance on automated unit testing alone to manage these risks can be a problem, why?
* As unit testing often has limited defect detection effectiveness.
* Automated tests at the integration and system levels are also required.
27
Organizational Options for Independent Testing
* Independent testers are often more effective at finding defects.
28
Organizational Options for Independent Testing: One or more testers may be embedded within the team, performing many of the testing tasks. However, given those testers’ position within the team, what is the risk that may occur?
There is a risk of loss of independence and objective evaluation.
29
Organizational Options for Independent Testing: However, time pressures, lack of understanding of the new features in the product, and relationship issues with business stakeholders and developers often lead to problems with which approach?
Other Agile teams retain fully independent, separate test teams, and assign testers on-demand during the final days of each sprint. This can preserve independence, and these testers can provide an objective, unbiased evaluation of the software.
30
Organizational Options for Independent Testing: What is the third option to have regarding independent testing?
* Option is to have an independent, separate test team where testers are assigned to Agile teams on a long-term basis, at the beginning of the project, allowing them to maintain their independence while gaining a good understanding of the product and strong relationships with other team members
* In addition, the independent test team can have specialized testers outside of the Agile teams to work on long-term and/or iteration-independent activities, such as developing automated test tools, carrying out non-functional testing, creating and supporting test environments and data, and carrying out test levels that might not fit well within a sprint (e.g., system integration testing).
31
Communicating Test Status, Progress, and Product Quality: How do agile teams progress?
* Agile teams progress by having working software at the end of each iteration.
32
Communicating Test Status, Progress, and Product Quality: Testers in Agile teams utilize various methods to record test progress and status, which are?
* Test automation results
* Progression of test tasks
* Stories on the Agile task board
* Burndown charts showing the team’s progress
33
Communicating Test Status, Progress, and Product Quality: Testers in Agile teams utilize various methods to communicate test progress and status, which are?
* wiki dashboards
* Dashboard-style emails
* Verbally during stand-up meetings
34
Communicating Test Status, Progress, and Product Quality: Testers in Agile teams utilize various methods to communicate test progress and status, what are benefits of communicating in such a manner?
* Communicating test status in such an automated manner also frees testers’ time to focus on designing and executing more test cases.
35
Communicating Test Status, Progress, and Product Quality: What is a Burndown chart?
* A burndown chart represents the amount of work left to be done against time allocated to the release or iteration.
* Teams may use burndown charts to track progress across the entire release and within each iteration
36
Communicating Test Status, Progress, and Product Quality: When do teams use Agile task boards?
* To provide an instant, detailed visual representation of the whole team’s current status, including the status of testing.
37
Communicating Test Status, Progress, and Product Quality: What are story cards?
* Development tasks, test tasks, and other tasks created during iteration planning are captured on the task board, often using color-coordinated cards to determine the task type.
* During the iteration, progress is managed via the movement of these tasks across the task board into columns such as to do, work in progress, verify, and done.
38
Communicating Test Status, Progress, and Product Quality: The daily stand-up meeting includes all members of the Agile team including testers. At this meeting, they communicate their current status. The agenda for each member is?
* What have you completed since the last meeting?
* What do you plan to complete by the next meeting?
* What is getting in your way?
39
Communicating Test Status, Progress, and Product Quality: To improve the overall product quality, many Agile teams perform customer satisfaction surveys to receive feedback on whether the product meets customer expectations. Teams may use other metrics similar to those captured in traditional development methodologies, such as?
* test pass/fail rates
* defect discovery rates
* confirmation and regression test results
* defect density
* defects found and fixed
* requirements coverage
* risk coverage
* code coverage
* code churn
40
Managing Regression Risk with Evolving Manual and Automated Test Cases: The risk of introducing regression in Agile development is high due to?
* Due to extensive code churn (lines of code added, modified, or deleted from one version to another).
41
Managing Regression Risk with Evolving Manual and Automated Test Cases: Why do teams need to automate as many tests as possible from previous and current iterations?
* This allows automated regression tests to reduce regression risk with less effort than manual regression testing would require.
* This reduced regression test effort frees the testers to more thoroughly test new features and functions in the current iteration.
42
Managing Regression Risk with Evolving Manual and Automated Test Cases: Good practices for test design and implementation need to be adopted early and applied consistently. The shorter timeframes for testing and the constant change in each iteration will?
* Increase the impact of poor test design and implementation practices.
43
Managing Regression Risk with Evolving Manual and Automated Test Cases: How can Agile teams review the functionality tested and the test results for any given build at any given point in time?
* By checking the automated tests and their corresponding test results into the configuration management system, aligned with the versioning of the product builds.
44
Managing Regression Risk with Evolving Manual and Automated Test Cases: To ensure the code changes do not break the software build, team must?
* Automated unit tests are run before source code is checked into the mainline of the configuration management system.
45
Managing Regression Risk with Evolving Manual and Automated Test Cases: To reduce build breaks, which can slow down the progress of the whole team, code should not be checked in unless?
* All automated unit tests pass.
* Automated unit test results provide immediate feedback on code and build quality, but not on product quality.
46
Managing Regression Risk with Evolving Manual and Automated Test Cases: In addition to test automation, the following testing tasks may also be automated:
* Test data generation
* Loading test data into systems
* Deployment of builds into the test environments
* Restoration of a test environment (e.g., the database or website data files) to a baseline
* Comparison of data outputs
Automation of these tasks reduces the overhead and allows the team to spend time developing and testing new features.
47
Agile Tester Skills: What are the skills?
* test automation
* test-driven development
* acceptance test-driven development
* white-box
* black-box
* experience-based testing.
48
Agile Tester Skills: Testers in an Agile team should have good interpersonal skills. Testers in Agile teams should:
* Be positive and solution-oriented with team members and stakeholders
* Display critical, quality-oriented, skeptical thinking about the product
* Actively acquire information from stakeholders (rather than relying entirely on written specifications)
* Accurately evaluate and report test results, test progress, and product quality
* Work effectively to define testable user stories, especially acceptance criteria, with customer representatives and stakeholders
* Collaborate within the team, working in pairs with programmers and other team members
* Respond to change quickly, including changing, adding, or improving test cases
* Plan and organize their own work
Continuous skills growth, including interpersonal skills growth, is essential for all testers, including those on Agile teams.
49
The Role of a Tester in an Agile Team: The role of a tester in an Agile team includes activities that generate and provide feedback not only on test status, test progress, and product quality, but also on process quality. Activities include:
* Understanding, implementing, and updating the test strategy
* Measuring and reporting test coverage across all applicable coverage dimensions
* Ensuring proper use of testing tools
* Configuring, using, and managing test environments and test data
* Reporting defects and working with the team to resolve them
*Coaching other team members in relevant aspects of testing
* Ensuring the appropriate testing tasks are scheduled during release and iteration planning
* Actively collaborating with developers and business stakeholders to clarify requirements, especially in terms of testability, consistency, and completeness
* Participating proactively in team retrospectives, suggesting and implementing improvements
Within an Agile team, each team member is responsible for product quality and plays a role in performing test-related tasks.
50
The Role of a Tester in an Agile Team: What are Agile test-related organizational risks?
* Testers work so closely to developers that they lose the appropriate tester mindset
* Testers become tolerant of or silent about inefficient, ineffective, or low-quality practices within the team
* Testers cannot keep pace with the incoming changes in time-constrained iterations
