Chapter 2

Question 1

Validation vs Verification:

What is validation? (Hint: Did we build the right thing?)

Answer 1

Checking the development results against the original requirements is called validation.

When performing validation, testers evaluate whether the product can solve a specific task and see if it is suitable for its intended purpose.

Question 2

Validation vs Verification:

What is Verification? (Hint: Did we build the thing right?)

Answer 2

Verification examines whether the specifications have been implemented correctly, regardless of the indented purpose of the product.

It proves the correctness and completeness of the results from a particular phase.

Question 3

What are the flaws of the Iterative model in regards to testing?

Answer 3

1. Testing of final increment can be rushed results in buggy deliverables.
2. Overlap in increments meaning developers work on another increment -> hard to fix bugs found from earlier increments.
3. Code changes or new code increases regression risk.

Question 4

What is regression risk?

(Regression: A return to a former or less developed state)

Answer 4

The possibility of unindented consequences when changes are made to the software. U change one part, other parts start to have problems.

Question 5

Risks exists in _ main areas, What are they? And briefly describe what they are

Answer 5

Answer 1: 4

Answer 2:
1. Coupling: Strong consequence of failure between the component and the system. (EX: Database fails, app fails)

2. Irreplaceability: Few similar components available (COTS)
3. Essential: Key features in a system is unavailable if the component malfunctions (EX: Database)
4. Vendor quality problems: Increased risk of bad components.

Question 6

How do you mitigate the risks? (“Exists in _ areas..”) (Hint: 4 options)

Answer 6

1. Integrate, track and manage vendor testing in distributed test effort.
2. Trust the vendor.
3. Fix vendor’s quality problems yourself, with permission.
4. Disregard and replace their testing, do your own testing.

Question 7

CMMI Model: State the 5 levels

(I, Miss, Dick, Quite, Often)

Answer 7

Initial, Managed, Defined, Quantitatively managed, optimising

Question 8

What are characteristics of good testing?

Answer 8

1. Have activities for testing for each development activity.
2. Test levels have focused objectives, with coordination to avoid gaps.
3. Test analysis, design should start early to avoid bugs
4. Testers with adequate skill should attend to bring their unique perspectives and skills.

Question 9

Component (Unit) Test:

What is the objective of Unit Tests?

Answer 9

1. Find bugs
2. Build confidence
3. Reduce risk in individual pieces of the system. (Reduce risk of specific components in the system)

Question 10

Component (Unit) Test:

What is the basis for Unit tests and the test types associated with it?

Answer 10

1. It is based on code, database, requirements/design and quality risks.
2. Test types include behavioural (black-box) and structural (white-box)

Question 11

Component (Unit) Test:

1. What are the Items under test / test objects?
2. Who is responsible for unit testing?

Answer 11

1. Components, programs
2. Usually programmers

Question 12

What are stubs and drivers?

Just reveal answer if u dont know:

Answer 12

Stubs

Definition: Stubs are simplified versions of modules that simulate the behavior of actual modules. They are used to replace missing or incomplete modules during testing.

Purpose: Stubs are typically used in top-down integration testing, where testing starts from the top-level module and progresses downwards. They provide a controlled environment to test the behaviour of a module without relying on the complete implementation of its dependent modules.

Drivers

Definition: Drivers are test programs that call the module being tested and provide necessary input data. They are used to simulate the behavior of higher-level modules.

Purpose: Drivers are typically used in bottom-up integration testing, where testing starts from the lowest-level modules and progresses upwards. They provide a controlled environment to test the behavior of a module without relying on the complete implementation of its calling modules.

Question 13

Unit Test typically:

1. Involves access to ___?
2. Is running in a development environment, by the ____ who wrote the code?
3. Requires ____, _____, and/or harness?

Answer 13

1. Code
2. Developer
3. stubs, drivers

Question 14

Component (Unit) Test:

What is test-first/ test-driven development approach?

Answer 14

1. Develop a set of unit tests.
2. Build and integrate code to pass the tests. (Run the tests and debug until the code passes the tests.)
3. Add more unit tests for additional functions.

Question 15

Integration Tests

1. What is the objective of Integration tests?

Answer 15

1. Find bugs
2. Build confidence
3. Reduce risks in relationships between components that come together.

Question 16

Integration Tests

What is the Integration Tests based on?

Answer 16

1. System Design
2. Architecture
3. Database schemas
4. Data flows
5. Workflows
6. Use-cases
7. Quality risks

Question 17

Integration Tests

1. What are the test types associated with Integration tests?

Answer 17

Behavioural tests: Functionality, resource use, performance

Structural tests: call-flows and data-flows

Question 18

Integration Tests

What are the test objects of Integration tests?

Answer 18

Collection of units/ components, database implementation, interfaces

Question 19

Integration Tests

Who is the one responsible for Integration tests?

Answer 19

Ideally both testers & programmers but actually no one is responsible.

Question 20

Integration Tests

What are the integration techniques for Integration tests AND give some details about them.

Answer 20

1. Big-bang integration
2. Bottom up
3. Top down
4. Functional or transactional
5. Backbone Integration

Question 21

Explain what you know about the “Big Bang Integration” approach to Integration Tests

Answer 21

You take all the tested modules (Example: components, units, classes, and functions) and put them all together and test.

It is quick but you do not know exactly where the bug is and why wait until all code is written to start integration when you should always start early testing (Principle 3)

Question 22

Explain what you know about the “Bottom Up” approach to Integration Tests

Answer 22

You start with the bottom layer modules, and you work your way up using appropriate drivers and tests. You repeat the process until you replace drivers with the actual modules.

It has good bug isolation, but what if nasty problems are at the top?

Question 23

Explain what you know about the “Top down” approach to Integration Tests

Answer 23

You start with the top layer modules and work your way down and you use stubs instead of drivers.

This is good bug isolation but what if the nasty problems are at the bottom?

Question 24

Explain what you know about the “Functional or Transactional” approach to Integration Tests

Answer 24

You identify, develop and integrate components required to implement a single function. Then you repeat the process, by replacing stubs and drivers with the actual components to implement the next functions.

It has good bug isolation and can find integrations bugs in risk order. But problem is what if you select a wrong function to begin with?

Question 25

Explain what you know about the "Backbone Integration" approach to Integration Tests

Answer 25

1. You use risk, architecture, and features as a guideline for integration. 2. You identify a set of critical modules that work together to support the KEY features and those that are most likely to have significant bugs. 3. Then, you build a backbone that contain those modules and test them with the help of stubs and drivers. 4. This approach contains good bug isolation, logical ordering of features to test, and most likely to find integration bugs in the order of importance.

Question 26

For the backbone integration, explain steps Backbone 0 (BB0), Backbone 1 (BB1), and Backbone N (BBn)

Answer 26

1. Backbone 0: You start with a basic backbone, with the APIs and basic networking architecture. You also test the basic functionality, error handling & recovery, reliability of it and the performance. 2. Backbone 1: Then you add the modules that implement the most critical core operations and services. You also test basic functionality of the modules, error handling & recovery, reliability and performance. 3. Backbone N: As the last step, test the end-to-end through GUIs on the host systems.

Question 27

Integration Tests: It is said that there can be more than one level of integration testing on a project. What are they?

Answer 27

1. Component integration testing: Testing interactions between the units and components following unit/component test 2. System integration testing: Testing the interaction between entire systems following system test

Question 28

Why is system integration testing complex?

Answer 28

Systems developed by different organisations / different periods, making system testing complex. (Maybe traceability can solve this) Multiple organisations controlling the systems; interfaces making any changes to it dangerous Business processes can span multiple systems, and changes might even break the systems Hardware / system compatibility issues can arise

Question 29

System Test What is the objective of System Test?

Answer 29

1. Find bugs 2. Build confidence 3. Reduce risk in the overall & particular behaviours, functions and responses of the system under test as a whole

Question 30

System Test What is System Test based on?

Answer 30

1. System requirements 2. High-level design 3. Use cases 4. User and tester experience with similar systems 5. Checklist of quality characteristics, customer environments

Question 31

System Test What are the test types for System Test?

Answer 31

1. Behavioural tests: Functionality, security, performance, reliability, usability, and portability (Black-box testing) 2. Structural tests: Statements, branch, loop coverage (White-box testing)

Question 32

System Test What are the test objects under System Test?

Answer 32

1. The whole system (In a realistic test environment) 2. User and operator manuals 3. Configuration data

Question 33

System Test What are the harnesses & tools for System Test?

Answer 33

API, CLI, or GUI (can be freeware or commercial)

Question 34

System Test Who is usually responsible for System Tests?

Answer 34

Typically independent testers, users (might be involved)

Question 35

Acceptance Test What are the objectives under Acceptance Tests?

Answer 35

Demonstrate that the product is ready for deployment / release HOWEVER, this does not include finding bugs

Question 36

Acceptance Test What are the acceptance tests based on?

Answer 36

1. User requirements 2. System requirements 3. Use cases 4. Business processes 5. Contracts 6. User & Tester experience with similar systems

Question 37

Acceptance Tests What are the variations of Acceptance Tests?

Answer 37

1. User acceptance testing 2. Operational testing 3. Contract testing 4. Alpha, Beta, and pilot testing

Question 38

Acceptance Test What do you know about User Acceptance Testing? (Hint: Include business users)

Answer 38

The business users verify the fitness of the functional purposes and key business application

Question 39

Acceptance Tests What do you understand from Operational Testing?

Answer 39

It is acceptance by system administrators ( Example: disaster recovery, user management maintenance, data load / migration, security

Question 40

Acceptance Test What do you understand about Contract Testing / Regulation Testing?

Answer 40

1. Verification of conformance 2. To contractually-agreed or legally mandated requirements, regulations or standards.

Question 41

Acceptance Test What do you understand from Alpha, Beta and Pilot Testing?

Answer 41

Potential or existing customers do testing and build confidence beta testing and field testing are performed in the actual environment(s)

Question 42

What is meant by "Pervasive Testing"?

Answer 42

Everyone participates in testing, to filter out the maximum percentage of bugs.

Question 43

Just a note: For functionality test Look for reasonable or required action that is either not provided, inaccessible, or seriously impaired For example: 1. Not provided: No "+" function on a calculator ( The essential function is missing / not provided) 2. Function is implemented, but the "+" key doesn't work ( Function is inaccessible) 3. Can only add integers, not real numbers ( Seriously impaired)

Answer 43

-

Question 44

Specialised functional test: 1. For Interoperability test, what does it test for?

Answer 44

Interoperability with programs, OS, databases. It Includes: interfaces between items and data interchange.

Question 45

Specialised Functional Test: 2. Performance and Reliability Test (Non-functional test) What are the test conditions for performance and reliability tests? (3 test conditions)

Answer 45

1. Stress: Create extreme conditions to cause failure through a combination of error forcing, capacity, and volume tests. 2. Capacity: Focus on functionality, performance or reliability issues due to resource depletion ( Ex: Fill hard drive to 80+%) 3. Volume: Focus on functionality, performance, or reliability issues due to rate of data flows ( Ex: Run 80+% of rated transactions per minute) Generally, the degradation of system starts to occur around > 80% of resources usage (So that's why they test with 80+% of resources)

Question 46

Regression and Confirmation Explain what does regression testing do:

Answer 46

Regression testing checks the effect of changes on the unchanged parts. Sometimes when you make a change in a module, other modules might also be affected, which may be an unintended consequence.

Question 47

Regression and Confirmation What does Confirmation testing do?

Answer 47

1. You ensure that the changes made to the system is present and working as intended. 2. Bug fixes to the system actually solved the initial issue.

Question 48

Regression and Confirmation How can we help with Regression and Confirmation testing?

Answer 48

Repeatability of tests.

Question 49

Maintenance Testing What are the 3 reasons/triggers for maintenance and maintenance testing?

Answer 49

1. You made changes/ modifications: Could be enhancements, bug fixes, operational environment, changes, and patches 2. Retirement of old components/systems: End-of-life of a subsystem or entire system mandate a replacement. Requiring maintenance. 3. Migration to new supported environment. ( Could be hardware and software)

Question 50

What are the different regression types? (Hint: 3 types)

Answer 50

1. Local- The fix creates another/ a new bug in the same functional area 2. Exposed- Fix reveals existing bug in the same functional area. (For instance, fixing a login authentication issue might expose a problem with session management that was already present but unnoticed.) 3. Remote- A fix in one area, breaks something in another area.

Question 51

What are the Regression strategies? (Hint: 5 total / 2 main 3 secondary)

Answer 51

1. Regression strategy 1: Repeat all the tests - If the tests addresses all the important quality risks, then repeating all the tests should find the most important regressions. 2. Regression Strategy 2: Repeat some tests - Often, used when full automation is impossible 3. Release more slowly - Release every 6 months, instead of every month 4. Combine patches with slower release to allow for flexibility while keeping regression risk low. 5. Involve customer or user in the testing phase. (Known as beta test)

Question 52

Note: For: Focus on each testing phase 1. Unit testing: Focus is the code within the unit itself, 2. Integration testing: It is the interfacing between units 3. System testing: It is the end-to-end functionality 4. Acceptance testing: It is the user perspective