Verification & Validation

Question 1

What is the difference between verification and validation?

Answer 1

Validation: have we built the right system?
- check all requirements are satisfied;find missing funtionality

Verification; have we built the system right?
- check system behave correctly; check for consistency and completeness

Question 2

3 methods of validation?

Answer 2

• Check every use case handled by system
• simulate system using CRC cards
• check use case scenarios using UML sequence diagram

Question 3

3 goals of verification?

Answer 3

• prove models are correct before coding
• consistent
• complete

Question 4

Describe breifly 4 different kinds of testing

Answer 4

• Unit testing; check each class works infdivually
• Integration testing; checks whole system works after assembly
• Regression testing; checks a system still works after modification
• Acceptance testing; is system intuitive for end-users?

Question 5

What is the oracle problem?

Answer 5

• code based testing relies on seeing if anything unusual happens when executing code paths, but what is correct?

Question 6

What is the oracle test?

Answer 6

-assumes code must conform to some correct specification

Question 7

Describe 3 different testing strategies

Answer 7

Manual; user written tests
Random; tools randomly generate test sequences
Model-based; test sets generate automatically

Question 8

What is testing coverage?

Answer 8

How much of the software is validated by the test set

Question 9

3 different levels of software coverage that can be achieved by white-box testing?

Answer 9

• Full statement coverage; every program node
• Decision coverage; every program branch
• Condition coverage; every boolean sub-expression

Question 10

2 methods of black-box testing

Answer 10

• equivalence partition testing; test all distinct sets of arguments that trigger different otucomes
• state-based testing; exercise the complete state space (reach every state and fire every transition)

Question 11

How do we write positive and negative OCL tests using preconditions?

Answer 11

+ supply a test satisfying all preconditions

- supply a test violating each condition

Question 12

How do we write positive and negative OCL tests using postconditions?

Answer 12

supply tests whose argument shsould trigger each kind of outcome
- supply tests whose inputs dont satisfy any of the postconditions

Question 13

How do we write positive and negative OCL tests using invariants?

Answer 13

+ does invariant hold after construction?

- force object into an inconsistent state and evaluate

Question 14

WHat method do we use to test from a state machine?

Answer 14

Generate test sequences to reach every state
Exercise every method once in every state
Check we reach the desired next state

Question 15

What is meant by the state cover?

Answer 15

Set of shortest sequences that is guaranteed to reach every state (after reset)

Question 16

What is meant by the characertization set?

Answer 16

smallest set of shortest sequences that will uniquely distinguish being in each state