6. Solution Evaluation

Question 1

3. Recommended Mindset for Evaluation

Answer 1

1. Evaluate early and often
2. Treat requirements analysis, traceability, testing, and evaluation as complimentary activities.
3. Evaluate with context usage and value in mind.
4. Confirm expected value for software solutions.

Question 2

2. Purpose of solution evaluation

Answer 2

Evaluation determines how well a solution meets the business needs expresses by stakeholders, including delivering value to the customer. SE activities provide the ability to asses whether or not the solution has achieved the desired business result.

Question 3

4. Plan for evaluation of the solution.

Answer 3

It takes time to identify and confirm evaluation criteria, implement anything necessary for taking measurements, take the measurements, and report and analyze the measurements.

Question 4

5. Determine what to evaluate

Answer 4

1. Consider business goals and objectives
2. Consider KPIs
3. Consider additional evaluation metrics and evaluation acceptance criteria.
   - project metrics as input
   - customer metrics
   - sales and marketing metrics
   - operational metrics and assessments
   - functionality
4. Confirm how the organization can continue with evaluation.

Question 5

6. When and how to evaluate solution results

Answer 5

1. Surveys and focus groups
2. Results from exploratory testing and user acceptance testing.
3. Results from DITL.
4. Results from integration testing
5. Expected vs. actual results for functionality
6. Expected vs. actual results for non-functional requirements.
7. Outcome measurement and financial calculation of benefit.

Question 6

7. Evaluate acceptance criteria and address defects.

Answer 6

1. Compare expected vs. actual results.
2. Examine tolerance ranges and exact numbers.
3. Log and address defects.

Question 7

8. Facilitate the Go/No-Go decision

Answer 7

Stakeholders in the RACI matrix who were identified as having a role to approve or Sign off on the solution are generally the individuals who make go/no-go decision.

Question 8

9. Obtain Signoff of the solution

Answer 8

RACI developed in BA Planning designates who is accountable for signoff on the solution.
PM and BA, working with auditing and stakeholders, should work together to determine the signoff practices required by the organization and the project.

Question 9

10. Evaluate the long-term performance of the solution

Answer 9

1. Determine metrics.
2. Obtain metrics/measure performance.
3. Analyze results.
4. Address limitations of the solution and organization.
5. Recommend approach to improve solution performance.

Question 10

11. Solution replacement/phase out

Answer 10

Four strategies for solution replacement/phase out:

• massive one time cutover
• segmented cutover
• time-boxes coexistence of the replacement and what is being phased out, with a final cutover on s specific future date.
• permanent coexistence of the old and replacement solutions (with all new business using the replacement solution)