Ch. 6: Testing, Operation, and Evolution

Question 1

Software quality is the degree to which the system satisfies the stated and implied needs of its various stakeholders, and thus provides …

Answer 1

… value.

Question 2

What is the difference between software quality and software quality management?

Answer 2

Software quality is concerned with whether and how the system provides value by satisfying stakeholders’ needs. Software quality in the broadest sense denotes a property of software. The term quality management (QM) encompasses all organized measures and activities that serve to improve the quality of products, processes or services of any kind.

Question 3

What are the main activities for quality management?

Answer 3

• Quality planning. This is the creation and documentation of the quality requirements together with the client.
• Quality control. This is monitoring, management, and control of activities for quality testing in the software development process.
• Quality assurance. This comes in the form of activities ensuring that defined quality requirements for products, processes, and services are met.
• Quality improvement. This is the evaluation of product and process data to improve the quality level.

Question 4

What is the difference between constructive and analytical QM?

Answer 4

• In constructive quality management, all quality properties for products or processes are defined a priori (i.e., before creation) in order to avoid errors during software development and to guarantee or increase the quality of the artefacts created. The measures for constructive QM are technical, organizational and socio-psychological.
• In analytical quality management, measures are carried out ex post (i.e., after creation) to test and evaluate the current quality level of the test objects in order to systematically track down errors and determine their extent. It can be divided in static analysis (as part of a review or audit, the test item is analyzed, appraised, and examined. The information obtained is collated, if necessary, condensed into metrics or key figures, and evaluated) or dynamic analysis (the system under test (SUT) is executed with specific input values, and the result of the execution is evaluated.)

Question 5

What is a test item?

Answer 5

All artifacts created in software engineering can be tested for compliance with quality requirements. This means that business requirements, technical specifications, architecture descriptions, and test cases can be tested in addition to the generated program code.

Question 6

What are the typical test levels/stages in software development?

Answer 6

• Unit test
• Integration test
• System test
• Acceptance test

Question 7

What is a unit test and what are some examples of it?

Answer 7

To enable the implementation of a software system by several software developers, the system is broken down into its logical components (units, modules, classes, and building blocks). Each component is implemented separately and incorporated into the system after completion (integrated). During, or after, the completion of a software component, it can be checked for compliance with specified requirements. The isolated testing of individual software components is called a unit test. Some examples of unit tests are as follows: premium calculation requires calculation of insurance premiums depending on the selected services, shopping cart requires calculation of the total of all items, address component requires the correct address for correspondence to be generated from the customer data

Question 8

What is a (component) integration test? What is its relationship with drivers, stubs, SUT and dummies?

Answer 8

During, or after, the integration, the interaction of groups of components is tested in integration tests. The integration tests check whether the components work together as described in the specification.
Missing components are simulated by drivers and dummies so the test can be started. Software fragments that simulate the calling of other components are called drivers. Stubs, on the other hand, simulate components that are called by other components.

Question 9

What is a system test?

Answer 9

After completion of the development work and the integration of all software components into a finished system, the system as a whole is tested with the system test. The aim of the system test is to check whether the system meets the specified requirements. In addition to the business requirements, the specified quality properties are also tested.
A major challenge in the system test is reproducing the customer’s productive environment as true to the original as possible. This includes the hardware and software environment in which the system is operated, the software systems with technical interfaces used by the system, realistic utilization, realistic user behavior, and real data records.

Question 10

What is an acceptance test?

Answer 10

The last test level is the acceptance test, in which the finished system is handed over to the client, installed, and tested under the actual operating conditions. The acceptance test checks whether the system has the contractually agreed performance characteristics from the client’s point of view. The most important difference compared to the system test is that it is carried out by a different organization than the one that developed the system. A successful acceptance test is often the contractually agreed-upon requirement for invoicing.

Question 11

What is the difference between traditional and Agile testing?

Answer 11

Traditional testing approaches perform the test levels one by one, based on a comprehensive project plan. Unfortunately, this is risky since it prevents early knowledge. In Agile approaches, knowledge is gained early in the software process through short feedback cycles. For this purpose, in contrast to traditional plan-driven approaches, only a small subset of the requirements are analyzed, specified, implemented, and tested by the development team from the start of the project.

Question 12

What are the typical characteristics of Agile testing?

Answer 12

• Test as early as possible and as often as necessary.
• As many steps as possible are to be automated (except for exploratory tests by experts)
• Concentrate on testing the basic functionality and select other test types according to actual needs
• Working in an interdisciplinary manner means, depending on the type of test, also involving the developers.

Question 13

What is a release plan during operation?

Answer 13

A release plan defines specific dates on which change requests and adjustments to software systems are implemented and put into operation. With the help of release planning, business departments, application development, and operations can internally plan and carry out the tasks required to import a new software version. This includes, in particular, quality assurance activities.
For large systems, between one and four release dates per year are common, and the specific dates are often based on industry-standard cycles or legal requirements.

Question 14

What are the key issues with few major releases?

Answer 14

First, to accommodate all desired adjustments in a few releases, changes must be carried out at many points at the same time. However, large adjustments always involve a high risk of errors. Second, business departments are dissatisfied with the IT support, and important change requests are usually only implemented after weeks or months. But a slow IT response time can result in competitive disadvantages. Third, a long time to correct newly discovered security gaps or errors in the application leads to possible attack scenarios.

Question 15

What is continuous integration during operation, delivery and DevOps?

Answer 15

Continuous integration is daily and weekly releases of software systems that can be created and automatically tested through automation and tool support. If all tests are completed positively, the adapted application can be deployed directly by setting up an automatic delivery.
The prerequisites for continuous integration and continuous delivery, however, are intensive cooperation between application development and operation. Such cooperative approaches are also called development operations (DevOps).

Question 16

What are the key tasks in IT operations?

Answer 16

Application operation ensures that the system is available to users and is protected against failure and threat scenarios. The application operation itself has no explicit constructive or quality assurance function in the development of a software system.

Question 17

What are the dependencies between IT operations and IT application development?

Answer 17

When the software is handed over, it must be ensured that it can be executed according to the company’s specifications and guidelines and that the software can be taken over into operation by the department responsible for IT operations after development and testing. Furthermore, the activities of application development and operation are internally coordinated with the help of release planning.

Question 18

Please explain the difference between the development of a new system from scratch and system evolution.

Answer 18

Basically, the evolution activities include the same tasks and activities as the initial creation of a software system. However, the further development activities already relate to a productively used system to which changes are to be made.

Question 19

Please name three challenges of system evolution.

Answer 19

• The architecture degenerates with every code change.
• The existing architecture does not support the new requirements.
• The documentation is incomplete and not up-to-date.
• Lack of knowledge about the architecture and programming language of the system.
• Existing functions that should remain unchanged must not be impaired during evolution.

Question 20

Why should documentation also be considered in the system evolution?

Answer 20

Only if the software architecture is maintained and held up-to-date continuously during the many smaller adjustments, the ability to evolve can be guaranteed over time. Experience shows that identifying the points at which adjustments are made easily takes up around 40 percent of the time spent on evolution activities.