Vocab
Capacity Testing
Determines:
* How many users
* How many transactions
* Number of data volumes from transactions
are needed to meet performace objectives
Vocab
Concurrency Testing
Focuses on
the impact of situations were specific actions occur simultaneously
Concurrency issues are notoriously difficult to find and reproduce
What are the three characteristics of performance testing?
- Time behavior - ability of a component to respond within a specified time and under specified conditions. (End-to-end timing, CPU cycles)
- Resource Utilization - Avilability of resources may need to be investigated
- Capacity - Required capacity limits of system behavior may need to be tested to evaluate system architecture
Vocab
Endurance Testing
Focuses on the stability over time.
Verifies no resource capacity problems, such as:
* Memory leaks
* Database connections
* Thread pools
Vocab
Load Generation
Simulates a load for testing:
* Time-based tests
* Resource usage
* Capacity
Steps to generate:
1. Define load scenarios
2. Choose your load generation tool, such as JMeter, Gatling, etc
3. Configure the load (see below)
4. Monitor and log analytics
Configure:
* Ramp-up time: The rate at which VUsers are added to the test.
* User think time: The delay between requests from each VUser.
* Request rate: The frequency of requests per second.
* Data volume: The amount of data transferred during each request.
Vocab
Load Testing
Can the system handle increasing levels of anticipated loads from transaction requests from controlled numbers of concurrent users or processes
Vocab
Spike Testing
Ability of a system to respond to sudden bursts of peak loads and return to a steady state.
Vocab
Performance Testing
Umbrella term
Any testing that focuses on performance (AKA responsiveness) of the system or component under different volumes of load
Responsiveness might be swapped out for performance on the exam
Vocab
Scalability Testing
Ability to meet future requirements beyond which are currently required.
Determine the ability of the system to grow, without violating currently specified performance requirements or failing.
Observability implications - - Threshold values can then be set and monitored in production to provide a warning about when a problem is about to arise.
Vocab
Stress Testing
Ability of a system or component to handle peak loads that are at or beyond the limits of anticipated workloads.
ALSO
Ability of a system to handle reduced resources, such as:
* compute capacity
* bandwidth
* memory
Is performance testing functional or non-functional?
Non-functional - the testing does not address the functionality of the application directly
What are the 5 performance testing principles?
- Aligned with the expectations of different stakeholders - users, system designers, and operations
- Reproduceable, with statistically identicle results (within thresholds)
- Yield results that are understandable and compared to expectations
- Conductable on systems that are complete, partial, or representative of production
- Affordable and executable within timeframe set by budget
What are the two testing types used in perf testing?
- Static
- Dynamic
What are some of the static testing activities?
- Requirements reviews
- Database reviews (schemas, stored procedures, ERDs, etc)
- System/architecture reviews
- Complex algorithms
What are some of the dynamic testing activities?
- Unit/component testing - profiling for bottlenecks and resource utilization
- Component Integration -examining key use cases and workflows
- System Testing - end-to-end behaviors
- System integration - data flows and workflows across key interfaces
- Acceptance Testing - build user, customer, and operator confidence in performance of the system and fine tune under real world conditions
True or False
The use of realistic environments, data, and loads are critical for accurate results, for system testing and system integration testing.
True
How soon should performance testing begin in an agile environment?
As soon as possible, starting with the requirements.
Performance requirements will be found in the “acceptance criteria” of a user story.
What if the necessary hardware isn’t available?
Use simulators for the time being
What are the 4 ways to generate loads?
- User Interface
- Crowds (testers)
- API
- Captured communication protocols (tool-based approach)
Describe 3 requirements for a performance test load
- Represents many user inputs
- May require dedicated hardware and tools for load generation
- Dependent on the absence of any functional defects in the SUT
General Knowledge
What is SUT
System Under Test
Describe Load Generation via User Interface (UI)
- Small number of users represented
- Requires same number of software clients
- Depends on UI stability
- Affected by frequent UI/end-to-end (E2E( changes
- Can be done with functional testing (ie E2E) tools, but will not be scalable
Describe Load Generation with Crowds
- Requires a high availability of testers
- Good for simulating various devices and geo-locations
- Difficult to reproduce results and organize the process
Describe Load Generation via API
API - Application Programming Interface
- Dedicated scripts repeatedly make calls to API
- Avoids the UI
- More users can be simulated
