Risk-based testing

Risk-based testing (RBT) is a type of software testing that functions as an organizational principle used to prioritize the tests of features and functions in software, based on the risk of failure, the function of their importance and likelihood or impact of failure.^[1][2][3] In theory, there are an infinite number of possible tests. Risk-based testing uses risk (re-)assessments to steer all phases of the test process, i.e., test planning, test design, test implementation, test execution and test evaluation.^[4] This includes for instance, ranking of tests, and subtests, for functionality; test techniques such as boundary-value analysis, all-pairs testing and state transition tables aim to find the areas most likely to be defective.

Types of risk assessment

Light-weight risk assessment

Lightweight risk-based testing methods mainly concentrate on two important factors: likelihood and impact.^[5] Likelihood means how likely it is for a risk to happen, while impact measures how serious the consequences could be if the risk actually occurs. Instead of using complicated math, these techniques rely on simple judgments and scales.^[6] For instance, a team might rate the chance of risk as high, medium, or low and its impact as severe, moderate, or minor. These ratings help prioritize where testing efforts should be focused.^[7]

Heavy-weight risk assessment

Heavy-weighted risk-based testing is a method used to test software by focusing on the areas where problems are most likely to happen. The testing team looks for the most important parts of the software that might fail and concentrates on testing those parts more thoroughly.^{[citation needed]}

There are four main types of heavy-weight risk-based testing methods:^[7]

1. Cost of Exposure: This looks at how much money a problem in the software might cause. It figures this out by thinking about how likely a problem is and how much it might cost.
2. Failure Mode and Effect Analysis (FMEA): This technique finds out what parts of the software might fail, why they might fail, and what might happen if they do. It helps find the important areas that need attention.
3. Quality Functional Deployment (QFD): This method helps connect what the users need with what the software does. It looks at risks that might come from not understanding what the users really want.
4. Fault Tree Analysis (FTA): This technique is used to figure out why something went wrong by looking at different reasons in a step-by-step way..

Types of risk

Risk can be identified as the probability that an undetected software bug may have a negative impact on the user of a system.^[8]

The methods assess risks along a variety of dimensions:

Business or operational

• High use of a subsystem, function or feature
• Criticality of a subsystem, function or feature, including the cost of failure

Technical

• Geographic distribution of development team
• Complexity of a subsystem or function

External

• Sponsor or executive preference
• Regulatory requirements

E-business failure-mode related

• Static content defects
• Web page integration defects
• Functional behavior-related failure
• Service (Availability and Performance) related failure
• Usability and Accessibility-related failure
• Security vulnerability
• Large scale integration failure

^[9]

Some considerations about prioritizing risks is written by Venkat Ramakrishnan in a blog. ^[10]

References

1. Bach, J. The Challenge of Good Enough Software (1995)
2. Bach, J. and Kaner, C. Exploratory and Risk Based Testing (2004)
3. Mika Lehto (October 25, 2011). "The concept of risk-based testing and its advantages and disadvantages". Ictstandard.org. Retrieved 2012-03-01.
4. Felderer, Michael; Schieferdecker, Ina (2014). "A taxonomy of risk-based testing". International Journal on Software Tools for Technology Transfer. 16 (5): 559–568. arXiv:1912.11519. doi:10.1007/s10009-014-0332-3. S2CID 11598143.
5. Mahesh, Hari (2023-11-03). "Risk-based Testing: A Strategic Approach to QA". testRigor AI-Based Automated Testing Tool. Retrieved 2023-11-18.
6. Schmitz, Christopher; Pape, Sebastian (2020-03-01). "LiSRA: Lightweight Security Risk Assessment for decision support in information security". Computers & Security. 90: 101656. doi:10.1016/j.cose.2019.101656. ISSN 0167-4048. S2CID 208109813.
7. "What is Risk Based Testing: With Best Practices". www.lambdatest.com. Retrieved 2023-11-18.
8. Stephane Besson (2012-01-03). "Article info : A Strategy for Risk-Based Testing". Software Quality Engineering IT. Stickyminds.com. Retrieved 2012-03-01.
9. Gerrard, Paul and Thompson, Neil Risk-Based Testing E-Business (2002)
10. On Risk-Based Testing [1]