Many critics of risk matrices are conflating a flawed method for a flawed tool. They might have used risk matrices poorly in the past and assume (you know what they say about assumptions) that others use them the same way.
Few, if any, risk managers have not used a risk matrix at some point in their careers.
Criticizing risk matrices, however, is like shooting ducks in a barrel. It’s wrong, for starters. Looking at risk matrices in isolation is as bad as blindly trusting quantitative risk analysis or probability distributions without the context of their inputs and outputs.
It's also my preference not to criticize something without checking the details. Despite the limitations, there are plenty of use cases for risk matrices. And without offering at least some attempted solution. Some critics of risk matrices are happy to appear erudite and learned but, in the end, have nothing to offer in its place.
A map is not the territory. A Monte Carlo analysis or a QRA study is not a risk assessment. Just like a laboratory trial to determine the LD50 of a compound is not a risk assessment. Such things are just inputs to a risk assessment.
Quantitative risk assessment (QRA) is an excellent method for evaluating the probability and consequences of potential risks or hazards to prioritize and mitigate them. But it does have some limitations and potential flaws that should be considered.
Taking them in isolation as individual tools, I could just as easily drive a bus through some of the holes in the quantitative risk analysis methods. How often have inexperienced or gullible executives fallen for the QRA mumbo-jumbo by unscrupulous quantitative risk consultants?
Precision should not be mistaken for accuracy.
If you live long enough, you will come across risk assessments purporting to assess risks to 3 decimal places. I'm not sure what the benefit of calculating risks to x.xxx but please let me know if you do.
A 0.001 % chance of nuclear war does not preclude it from starting tomorrow. Tools like Bow-Tie, scenario modeling, and all-hazards approaches can be far better than either QRA or risk matrices in many circumstances.
And in the end, with risks such as climate change or nuclear war, the treatments you choose and implement will be the most valuable part of the exercise. If you can use a risk matrix as a thinking tool or a presentation tool, then why not use it?
If you torture numbers enough, they will confess to anything.
One potential flaw of QRA is that it may not adequately account for uncertainty or variability in the data or assumptions used to calculate the risk. This can lead to a false sense of precision and accuracy, as the results may not reflect the full range of potential outcomes.
Another potential issue with QRA is that small changes in the input data or assumptions can lead to significant variations in the output risk estimates of NPVs, QRAs, ROIs, and Monte Carlo simulations, to mention just a few. This can make it difficult to assess the risk accurately and may require frequent reevaluations to ensure that the risk estimates remain accurate.
QRA relies on mathematical models that are typically time-consuming work for specialists. You often have to question whether the time and expense are worth it for the risks you are managing. And the benefits of the objectives you may or may not be chasing.
The output from QRA analyses can be valuable input for risk assessment, but it is important to remember that they are not complete risk assessments. Other factors, such as real-world usage patterns, comorbidities, and other contextual factors, can also impact the risk and should be considered part of a comprehensive risk assessment.
Risk matrices and QRA both have their pros and cons. Neither is perfect, but a good risk manager has many tools.
