Risk registers: good, bad, odd and ugly use

This article classifies risk registers across two dimensions:

  1. The quality of the thinking that went into them: was this straight or crooked, sharp or muddled?
  2. The overall place within risk management: is the risk register the only tool, or is it part of a suite?

The best place to be, unsurprisingly, "straight and suite". But some other quadrants are heavily populated. This article focuses mainly on 2 above.

Being clear: my view

I'm in favour of risk registers in the same way that I'm in favour of surgical checklists, personal organisation tools, shopping lists and the periodic table.

But a personal organiser is only as good as its inputs. Knowing the periodic table won't make you a good scientist. It's similar for risk registers, but worse. It is very easy to convince ourselves that filling in a risk register is a big part of risk management, especially when we can find and filter risks so easily. It's just a start.

I am deeply skeptical about the use of risk registers as set out in guides and their use in practice. A missed opportunity and downright dangerous.

The big picture: why we should care

The risk register diagram below comes from the orange book, a UK government publication offering public sector guidance on risk and uncertainty (2004, but never withdrawn). It will come as a surprise to some to learn that there are widely differing views on the appropriate role (if any) for risk registers within risk management.

Standards, guides and users

Many users of risk registers will have been steered in their direction by auditors or risk management standards such as COSO or ISO31000. The frequent use of risk registers can be regarded as useful or an imposition.

Few will have given much thought to the possibility that a risk register is the siren song of risk management.

Some risk experts dissent strongly

Leaving aside those who don't want to do risk management at all, the strongest opponents of risk registers are typically risk experts. Noting that bad use of risk registers is endemic, they argue that risk register are poor tools, noting that they are almost always associated with bad risk assessment methodologies and non-integrated forms of risk management.

It should make you stop and think. I's made me think hard and as a result I've written a series of articles.

Good, bad, odd and ugly use: the quadrants

The diagram below shows the classification introduced at the start of the article: how good is your risk register and do you use it with other tools?

  1. Good. The risk register is marked by sharp thinking; risk is more than future events, probability-impact (aka likelihood and impact/severity) risk assessment has been replaced by something coherent. The risk register is largely about risk administration and it is used as part of a suite of risk management tools, including models – see below.
  2. Odd. As above, but the risk register is the primary (and in the extreme cases only) risk management tool. There probably aren't many organisations in this quadrant; those deploying mainly one tool are unlikely to produce a good risk register.
  3. Bad. You use a suite of tools, including a risk register. Perhaps generously, I've suggested you're part of the "mass market" (I suspect a tendency for the role of risk registers to grow if left unchecked). So, do you trust your risk register?
    • Is it a Board level tool? Do the "top risks" form part of strategic discussions?
    • Did you know the former government actuary says heatmaps don't work for Boards?
    • Have you had that discussion, or is the risk register politely noted before moving on?
    • Why should others pay attention to a risk register that doesn't work for the Board?
  4. Ugly. As noted above, if a risk register is your main (or, worse, only) risk management tool, the chances are that it's in bad shape. And that's more likely to be the result of flawed thinking than technology problems. Good news: there is hope.

Broad areas of criticism

It would be easy for a risk expert to attack risk registers, based on their inherent limitations or poor practical use. Defences to these attacks could be:

  • Inherent limitations. Risk registers were never intended (e.g.) to project cashflows. That's the job of a cashflow projection model.
  • Poor practical use. There is no reason why risk registers should be restricted to operational event-like risks rather than more general uncertainty.

I have given a much longer list in Risk registers: the claimed flaws and offered improvements in many places in e.g. Better risk assessment. Those links improve thinking and practice regarding stand alone risk registers, so here I'll focus on how a reasonable risk register fits with other tools, specifically with models.

Using register and models together

A wide range of techniques and tools

Matthew Leitch's article on future risk management guidance and standards includes an appendix explaining eight risk management techniques, including scenario planning, war gaming, decision support models incorporating probabilities and agile delivery techniques. This Wikipedia article gives others.

But for now, to show how tools can complement each other, I'm going to focus on just two.

The registers v models comparison

The results of the table comparison below can be grouped into four broad areas:

  1. Models outperform registers in the "quantification" area, where the table's third column is shaded.
  2. Registers outperform models in the "management and monitoring" area, where the second column is shaded.
  3. Communication and discussion. The first part of the table (unshaded). Both tools can be useful, if used with skill.
  4. Risk appetite and limits. This area remains a bit of a minefield and can generate a lot of work with much less value.

Further discussion and links are below the table.

Area [note]RegistersModelsComment
Communication [1] Both can succeed or fail (heatmaps are usually nonsense, models can be a black box). See [1] below.
Board discussion Each brings a particular focus and each has a role to play. See [1] below.
Planning and strategy Strategy needs values, especially in the private sector. Registers do not project or value cashflows.
Quantification [2] Models help quantify risk and value. Registers typically only record the results. See [2] below.
Aggregation and dependency Registers have their work cut out to achieve anything here. See [2] below for more on this rich topic.
Stress and scenario testing Models score again for being able to show the effect of moving one or more of the parameters.
Time effects Unlike registers, models can project cashflows over time and produce discounted values.
Management of risk [3] Registers contain a record of controls and can be used much more actively e.g. risk-control matrices.
Active monitoring A register is a powerful centralised tool for monitoring consistency and (possibly contingent) commitments.
Audit and documentation Registers are a central repository and are transparent. Models are generally neither.
High level risk appetite [4] Meaning (briefly) high level targets for risk and return and their trade offs as measured by (e.g.) RAROC.
High level risk tolerance Meaning (briefly) the high level limits on risk as measured by (e.g.) share of economic capital.
Low level risk limits Much lower level limits e.g. at the "individual risk" rather than risk type level.

[1] Communication, Board discussion, planning and strategy

Models can:

  • produce values (outputs) from assumptions (inputs) but typically give few explanations.
  • produce "value" and "risk" outputs. A good simplified model allows "what if" results in real time.
  • seem like black boxes. They may produce a reaction of cynicism or fear, curtailing discussion.

Registers can:

Perhaps the balance depends on whether the Board's focus is quantification (of risk or value) or management (of risk).

I'd guess the average model is better than the average risk register – they are more integrated with things people care about. So make registers useful.

[2] Quantification, dependency and aggregation, stress testing and time effects

Where an item is modelled there is less need to rely on "expert judgement". Models quantify impacts and naturally capture time effects. Some include a probabilistic element, rather than being simply deterministic. The allowance for dependency is often somewhat speculative, although better than nothing.

Model output can be used to populate the risk assessments in risk registers. For non-modelled items there are better methods than probability-impact approach which is all-too-common in risk registers. This risk register study has some neat ideas on recording links between risk register items. A matrix approach might work (relatively!) well, by encoding the type and level of dependency (A causes B, B causes A, A and B depend on C etc). Dependency is tricky.

[3] Management, active monitoring and documentation of risk

This point might explain the popularity of risk registers with auditors, who often have a background in internal control (I've not seen e.g. models whose documentation so easily takes advantage of spreadsheet-like functionality). Unfortunately we too often have substandard risk registers which:

  • Focus too much on financial and operational event-like risks
  • Use the probability-impact method advocated by COSO's risk assessment (among others)
  • Follow a template audit approach: "tell me your gross and net risks and I'll audit based on the mitigation you're claiming"

Integrated risk management and risk registers

A criticism of risk registers is that they reinforce a "non-integrated" approach to risk management. The preference – in the extreme – is to make all risk management part of day-to day decisions, usually at the "front line". This approach is well motivated in wanting to associate risk management with core decision making. But in extremis it leads to the suggestion that organisations should not have a separate risk function and has some huge disadvantages, particularly for financial institutions (where it would not typically be allowable). Risk registers: who, what, why and how looks at the needs of five stakeholders.

[4] Risk appetite, tolerance and limit

People have tied themselves in knots over this concept, but the underlying reality is the attempt, initially by financial regulators and latterly by the Financial Reporting Council to avoid:

  • the public and private sector disasters chronicled in the AIRMIC Roads to Ruin report (2011, now freely available) and
  • the range of dysfunctional corporate behaviours documented in ACCA's 2012 The Reality of Risk report and survey.

The ACCA survey designer, Matthew Leitch, writes in Making sense of risk appetite, tolerance, and acceptance that:

The common factor in these initiatives is the attempt to influence directly by policies and their implementation important decisions taken inside organizations, in such a way that the limitations of knowledge are better handled. In other words, the aim is to get people to think effectively about 'risk' in some sense when they make important decisions.

Final note: we need to take care over the risk limits shown (and acted on) in risk registers. It's not just that they often don't integrate with the core business. The "risk limits" are arbitrary across a heatmap and usually don't even link to a firm's corporate risk appetite. Take especial care.

Where next? The risk register series

User beware. Many risk experts have warned of the common flaws in risk registers. It doesn't have to be this way. The first half of the set of articles below is generally positive, starting with how five potential audiences might make better use of risk registers. The second half warns of some really dangerous flaws.

© 2014-2017: 4A Risk Management; a trading name of Transformaction Development Limited