How to run a Risk and Control Self Assessment

Nine steps, in this order: fix the scope, name the people and the challenger, list the risks, list the controls and mark the key ones, rate design, rate operation against evidence, judge what is left, raise an issue for every control that failed, then sign it and hand it on. Each step out of order leaves a mark that can be read in the finished document.

The definition of a risk and control self assessment, and the reason what comes back is kinder than an independent measurement of the same controls, are covered separately. The runbook is the rest: what happens on the first morning, what happens next, and what each step has to hand the step after it. An assessment can be run end to end, in order, by somebody who has never watched anybody else do it.

Nine steps, numbered RC1 to RC9. The numbering is a working scheme rather than anything an authority publishes. How often an assessment is run, how much of an institution it covers, and what rating words or scale it uses are decided inside an institution and named by whoever supervises it. The order is most of the method.

In what order do the nine steps actually run?

In the order they are numbered. Each step decides what the next step is allowed to be, so reversing any pair produces a defect that can be read straight off the finished document months later.

Step outside banking for a moment. A residential society runs a safety walk once a year. Suppose the committee agrees on the way down the stairs that the building is basically fine, and only then walks around writing things in the report. They are now looking for confirmations rather than for problems, so every item they find will be one they already have an answer for. The walk happened. The report exists. The report contains nothing anybody did not already believe on the staircase.

A safety walk run in that order is a reversal, and reversals leave marks. The reversal leaves a report in which every risk has something written against it and nothing is left uncovered. Fixing the answers first and writing the questions backwards out of them produces exactly that shape. A genuine list contains items nobody has an answer for yet. A list without them went wrong somewhere in the order.

Read the middle column above as inputs and the bottom line of each box as outputs. The sequence is made of nothing but those two things. RC1 hands RC2 a population. RC2 hands RC3 a room. RC3 hands RC4 a list of risks to hang controls on. RC4 hands RC5 and RC6 the set of things to be rated. RC5 and RC6 hand RC7 two ratings to judge against. RC7 hands RC8 the places that are not good enough. RC8 hands RC9 a set of issues with names and dates on them. And RC9 hands the whole thing to whoever looks at it next.

Here is the same sequence written out flat, with the one thing each step produces. If a step has produced nothing anybody can point at, that step has not happened, whatever the plan says.

The step numbering RC1 to RC9 is a working scheme, not a published one. The nine processes, the control count and every figure below belong to one invented bank.

What has to be settled before anybody is asked a single question?

RC1. Fix the scope

The first step is done by the people accountable for the processes being assessed, with the risk function holding the method. Its input is the list of processes the institution runs. Its output is one written paragraph, and that paragraph fixes three things and stops.

The three are these. Which processes are in. Which controls are counted, meaning which ones the institution says it relies on rather than everything anybody does that helps. And over what period, meaning the exact stretch of time the ratings will describe. The scopeThe processes, the controls and the period an assessment covers, settled in writing before any question is asked so that the population cannot be renegotiated once the answers are in. is written down before anybody is asked anything, and it is the only thing standing between a fixed population and one that can be renegotiated later.

At Vindhya Commercial Bank Limited, an invented bank carrying invented figures throughout, the paragraph fixed nine processes, numbered PR1 to PR9: account opening and customer onboarding, lending and disbursal, collateral management and valuation, payments and settlement, trade finance, treasury dealing and settlement, deposit servicing, financial reporting and close, and access management and information security. The paragraph fixed 214 key controls as the population. The period it fixed was the twelve months to month 12, all of it, with no months left out.

Notice what that paragraph does not do. Because this bank never published the split, the paragraph does not say how many of the 214 sit in any one process, and no number for it exists. A runbook step that cannot be completed from the record stops at the record, and the honest output here is a total with no breakdown behind it.

Who is in the room, and who is there to disagree?

RC2. Name the participants and the challenger

The second step is done by the risk function, agreed with whoever will sign the result at the end. Its input is the scope paragraph from RC1. Its output is a named list of people, and one of those names is there for a different reason from all the others.

The participants are the people who know what actually happens: the ones who run the process day to day, the ones who supervise it, and the one who will write the ratings down. Then one more name. A challengerSomebody in the room whose job is to disagree, and who does not run the process or hold any rating in it. is somebody whose job in the session is to disagree, who does not run the process and holds no rating inside it. At this invented bank that name was never added.

Every other step produces a document and RC2 produces a person, and that is exactly why RC2 is the step that goes missing. Nobody decides to skip it. RC2 simply leaves nothing behind, so nothing in the finished pack marks its absence, and a pack assembled without a challenger is indistinguishable from a pack assembled with one until somebody measures the same controls a second time.

A two person tailoring shop makes the difficulty plain. One of them cuts and the other stitches, and asked whether the measurements get checked before cutting, both will say yes, and both will be describing the same shared habit rather than checking each other. The moment a third person walks in and asks to see the last ten jobs, the question changes from what usually happens to what happened. The third person is not cleverer than the two. The third person simply has nothing riding on the answer.

How is a risk list built so that it is not just a list of last year?

RC3. List the risks

The third step is done by the participants named in RC2. Its input is the scope paragraph and three separate sources. Its output is a risk list for each process that is in scope.

The first source is the institution's standing list of categories, the one that says what can go wrong anywhere in a bank of this kind. The standing list supplies completeness. The second source is the record of what has actually gone wrong here, meaning the losses booked and the near misses caught. The loss record supplies specificity. The third source is the people in the room, and what they know has nearly gone wrong and reached neither of the first two. Only the third source can put a risk on the list that has not yet cost anybody anything.

The invented bank's record carries thirteen operational risk incidents for the year, numbered I1 to I13, costing Rs 43.8 crore net of recoveries, and five near misses, numbered N1 to N5. Take one pairing the bank itself records. In month 6 the collateral valuation feed went stale for two working days and a data quality check caught it, which is near miss N3. In month 10 the same feed went stale for eleven working days and 340 loans were wrongly marked, which is incident I10, at Rs 1.4 crore net. Both of those are in the second source, and a risk list built from the second source alone would have carried the feed only after month 6 and never before it.

The pairing shows the shape of the problem with the second source, and the shape is not a criticism of the record. A record can only hold what has happened. Everything it contains arrived by costing something or by very nearly costing something. So a risk list assembled only from it is a careful, accurate, well evidenced description of the year that has just finished.

What makes a control a key control, and who decides?

RC4. List the controls and mark the key ones

The fourth step is done by the participants, with the risk function testing the list they produce. Its input is the risk list from RC3. Its output is a control list with a flag against the ones that will actually be rated.

Almost everything in a working process reduces risk a little. Somebody double checks a figure out of habit. A screen happens to sort in an order that makes an error obvious. A colleague notices things. None of that is nothing, and because there is no moment at which it either happened or did not, none of it can be rated. A key controlA control an institution has decided it relies on, as distinct from everything in a process that reduces risk a little. is one the institution has decided it relies on, and the deciding is the step.

Being a key control is a status somebody grants and records, not a property the control has by itself. The list can therefore change without a single thing in the process changing. The participants propose, the risk function tests what has been proposed against the risk list, and whoever will sign accepts the result. At this invented bank that produced 214 key controls across the nine processes PR1 to PR9, and that 214 is the number every later figure in this guide sits on.

How are design and operation rated, and on what evidence?

RC5. Rate the design

The fifth step is done by the participants, one key control at a time. Its input is the control list from RC4. Its output is a design ratingA judgement about whether a control would achieve its objective if it happened every single time. against every key control on it.

The question the step asks is fixed and narrow: would this control achieve what it is there for, if it happened every single time? Nothing about whether it did happen. The material that answers it is the procedure, the system setting, the description of what is supposed to occur. All of that can be read on a quiet afternoon with nobody else in the room.

RC6. Rate the operation against evidence

The sixth step is done by the same participants, holding up something the process itself produced. Its input is the control list, the design ratings, and whatever the period actually threw off. Its output is an operating ratingA judgement about whether a control did in fact happen every time over the period, which needs a different kind of material from a design judgement. for each key control, together with the material each rating rests on.

The RC6 question is a different one: did the control in fact happen every time, across the period fixed back at RC1? The word evidenceSomething that would exist whether or not anybody had asked, as distinct from a description of what normally happens. is doing one specific job in this step, and the job is to insist on something that would exist whether or not anybody had ever asked. A count of the occasions a check did not run, produced by the system that runs it, exists on its own. A description of what normally happens does not; it comes into being because somebody asked for it, and it answers the RC5 question rather than this one.

RC6 is where most sessions run thin, and the reason is practical rather than moral. The material that answers RC5 is easy to bring to a room. The material that answers RC6 has to be pulled out of a system by somebody, in advance, for every control being rated. If nobody did that before the session, the room can still answer RC5 honestly and cannot answer RC6 at all, and what usually happens next is that the RC5 answer gets written into the RC6 box.

What is left over, and when is that judged?

RC7. Judge what is left

The seventh step is done by the participants once both ratings exist and never before. Its input is the design ratings from RC5 and the operating ratings from RC6. Its output is a residualWhat remains once the controls and their ratings have been taken into account, which is the third judgement and the one that comes last. judgement for each process in scope.

The step is no larger than that, and its position in the sequence is doing more work than anything inside it. The three judgements an assessment makes, and how they relate to each other, are set out separately under the same subject. The runbook settles only where the third judgement goes, and it goes seventh.

Put RC7 first and the two ratings above it stop being measurements and become supporting material for a conclusion that has already been reached. The mark that reversal leaves is a uniformity that is very easy to spot afterwards: a process carrying a comfortable residual judgement in which every single control also came out effective, with nothing awkward anywhere in it. Real processes are not that tidy. A process with one broken control and a low residual is a much more believable document than a process with none.

What comes out at the end, and who is it handed to?

RC8. Raise the issues

The eighth step is done by whoever is accountable for each control that did not pass. Its input is every rating from RC5 and RC6 that came out short. Its output is an issue, and an issue here means three things together: a named owner, an agreed action, and a date.

All three or none. A weakness written down with no name against it belongs to nobody. A name with no action is a person carrying a label. An action with no date closes whenever somebody remembers. The step is not finished when the weakness has been described; it is finished when a person, a thing to do and a day are attached to it. Tracking that issue afterwards, and how long it takes to close, is a separate subject under the same material, and RC8 stops at raising it.

RC9. Sign it and hand it on

The ninth step is done by the person accountable for the process, by name. Its input is everything the eight steps above produced. Its output is a signed result, and the result carries two things rather than one.

The first is the answer: the ratings, control by control, the period they cover and the population they sit on. The second is the recorded disagreementA note of where participants in the session rated the same control differently, which is the output most often left out of the finished document., meaning a note of every control on which the people in the room did not land in the same place. A control two participants rated differently is a live question about what actually happens, and a control everybody agreed on is a closed one, so a result recording only the agreed rating has kept the closed questions and thrown away the open ones.

A sign-offThe point at which somebody accountable accepts the result, which is a decision that can be asked about later rather than an administrative step. is a decision and not a formality. A sign-off converts a document that exists into a statement somebody made. Where the signed result goes next, meaning which committee receives it and what an institution does with it after that, sits with the governance material. The runbook says only that the handing on is a step, that it has an output, and that the output is bigger than the ratings.

What did this invented bank's assessment actually produce?

Run end to end, the nine steps at Vindhya Commercial Bank Limited produced one rated list. Of the 214 key controls in scope, 196 came out effective. As a rate that is 91.6 per cent, and as a body of work it is eighteen named things somebody has to fix.

Some time later the same 214 controls were measured independently, and that measurement reached 172. On the same population that is 80.4 per cent. Put the two on the same 214 and the difference is 11.2 percentage points, which is exactly 24 controls. Both figures have to sit on the full population to be compared at all, and the reason a different pairing of these numbers is wrong, along with the whole account of why the business's answer runs high, is covered separately. The like for like reading is taken as settled here.

Hold on to what 24 is and what it is not. The 24 is a measured distance between two finished exercises: what one described and what the other found, on the same controls, in the same period. The 24 is not a price anybody paid, and because the method was only run once, nothing in this bank's record measures what would have happened if it had been run differently.

What does running the steps out of order actually look like?

How much independent challenge is the right amount?

RC2 is the step with no output document, so the honest way to ask what it is worth is to put a dial on it and turn it. Call the dial q: the share of the 196 ratings the business called effective that a challenger reviews before the result is signed. At this invented bank q was nil.

Turning the dial at all requires an assumption, and the assumption is nowhere in the record. One such assumption is that challenge lands on wrong ratings at the same rate the independent measurement later did. On that assumption the ratings withdrawn are 24 times q, settled to whole controls because a rating cannot be part withdrawn, and what stands is 196 less that number, out of 214 throughout. The independent result is held still at 172.

Every figure belongs to one invented bank. The left hand row is what happened; the other four evaluate one assumption at four settings, and none of them is a measurement.

Challenge over a quarter of the ratings recovers 6 of the 24 controls of distance, over a half recovers 12, over three quarters recovers 18, and only complete challenge recovers all 24. The last of those four settings matters more than it looks. The whole 24 arrives at exactly one setting of the dial, and it is the setting this guide refuses.

Look at the far right of that picture and say what is actually happening there. Every one of the 196 ratings has been reviewed by somebody who did not make it. A full review is no longer a challenge to a self assessment. A full review is a second measurement of the same controls by somebody independent, and that is precisely what the independent measurement already was. The gap closes because one of the two exercises has stopped existing.

Which leaves the trade in plain view. The value of the process is that the people who run the work describe and rate it, because nobody else can describe it at all. The cost of the process is that the people who run the work describe and rate it. RC2 is where an institution picks its position between those two sentences, and the far end of the dial is not the answer to the problem. The far end solves the problem by removing the process that had it.

A finished assessment has just been handed over. What gets read first?

Most people meet this process from the other end. A pack lands on the desk, a hundred printed sides of ratings for a process somebody else runs, with a question attached about whether it can be relied on. The workshop cannot be re-run. Checking whether the nine steps happened in order can still be done, and the pack will show that to anyone who knows where to look.

Read the scope paragraph first, and read it for its date rather than its content. A scope written before the sessions is a definition of the population. A scope written after them is an account of what got left out, and it reads differently: it explains rather than states. Then check whether it names a population at all. A pack that reports a percentage and never says what it is a percentage of has hidden the only number that turns a rate back into a list of work.

Then look for a name against the challenger role, and if there is none, assume the whole distance is still there. The challenger name is the single fastest read in the pack. Nothing else in a set of ratings gives as much for as little effort. Asking for that name is the same move a household makes when somebody says the accounts are fine and one person asks to see the bank statement: the request does not change the fact, it changes the ground the fact is resting on.

After that, three quick tests. Does any control anywhere carry a recorded disagreement? A pack with none across a population this size has either had an unusually quiet year or has dropped the field. Does every issue raised carry all three of an owner, an action and a date? A count of issues with only two of the three is a count of intentions. And does any risk on the list have no control against it? If not one does, RC3 and RC4 probably ran in the wrong order, and the risk list in hand was assembled out of the control list.