The Control Lifecycle: Objective to Testing

A control passes through five stages. CL1 the control objective, being what must not go wrong. CL2 the control activity, being what somebody actually does. CL3 the control design, being whether that activity would meet the objective if it happened every time. CL4 the control operation, being whether it did happen every time. CL5 the testing that evidences the last two. Skip CL1 and every stage after it is unanchored.

A kitchen holds the whole of this subject already, and nobody in it uses any of the words. A household is cooking for a hundred and twenty guests on the morning of a wedding lunch. Somebody says, unprompted, that they will keep an eye on the gas. Nobody has said why the gas needs watching, so the offer sounds like care and is not yet a control. Asked what must not happen, the household answers instantly: the cooking must not stop halfway through the morning with the rice half done and no way to finish it. The sentence about what must not happen, and not the offer to keep an eye on things, is where a control begins.

Now the sentence can be turned into something a person does. The spare cylinder is lifted and weighed on the bathroom scale the night before by the cousin who does the shopping, and the weight goes on a strip of paper taped inside the kitchen door. Now ask a strange question, and ask it before the wedding rather than after. Suppose that happened exactly as described, every single time, for every wedding this household ever caters. Would the cooking still stop? Then ask a completely different question on the day itself: did the cousin actually weigh it? The two questions sound like the same question and they are not. Almost everything difficult about control comes from people answering one of them while believing they have answered the other.

A control moves through five stages in one fixed order, from what must not go wrong to the evidence that somebody has checked. Vindhya Commercial Bank Limited, an invented bank, carries the example, and every count below belongs to that bank and to nothing else. The bank holds 214 key controls across nine processes numbered PR1 to PR9, and those 214 controls are walked through two of the five stages in order, with an exact count falling out at each one. No threshold and no professional standard fixes the number of stages: the five below are the separable questions, kept apart so that each one can be answered on its own.

What are the five stages of a control, and why does the order matter?

Five stages, numbered CL1 to CL5 throughout this guide so that they stay apart. CL1 is the objective: what must not go wrong. CL2 is the activity: what a named person actually does. CL3 is design: whether that activity would meet the objective if it happened exactly as written, every time. CL4 is operation: whether it did happen. CL5 is testing: gathering the evidence that answers CL3 and then CL4. The numbering is a device of this guide rather than anybody's published scheme, and the stages themselves are ordinary craft that works in a bank, a workshop or a kitchen.

The order is not a matter of tidiness. Each stage takes its meaning from the one before it, so reversing any two produces something that looks like a control and cannot be failed. Design is not a quality a control has on its own. Design is a relationship between the activity and the objective. An activity written without an objective has nothing to be measured against, so the activity cannot be well designed or badly designed. The objective is what design measures the activity against, and that is the whole reason CL1 sits first. The reason is not that objectives are inspiring. Without an objective, CL3 has no question to ask. Once CL3 has no question, CL5 has nothing to test, and the last three stages collapse into a general impression of whether things seem to be going all right.

The collapse shows up in ordinary life. A shopkeeper who says he checks his stock every evening has given CL2 with no CL1. Checked against what, for what? If the objective is that nothing leaves the shop unpaid for, counting stock at closing time is a reasonable activity. If the objective is that no item is ever out of stock when a regular customer asks for it, counting at closing time is far too late and the design fails on its own terms even if the count is perfect and daily. Same activity, same diligence, two different objectives, and only one of them is met. Nobody can tell which objective is met until somebody states the purpose of the counting.

What is a control objective, and why is it written before anything else?

A control objectiveThe sentence saying what must not go wrong, written before anybody decides what to do about it. is a sentence about a bad outcome, written in advance, in words specific enough that somebody could point at a day and say it happened. An objective is not a goal, not an aspiration and not a value. An objective is closer to a description of a headline the institution would hate to read. In process PR3 of this invented bank, collateral management and valuation, the objective is that the value the bank carries for the security it holds is never materially wrong for long enough to matter. In PR4, payments and settlement, it is that money never leaves the bank twice for one instruction, and never leaves at all for an instruction nobody authorised.

Notice what those sentences do not contain. The two sentences name no system, no report, no person and no frequency. The omission is deliberate. An objective has to survive the replacement of everything that currently delivers it. Systems get changed, teams get reorganised, and the sentence about what must not go wrong should still read exactly the same afterwards. An objective written in terms of the current activity is not an objective at all; it is the activity looking at itself in a mirror. If the objective in PR3 were written as the valuation team runs the monthly report, then the moment the monthly report is replaced by something better the objective disappears with it, and there is no longer anything to judge the replacement against.

One failure mode follows from getting this backwards, and it is common enough to be familiar. An institution decides it needs a control library. Somebody sensible goes to each area and asks what checks they perform. The answers are honest, detailed and complete. The result is several hundred activities, each of them real, each of them performed by somebody who can explain it. And because nobody wrote down what any of them was for, there is no way to say whether any of them work. Asked whether a control is well designed, the honest answer is another question: designed to achieve what? The library has recorded habits, some of them excellent, and habits cannot be assessed.

Working the other way round is slower and it produces something testable. The objectives come first, one per thing that must not go wrong in each process, and only then does anybody ask what activity would meet each one. Two things fall out immediately that the first method can never produce. Some objectives turn out to have no activity attached at all. A gap of that kind is the one thing cataloguing what people already do can never find. And some activities turn out to attach to no objective. Such an activity is work performed for reasons nobody can now reconstruct, usually a problem in some earlier year that was solved by adding a step, after which the step outlived the problem.

What does a control activity have to say before anybody can test it?

A control activityWhat a named person actually does, how often and on what evidence. is the second stage, CL2, and it is where the objective stops being a wish and becomes something a human being does on a particular day. The activity is not the outcome. The distinction between activity and outcome sounds pedantic until an argument caused by ignoring it plays out. The outcome the bank wants in PR3 is that collateral values are right. The activity is that a named officer compares the valuation feed against the previous run, checks that a value has arrived for every facility on the list, and signs the exception sheet. The outcome is a state of the world. The activity is a piece of behaviour, and only the behaviour can be designed, performed, skipped or evidenced.

A written control has five fixed parts, and a control missing any of them cannot be tested at CL5 at all. First, the objective it serves, so anybody reading the control knows what it is for. Second, the activity itself, in the present tense, describing what is actually done rather than what is intended. Third, the named person or named role who performs it. A control performed by everybody is performed by nobody. Fourth, the frequency. A check is a different control at daily, monthly and annual. And fifth, the record it leaves behind. The record is the part that decides whether the control can be tested a year later, and the record is the part that gets left out most often. At the moment a control is written, the record feels like paperwork rather than the point.

Consider what a control looks like when the fifth part is missing. Management reviews exceptions regularly. Every word of that is true at this bank and at almost every institution. The sentence names nobody, so there is nobody to ask. The sentence sets no frequency, so no month can be identified as a month when it did not happen. And it leaves no record, so there is nothing to sample and nothing that would count as evidence either way. A tester arriving a year later cannot conclude that the control failed, and, far more damagingly, cannot conclude that it worked. A control that cannot be tested is indistinguishable from a control that nobody performs, and the institution has no way to tell those two situations apart from the inside.

One more distinction belongs here because it changes what the record looks like. A preventive controlA control placed before the event so the thing cannot happen. sits before the event so the thing cannot happen: the payment system refuses to release an instruction that has not been approved by a second person. A detective controlA control placed after the event so the thing that happened is noticed. sits after the event so that a failure which did happen gets noticed: the daily reconciliation that finds the duplicate that went out yesterday. Both are legitimate. The two leave different traces. A preventive control's evidence is often a system setting nobody looked at. A detective control leaves a list of what it caught. The difference matters enormously at CL5 and is decided all the way back at CL2.

What is control design, and what exactly is being judged?

Control designWhether the activity would meet the objective if it happened exactly as written, every time. is stage CL3, and it asks a conditional question. Take the control exactly as written. Now assume, for the length of the question, something that is almost certainly untrue: that it happens exactly as written, every single time, performed perfectly, never skipped, never rushed, never delegated to somebody who did not understand it. Under that assumption, would the objective be met? If yes, the control is designed effectively. If no, it carries a design gap, and it carries that gap regardless of how well anybody performs it.

The assumption feels like ignoring reality, and people find it the hardest part of CL3. Assuming perfect performance is the opposite of ignoring reality. The assumption is a way of isolating one variable. There are two entirely different reasons a control can fail to protect an institution: the control could never have worked, or the control could have worked and did not happen. Both produce the same bad outcome and they need opposite fixes, so any method that answers them together leaves the fix to be chosen by whoever argues hardest. Assuming perfect performance for the length of the design question is the only way to see the first reason clearly. Last month's record is the answer to a different question, asked afterwards, so design is judged on the written control and never on the record.

An instance makes it concrete. A control in PR4 requires a checker to compare the payment instruction against the underlying document before release. Perfectly reasonable, until the next line shows that the same checker can also authorise the release. Now run the conditional question. Suppose this happens exactly as written, every time, forever. A person who intends to push a bad instruction through completes the comparison and approves the release, both, alone. The objective, that money never leaves for an instruction nobody independent has looked at, is not met on any day of the year. A checker who can also authorise is a design gap. The control performed perfectly still does not do the job, so no amount of retraining, reminding or performance managing will close it. How duties are split between people is covered separately, and what matters here is only that this failure lives at CL3.

Now the mirror image, and it is where most reviewers slip. Somebody says a control is badly designed because it was skipped in three of the twelve months. Read that carefully. Being skipped is a statement about what happened, so it belongs to CL4. Being skipped might mean the control is impossible to perform in a busy month. A finding of that kind is real and important, and it is still a finding about operation, so the fix is about capacity, cover and monitoring. Calling it a design gap sends somebody off to rewrite a control that was perfectly capable of meeting its objective, and at the end of that work the control will still be skipped in three months of twelve.

What is control effectiveness, and why is it a separate question from design?

Stage CL4 asks a question of fact. Did the control actually happen, as written, throughout the period? Control effectivenessWhether a control that was designed properly actually operated as stated throughout the period. is the word for the answer, and the word does a lot of quiet damage because it gets attached to both stages by people who mean different things. Somebody says the control is effective. Effective at what? Effective as designed, meaning it would work? Or effective in operation, meaning it ran? The two claims are different and carry different evidence, and an institution that lets one word carry both has removed its own ability to say which fix is needed.

The word throughout is doing more work than anything else in the sentence. A control that operated on the day the tester visited is not a control that operated throughout the period. A control that operated in eleven months of twelve did not operate throughout the period either, and the honest conclusion is that it did not operate effectively, however good eleven out of twelve sounds. A control exists to hold on the day the bad thing arrives, and nothing in the record says the missing month was a quiet one, so the harsh rule is the right one. The bank's own collateral valuation control in PR3 failed for eleven working days in month 10. Incident I10 in its loss record is that failure, and the loss that followed was small at Rs 1.4 crore against a year of Rs 43.8 crore of net operational loss. The size of the loss is luck. The state of the control is the finding.

Now the structural point, and it decides the arithmetic that follows. At this bank, design was tested on all 214 key controls, and 198 were designed effectively while 16 carried a design gap. Operation was then tested only on those 198. Testing operation on 198 rather than on all 214 is not an oversight and not a shortcut to save money. A perfectly reliable performance of a control that could never have met its objective would still leave the objective unmet, so there is no point asking whether such a control was performed reliably. The 16 are already failures. Asking a second question about them would produce an answer that changes nothing and could only confuse the reader of the result.

The picture therefore has three states rather than four, and a lot of published control pictures quietly overreach here. Of the 198 that passed design, 172 operated effectively and 26 did not, and 172 plus 26 is 198. The remaining 16 sit in a state of their own: the operating question was never asked of them, so nobody knows and nobody needs to. A diagram that splits those 16 into operating successes and operating failures is inventing a result the testing never produced, and once a picture starts asserting things nobody measured, a reader has no way to tell which of its other cells were measured either.

What does control testing at CL5 actually produce?

Control testingGathering evidence over a stated period to answer the design question and then the operation question. is the fifth stage and it adds no new question of its own. People treat testing as a separate judgement about whether a control is any good, so the point is worth saying twice. Testing is not a judgement. Testing is the work that answers CL3 and then CL4 with evidence a second person could go back and check. Take that away and the two earlier questions are still perfectly good questions, answered by opinion rather than by anything. Testing is what converts an opinion into a conclusion somebody else can inspect.

Testing takes in a written control and a record covering a stated period. Out comes evidence, and then two conclusions rather than one. Conclusion one: on the control as written, was it designed effectively? Conclusion two, asked only if the first was yes: over the period stated, did it operate effectively throughout? A test that produces one merged verdict on whether the control is fine has thrown away the single piece of information that decides what to do next. How a sample is drawn, how many items are looked at, what counts as evidence for a preventive control against a detective one, and how a tester concludes on a control that runs only once a year, are all real craft and they are taken up separately later in this sequence.

Two properties of the output come back throughout the rest of the sequence and are worth fixing now. First, a conclusion always speaks for a period, and the period is part of the conclusion rather than a footnote to it. Second, the conclusion has to be reproducible: another competent person given the same record and the same control should be able to arrive at the same answer, and if they cannot, what was produced was a view rather than a test. The everyday version is a mechanic and a car. Anybody can say the car seems fine. A test says which parts were looked at, on what date, and what was found, and it survives being handed to a second mechanic.

What happens when 214 controls are walked through the stages in order?

Here is the whole sequence on one invented population, and every count below belongs to Vindhya Commercial Bank Limited and to nothing else. The bank holds 214 key controls across the nine processes PR1 to PR9. Design, stage CL3, was tested on all 214 of them: 198 were designed effectively and 16 carried a design gap, and 198 plus 16 is 214. Operation, stage CL4, was then tested only on the 198 that had passed design: 172 operated effectively and 26 did not, and 172 plus 26 is 198. End to end, 172 of the original 214 came through both stages, or 80.4 per cent.

The counts are unarguable and the rates are where people get into trouble, so the funnel reads as counts before it reads as rates. Two hundred and fourteen controls went in. Sixteen fell out at the first stage and never reached the second. Twenty six fell out at the second stage. Sixteen plus twenty six is forty two, and forty two controls failed one stage or the other, or 19.6 per cent of 214. One hundred and seventy two survived, or 80.4 per cent, and 80.4 plus 19.6 is 100. Nothing in that paragraph requires any arithmetic more difficult than addition, and it is the version to hold on to when a percentage arrives instead.

The forty two failures produced forty two findings at this bank, one written for each control that failed a stage. The one for one relationship is a choice about how findings get written rather than a law of nature, and it is worth knowing that somebody made the choice. An institution that grouped its findings by cause could report the same forty two failures as nine findings, and an institution that wrote one per control per stage could report more. A finding count is only comparable between two institutions once it is known what each of them writes a finding about. How a finding is written, what parts it has and how it is rated are taken up separately later in this sequence.

Why do the two pass rates multiply rather than average?

Now the arithmetic that catches almost everybody, including people who have been doing this for years. The design pass rate is 198 over 214, being 92.5 per cent. The operating pass rate is 172 over 198, being 86.9 per cent. Both are honest numbers about real tests. So what is the end to end result? The instinct is to average them for 89.7 per cent, and 89.7 per cent feels like a fair summary of two stages that scored 92.5 and 86.9. The average is not a summary of anything. Rates that describe stages in sequence multiply, and averaging them describes a population nobody ever tested.

Multiply and the reason is visible. Because the 198 cancels, 198 over 214 times 172 over 198 leaves 172 over 214. The result is exactly 80.4 per cent, and it is the same 80.4 per cent as the count of survivors. The average of 89.7 per cent is 9.3 percentage points kinder than the truth and corresponds to no group of controls anywhere in this invented bank. The overhang is not a rounding difference and not a matter of opinion. Applied to 214 controls it would claim about 192 effective controls, twenty more than the testing found, and nobody could point at which twenty.

The intuition that survives without the algebra is a queue at two counters. A hundred people join the first queue and ninety two get through. Only those ninety two reach the second counter, where eighty seven in every hundred get through, so about eighty do. The second counter only ever handled ninety two, so nobody would say that eighty nine and a half people made it. The averaging error happens in institutions precisely because the two rates arrive on separate lines of a report, each with its own denominator quietly attached, and the reader does the natural thing with two numbers that look similar.

RCSA vs Control Testing: why do two honest numbers disagree?

The same 214 controls were looked at twice in this invented year, by two different instruments, and both produced a number. The risk and control self assessmentThe first line rating its own controls, which the operational risk sequence holds as a process. is the business rating its own controls. The self assessment covered all 214 and rated 196 of them effective. Independent testing is somebody outside the business gathering evidence and concluding on it. Independent testing covered design on all 214 and operation on the 198 that passed, and found 172 effective. Both exercises were carried out properly. Neither number is a lie. How they are run, who completes them and on what cycle is covered separately under operational risk; what matters here is only how to read the two results side by side without being misled.

The two instruments answer the same question with different weight, and the difference is not mainly about honesty. The difference is about what each instrument can see. The business knows what actually happens on a Tuesday afternoon and has no distance from it. The independent tester has distance and no idea what happens on a Tuesday afternoon except what the record shows. The business is also being asked to judge its own work. Judging your own work is hard to do well even with no incentive at all, and there is usually some incentive. So the self assessment is broad, cheap, current and generous, and the independent test is narrow, expensive, slower and unforgiving. An institution wants both, and it wants to know which it is looking at.

Now the trap, and it is the reason this section exists. Somebody puts the two headline numbers on one slide: the business says 91.6 per cent effective, independent testing says 86.9 per cent, a difference of 4.7 percentage points, and the meeting concludes that the self assessment is roughly right with a modest optimism. Every number in that sentence is correct and the comparison is meaningless. The 91.6 per cent is 196 out of 214 and the 86.9 per cent is 172 out of 198, so the two percentages sit on different populations and putting them beside each other compares nothing at all.

Do it like for like. Both on the full 214: the business found 196 effective, being 91.6 per cent, and independent testing found 172 effective, being 80.4 per cent. The gap is 11.2 percentage points, and 196 less 172 is 24, so the gap is exactly 24 controls. So the honest gap is 11.2 points and the flattering version showed 4.7, understating it by 6.5 points. Nobody typed a wrong figure to produce that. The flattering comparison is produced by choosing a base rather than by making a mistake. A comparison built that way survives review very easily, and the only defence is to check what sits under every percentage before comparing any two.

One more test of the same idea shows how little room there is to argue. For the two numbers to agree, what base would the testing result need? For 172 to be 91.6 per cent, the base would have to be about 188 controls. A base of 188 is ten fewer than the 198 that were ever tested for operation, and there is no honest way to arrive at it. There is no base that closes the gap. The rule that comes out of this, and it is the rule the whole of this sequence relies on, is to divide the counts and never the rounded percentages, and to state the denominator every single time a share appears.

Which stage does a failure belong to, and why does the answer change the fix?

Every one of the 42 failures at this bank belongs to exactly one stage, and the stage decides the fix. A design gap at CL3 is closed by changing the control itself: adding a second person, moving the check earlier, widening what gets compared, or replacing the activity with a different one that meets the objective. An operating failure at CL4 is closed by making the existing control actually happen: cover during leave, a shorter queue at month end, somebody who notices on the day it is skipped rather than at the year end test. The two fixes are not interchangeable, and applying either one to the other stage's failure spends the money and changes nothing.

Watch how the classification goes wrong in real rooms. The finding says a control failed. The person accountable answers that the team will be reminded and retrained. Reminding is the response available to a manager with no budget to change a system. If the failure was at CL4, that is exactly right and it may well work. If the failure was at CL3, the retraining is worse than useless: it produces activity, it produces a closed action, it produces a control performed more carefully than before, and the objective is still not met on any day of the year. The agreed fix was implemented in full, so a year later the same finding comes back and everybody is puzzled.

There is a diagnostic question that settles it in one move, and it is just the CL3 question asked out loud. If this control happened exactly as written, every single time, would the bad thing still be possible? If yes, it is a design gap and no amount of performance fixes it. If no, then the control would have worked and something stopped it running, so it is an operating failure and rewriting the control will not help. The household version is a smoke alarm. If it has no battery, that is operation, and the fix is a battery and somebody whose job it is to check. If it is fitted in the garage, the fix is not a better battery. The alarm works perfectly and it is in the wrong room, and it can be tested every week for a decade without that ever being discovered.

How does anybody outside a control function actually use this?

Four kinds of reader use the five stages without ever running a test themselves, and the sequence is worth more to them than the vocabulary is. A lending officer assessing a borrower is shown a document describing the borrower's controls. The document is almost always a list of CL2 activities: reconciliations performed, approvals required, checks in place. The useful question is not how long the list is. The useful question is whether anybody has ever asked the CL3 question about any item on the list, and whether anybody independent has asked the CL4 question. A list of activities with no design conclusion and no operating conclusion behind it is a description of intent, and intent is not evidence.

An analyst reading a listed company's disclosures on internal control uses the sequence to read the silences in a disclosure. A statement that controls were reviewed does not say which stage was reviewed, and the two stages carry very different weight. A statement that controls were designed properly says nothing at all about whether they ran. A statement that they operated effectively throughout the period says a good deal more. The word throughout is doing real work, and the person signing has to be able to stand behind it. The reader's advantage is knowing that a single word like effective can be attached to two different questions, so the first move is always to ask which one.

An investor or a buyer looking at an acquisition reads it hardest of all. A buyer inherits both the numbers and whatever was or was not checking them, and CL3 failures travel with the business while CL4 failures sometimes travel with the people who are about to leave. And a household reads a version of it too, without naming any of it. Anybody who has ever said that the electricity bill is on automatic payment, and then discovered that the automatic payment had been failing quietly since the card expired, has met the difference between a control that was designed to work and a control that was actually working. The design was sound. The operation stopped. Nobody had a CL5 step that would have noticed.

One habit is worth carrying away whatever the work. When somebody says a check is in place, two questions are needed rather than one. Would that check catch the thing it exists to catch, if it happened exactly as described every single time? And did it happen? The first question is answered by reading. The second is answered only by looking at what was left behind. Most arguments about control, in institutions and in households alike, are two people answering different questions and assuming they disagree.