The Analysis Workflow: From Question to Reproducible Answer, and the Notebook That Stays Auditable

An analysis workflow is the ordered set of steps that turns one question into one answer somebody else can rebuild. Run every cell on the Neelbagh returns and the figure is Rs 54,196.55/-. Skip one cleaning cell of four and it reads Rs 54,196.55/-, Rs 55,723.30/-, Rs 53,603.33/- or Rs 69,093.10/-, and the notebook prints no warning at all.

Every rupee figure below was produced by running the cleaning cells over the thirty two rows the market office typed. The rows themselves are printed below, so the arithmetic can be rerun against them.

The last of eight guides adds no seventh tool. Everything it uses is already in place: the written down path from raw source to a table fit to compute on; the two doorways a record can come through; where a stored table sits, what shape it declares and what a key is for; the table an analyst works in and what a grouping check tells; that putting two tables side by side changes the row count before it changes anything else; and that every answer has a count sitting underneath it. A correct method run in the wrong order, or with one step quietly missing, moves the answer by up to Rs 14,896.55/- on a record of thirty two rows.

Three things hold this guide up. The first is the Neelbagh stall record, the market office file carrying one row for each stall in each of four months, thirty two rows in all, with its faults already in it and already named in the guides before this one. The second is the day book, the office running log of twelve notes. The day book is the only thing that settles which of two identical looking figures in month three is real. The third is plain tallying. Nothing below is more than a tally of surviving rows, a tally of cells that actually ran, or a single average with its denominatorThe number sitting under a division. It decides what the answer is an average of, which is why it gets named in the same breath as the figure it produced. named beside it.

What is an analysis workflow, and what does it produce?

A workflow is not a mood and it is not a habit of tidiness. A workflow is an ordered set of steps with a stated start and a stated end. The start is one question in one sentence, and the end is one answer that somebody who was not in the room can rebuild from the same materials. On this record the question is fixed and short: what were the average takings per stall month at the Neelbagh market across months one to four?

Five cells carry that question to its answer, and the order is the whole of the method. Cell one writes the question down, together with the grainThe unit that one row of a table stands for. Here one row is one stall in one month, which is why thirty two rows is not thirty two stalls and never was. of the table it is being asked of. Cell two reads the source exactly as the office handed it over and prints how many rows arrived. Cell three cleans, in four rules, printing the row count it is left with after each one. Cell four computes the figure and prints the count underneath it. Cell five writes the answer out beside the question and the counting conventionA counting choice written down in advance so a second person can repeat it exactly. Without one, two careful people can both be right and still disagree about the same file. it was reached under.

The product of a workflow is not the figure. The product is the pair: a figure and the path that produced it. A figure without its path cannot be defended and, worse, cannot be corrected. If somebody finds a fault in the record next week, an analyst holding only Rs 54,196.55/- has to start again from nothing. An analyst holding the path changes one cell and reruns.

The everyday version is a household one. Two homes run on one salary each. The first files every receipt in the order it was spent, in one envelope for each month. The second writes the monthly total on the front of the envelope and throws the receipts away. Both can say what they spent last March. Only the first home can answer the real question. The real question is never the total, but why March was higher than February, and whether the school fee has already been counted.

What is a research notebook, and what is it not?

A research notebook is a record of what was actually run, in the sequence it ran, with the output of each step kept beside the step that produced it. The definition is that short, and every word in it is load bearing. Actually run, so a cell that was meant to run and did not is not in the record. In sequence, so the order is part of the record rather than a detail. Output kept beside it, so nobody has to take the author's word for what a step returned.

And here is what it is not: a research notebook is not a scratchpad. A scratchpad is where things get tried, broken, jumped back up to, fixed by hand and tried again, and there is nothing wrong with having one. The fault is letting the scratchpad be the record. A cell that was edited after it ran, or run out of sequence, has quietly turned the notebook into fiction, and the cruel part is that the fiction looks complete. Every cell has a number under it. Every number was really printed by something. The trouble is that the cell that printed it no longer exists anywhere in the file.

The discipline that catches this is one line long, and it is the reason the right hand column of the figure above exists. Every cell prints the row count it ended with. Now the sequence is checkable from the printed output alone: thirty two, thirty one, thirty, twenty nine, twenty nine. If the printed counts do not descend in that order, the cells did not run in that order, and that is visible without asking anybody anything.

# CELL 1  the question, written before anything is read
question = "average takings per stall month, Neelbagh, months 1 to 4"
grain    = "one row is one stall in one month"

# CELL 2  the source exactly as the office handed it over
rows = office_file_as_handed_over()      # the 32 rows shown in this guide
print("read as handed over:", len(rows))     # 32

# CELL 3  four rules, each printing what it left behind
rows = drop_empty_takings(rows);      print("after the blank drop:", len(rows))      # 31
rows = drop_office_code(rows);        print("after the code drop:", len(rows))       # 30
rows = drop_withdrawn_return(rows);   print("after the duplicate drop:", len(rows))  # 29
rows = correct_typed_takings(rows);   print("after the correction:", len(rows))      # 29

Cells one to three of the notebook. The three cells read the thirty two rows the market office typed, apply the four cleaning rules in order, and print the row count each one ended with.

# CELL 4  the figure, and the count that sits underneath it
usable = [r.takings for r in rows if r.takings is not None]
print(len(usable), sum(usable))          # 29   1571700
print(sum(usable) / len(usable))         # 54196.551724...

# CELL 5  the answer, written out beside what it answers
write_answer(question, grain,
             convention="takings cells carrying a usable number only",
             figure=sum(usable) / len(usable))

Cells four and five. The count is printed before the figure rather than after it, so the denominator is never something a reader has to go looking for.

What does skipping one cleaning cell do to the answer?

Now the part of this guide that nothing before it could have taught. The notebook has four cleaning rules. Each one in turn is switched off, everything else runs, and the result is read. Four skips, and the record is the same record every time.

Skip the blank drop and the answer holds at Rs 54,196.55/- exactly. The answer moves by nothing. The empty takings box was never in the count in the first place. The row survives, so thirty rows go into the calculation instead of twenty nine, but the convention counts takings cells carrying a usable number, and an empty cell has never been one.

Skip the office code drop and the answer reads Rs 55,723.30/-, up Rs 1,526.75/-. The row carrying the office code for no return received now counts as though ninety nine thousand nine hundred and ninety nine rupees of takings had been declared. Skip the withdrawn return drop and the answer reads Rs 53,603.33/-, down Rs 593.22/-. The first NB-03 return for month two was withdrawn and revised, and leaving it in counts one stall month twice at the lower of the two figures. Skip the typing error correction and the answer reads Rs 69,093.10/-, up Rs 14,896.55/-. A stall that took Rs 48,000/- in month three is being read as having taken Rs 4,80,000/-.

The size of the largest move is not what that table shows. The table shows that one skip moves the answer not at all, one moves it down and two move it up, so the direction of the error cannot be read off which cell was skipped. A reader who arrived expecting an unfinished analysis to read high has already been wrong once on this record, at the withdrawn return. A reader who then decides unfinished analyses read low is wrong twice over. No rule of thumb is available, and the sequence in the notebook therefore has to be printed rather than remembered.

Can a half cleaned answer be worse than an uncleaned one?

Set the two side by side. The notebook that ran three of its four cleaning rules, missing only the typing error correction, lands on Rs 69,093.10/-. The file that nobody touched at all reads Rs 69,035.45/-. The answer, when everything runs, is Rs 54,196.55/-. The half cleaned notebook therefore sits Rs 14,896.55/- from the answer, and the untouched file Rs 14,838.90/- from it.

The half cleaned notebook has ended further from the answer than the file it started with, by Rs 57.65/-. Three rules of four ran, and the result went backwards. The result is worth sitting with, and it kills an intuition almost everybody carries into this work. Cleaning is supposed to walk towards the truth one step at a time. A partly cleaned figure is supposed to be a partly correct figure. It is not. Cleaning is not a distance being closed. Each rule moves the figure in whatever direction that particular fault happened to push it, and three rules that pushed one way can leave the figure past where it began once the fourth, the one that mattered most, never ran.

One more thing about that figure, and it matters for the sequence rather than for the arithmetic. Rs 69,093.10/- is not a new result. The figure is the same computation the opening walkthrough reaches at its third stop, the state of the record with everything cleaned except that one typed cell. The opening walkthrough runs the pipeline forward and arrives there; this guide switches one cell off and arrives there. One computation, named the same way from both directions, and not two findings that happen to agree.

How to reproduce a financial analysis from source data: what does a second person actually need?

Six things, and the list closes. Not seven, and none of the six is optional. The closed list is what makes it a usable test rather than a wish.

1. The source exactly as it was handed over. Not a cleaned copy. The thirty two rows the market office typed, faults and all.
2. The question in one sentence, with the grain of the table it is asked of. Average takings per stall month, asked of a table where one row is one stall in one month.
3. The cells, in the sequence they ran. Not the cells as they now sit on the screen. The two can be different things entirely.
4. The convention, with its denominator named. Takings cells carrying a usable number, so twenty nine of them, not thirty and not thirty two.
5. The row count at every step. Thirty two, thirty one, thirty, twenty nine, twenty nine.
6. The answer. Rs 54,196.55/-, so there is something to check against.

The test that decides whether all six are present is blunt: a stranger holding all six reaches Rs 54,196.55/- without asking a single question, and a stranger missing any one of them cannot. Take away the source and there is nothing to run the cells against. Take away the sequence and they can reach four different figures, as the panel above showed. Take away the convention and they can reach four more, one for each denominator they might reasonably choose. Take away the row counts and they can get the right answer without ever knowing whether they got it the right way. Getting it right without knowing is the same as not knowing. Take away the answer and nobody can tell a successful rebuild from a failed one.

How to build a reproducible financial analysis: which three rules never bend?

Three rules, and each one exists because of a specific way that analyses stop being rebuildable.

Never change the source, ever. Every correction is a cell, not an edit. The moment the record the office handed over is opened and the typed takings fixed by hand, the fault vanishes and so does every trace of the decision made about it. Next month, when somebody asks why NB-08 reads Rs 48,000/- when the office file says Rs 4,80,000/-, there is nothing to point at. With the fault kept in the source and the correction in a cell, the question answers itself.

Never type a figure in by hand. A typed figure has no provenanceThe written trail behind a figure: which record it was drawn out of, and which steps touched it before it arrived. A figure carrying none of that can be repeated but never rebuilt.. A typed figure cannot be rebuilt, cannot be traced back to a cell, and does not change when the source changes. Typing a number is faster than computing it, so this is the rule people break most. A typed figure also looks exactly like a computed one. Breaking the rule therefore costs the most.

And rerun the whole thing from a clean start before anything is sent. A notebook that works only in the accidental order it happened to be run in is a notebook that works once. Restarting, running every cell from the top, and reading the printed row counts in order settles it. If the sequence still gives thirty two, thirty one, thirty, twenty nine, twenty nine and Rs 54,196.55/-, it is a workflow. If it gives something else, it was a scratchpad wearing a workflow's clothes, and that has been found out cheaply.

How to create a reproducible Python financial analysis project: what sits in the one folder?

Four things in one folder, and the arrangement is a layout rather than a purchase. Adopting it costs nothing.

The reason the answer is written out as a file rather than read off a screen is that a screen keeps no history and a file does. Next month the record gains a fifth month, the notebook reruns, and the answer file changes. The change, and its size, are then visible. A figure that only ever existed in a cell output has nothing to compare against. The two analysts below are standing in exactly that spot.

Why do two people get two answers from one file?

Two analysts at the market office are asked the same question and given the same file. The first reports Rs 69,035.45/-. The second reports Rs 55,723.30/-. The two figures sit Rs 13,312.15/- apart, and both people are certain they ran the analysis.

Here is the uncomfortable part. Neither of them is careless. The first read the file, computed the average over every takings cell carrying a number, and reported it. Nobody had made the question precise, so that reading is defensible. The second cleaned, but skipped the office code drop, so a code meaning no return received went into the arithmetic as though it were money. Each of them can describe what they did. Neither of them wrote down which cells ran, in what order, with what count after each, and so there is no artefactSomething a piece of work leaves behind that can be picked up and examined afterwards, as against a memory of having done it. anywhere that can settle the disagreement.

Two clerks in one office both say they totalled the day book. One totalled it, the other totalled it too, and neither wrote down which sheets they turned. There is now no way to find out who missed a sheet short of doing the whole thing again, and by then it is a third total.

What gets checked in the minute before a figure is sent?

One caution on row orderThe sequence the lines were keyed in. The keying order belongs to whoever typed the file, and it says nothing at all about the market the file describes., the easiest thing on this record to read a false meaning into. The direction each skipped cell moved the answer is a fact about these thirty two rows and these four faults. Moving the same faults around changes the directions. Four faults on one record license no rule about which way an unfinished analysis reads, and that is exactly why the sequence gets printed instead of remembered.

What is covered elsewhere