How to Build a Baseline Regression Model, Step by Step

A baseline is a model, not the absence of one. The baseline guesses a single number for every month, and every later model has to beat it. On ten invented months the best single guess is the average of 2.00 per cent, and it misses by 9.4499 percentage points typically. The fitted line halves that to 4.6690, and the ratio between the two is what 0.7559 reports.

Two words in that paragraph need settling before anything else. A traded unitSomething with a price attached and nothing else settled about it. No industry, no issuer, no business behind it. The price is free to move from month to month, and that movement is the only property any of the arithmetic here needs. is a nameless something with a price. Its monthly change is how far that price moved over one month, written as a percentage of where the month started. When this note says the outcome moved 18.50 per cent in one month and the guess was 2.00 per cent, the gap between them is 16.50 percentage pointsThe unit counted in when one percentage is taken away from another. A move up from 2.00 per cent to 18.50 per cent covers 16.50 of them. Saying it rose 16.50 per cent would claim something quite different, so the two are never swapped., and that gap is what everything here is built out of.

Everything after this is an order of work. Not a formula, not a new measure, not a better model. An order, and the order is where the honesty lives.

What is already settled, and what does the order of work add?

Three things are taken as already built and used without being taught again. How a straight line is fitted to a set of paired months, and what it assumes while doing it, is covered separately. The claim each of the two numbers in the fitted line makes is covered separately. And the misses the fitted line leaves behind, read one at a time, are covered separately as well.

The running order is what those three leave out, and a run of steps can do two things no single step can: it can put a number beside a number that makes it mean something, and it can force a check to happen before the result is written rather than after. A reader who already knows what a slope is and what 0.7559 means can still ship a result nobody else can argue with, and an unarguable result is the failure this order of work prevents.

What is a baseline, and why is it the first thing built?

Picture a stall outside one office building, selling one thing, six days a week. The person running it has a rule: today will take what an average day takes, so put out forty plates. The rule is not sophisticated and it is not nothing. The rule is a forecast. It makes a number in advance, the day happens, and at closing time the miss is right there in the unsold plates. Over six days that rule was out by about 3.67 plates a day.

Now the important part. If somebody arrives with a cleverer rule, one that looks at the weather and the day of the week and the queue at the bus stop, there is exactly one question worth asking about it: does it beat 3.67 plates a day? Not is it clever, not does it use more information, not does it feel modern. Does it beat the number the simple rule already put on the board.

A baseline in a fitted model is that stall rule, moved across. The baseline guesses one number for every month in the record, it can be scored on exactly the same measure as anything fancier, and it usually turns out to be harder to beat than people expect. Nobody has said what a fitted model with no baseline beside it is better than, so it has not been shown to be worth anything. The score is not a formality attached to the result. The score is the result.

What is the best single number to guess?

The record here is ten paired observationsTwo readings taken off one case, which then have to stay together. Every month here supplies one figure for the input and one for the outcome, and splitting them apart throws away the only thing the pair says jointly., invented for teaching. Each month carries a monthly change for the Nakshatra unit, the input, and a monthly change for the Vasant unit, the outcome. The ten months are held in time orderThe cases arranged in the order they actually happened, earliest first, rather than sorted by size or by anything else. Sorting them would make some charts tidier and would throw away every question about what follows what. and they stay in it. One of the checks further down reads them along that order, and a sorted record cannot answer it.

Now the baseline question. One figure has to be committed to and used for every one of the ten months. Which figure? Almost everybody says the average without being able to say why, and it is worth watching the arithmetic decide rather than taking it on trust.

Three candidates. A guess of 0.00 per cent for all ten months brings the squared misses to 933. A guess of 2.00 per cent, the average of the ten, brings them to 893. A guess of 5.00 per cent brings them to 983. Moving away in either direction makes the score worse. A lowest point looks like that from either side.

The average of 2.00 per cent wins, and its score of 893 is exactly the total movement of the outcome about its own average. The same 893 turns up underneath 0.7559 further down. The baseline is not a separate object bolted on for comparison. The baseline is the quantity the fitted line is already being measured against, made visible and given a number of its own.

One honest refinement, and it belongs here rather than in a footnote. The average is the winner on squared misses and it is not the unique winner on absolute misses: on this record any guess from minus 1.00 to 2.50 per cent leaves the average absolute miss at 7.50 per cent. Which is a small thing that carries a large instruction. The answer to what is best moves when the measure moves, so the measure scored on has to be stated.

In what order do the steps go?

Seven steps, and each one is a single line. The content of each step is either already built or is one instruction. No step is hard on its own, and the whole of the difficulty is doing them in this order rather than the order that feels natural.

1. Write down the question, and whether the answer wanted is a size or a label.
2. Fix the record and the order it is held in, and change neither afterwards.
3. Score the baseline. Before anything at all has been fitted.
4. Fit the line, with one input.
5. Score the fitted line on the same measure the baseline was scored on.
6. Look at the misses, against the input and against the clock.
7. Write the result down, with the check that failed sitting beside the ones that passed.

The order that feels natural runs differently. Fitting is the interesting part, so the natural order fits first and fetches a baseline afterwards if a reviewer asks for one. A baseline scored after the fit is a baseline chosen to lose, so step three before step four is the whole discipline. Once 0.7559 has been seen, the number the baseline has to sit below is known, and one of the several available gets picked, without anyone meaning to.

What tells whether the fitted model beat the baseline?

Step five, and it is one rule: score both on the same measure, and put them on the same line. The measure has to be in the units of the outcome itself, per cent here, and per cent is the only form a reader can argue with. Somebody can disagree with a claim that the typical miss fell from 9.4499 per cent to 4.6690. 0.7559 has no units and therefore no foothold, so nobody can disagree with it.

Now the comparison in one number. R squared is one minus the fitted squared misses over the baseline squared misses. One minus 218 over 893 is 0.7559. The two rows above are the whole content of 0.7559, and the single number carries nothing at all those rows do not already carry. Reported without those two rows it is a number without a scale, and a reader has no way to know whether the improvement is worth having in the units they actually care about.

The bottom row of that table is where people are quietly misled. On squared misses the fitted line keeps 24.41 per cent of the baseline. On the typical miss, in the outcome's own units, it keeps 49.41 per cent. The two figures describe the same improvement, and one of them sounds twice as good as the other. Why the squared scale behaves that way is covered separately. The instruction that belongs in this order of work is to report both, so the reader can see the spreadHow far apart a set of figures sit from each other, measured about their own average. Built and used earlier, and taken as known here. of the misses in units they can picture.

What is checked before reporting anything?

Three checks, and the procedure says all three happen before the writing up, not after somebody queries the result. Each is a picture and each has a named way of failing.

Check one, the misses against the input. Draw a scatterA plot where every case becomes one dot, positioned by two of its figures at once. Nothing is joined up and nothing is summarised; the picture is only the pile of dots. of the miss against the input value for each month. The shape is what matters. A run of dots that sags in the middle and rises at both ends says the straight line was the wrong shape for the job. On this record there is no tilt at all: the misses and the input are exactly uncorrelated, and the ten months were constructed that way. Check one passes.

Check two, the misses against the clock. The months go back in the order they happened and the misses are drawn along it. Runs are what matters: stretches where the misses sit on the same side of the line for several months together. Here the first three misses are all above the line and the last four are all below it, and five of the nine consecutive pairs sit on the same side. Measured, the lag one figure on the misses is 0.4862. A miss above the line is followed by another miss above the line about half the time. The record fails check two, and the procedure requires that failure to be written down rather than fixed at this stage. What that figure means and what is done about it is covered separately, and the whole point of the order of work is that the failure is recorded whether or not the cure is known.

Check three, whether any input is a near copy of another. Two inputs saying almost the same thing leave the fit looking exactly as healthy as before while the numbers attached to each of them stop meaning anything. Here there is one input, so there is nothing to compare it with and the check does not apply. A blank line reads as untested, so say so in the write up rather than leaving it blank.

What is written down beside the result?

Six items. A result missing any one of them cannot be checked by a second person. Being checkable is the only property that makes a result useful to anybody who was not in the room while it was built.

The fourth row is where most write ups quietly go wrong, and it is not the number that goes missing but the unit attached to it. A coefficientOne of the numbers inside a fitted line, sitting either on its own or multiplied by an input. What each one claims, and how far it can be trusted, is covered separately. written as 1.5000 with nothing beside it can be read four different ways by four readers. Written as per cent of the outcome for each one per cent of the input, it can be read exactly one way. And the last row is what distinguishes a result from a claim: writing down the check that failed, beside the ones that passed, in the same size type.

Where does this procedure stop?

The procedure stops at one fitted model, honestly scored against one baseline, with its failures written down. The finished object is complete, and nothing is missing from it. The procedure does not choose between several fitted models. It does not hold part of the record back in order to score on the part held back. Nor does it penalise a model for the number of inputs it uses. Penalising is a separate idea with its own arithmetic, and adjusted R squared appears in the write up above as 0.7254 without comment only because reading it properly is covered separately.

All three are real, all three are useful, and every one of them is a different subject. Cross validation scores a model on part of a record deliberately kept away from the fitting, and it is covered separately. Regularisation pulls the numbers in a fitted model towards zero on purpose to buy stability, and it is covered separately. Choosing among a shortlist of fitted models by any criterion at all is covered separately. Stopping here is not a limitation of the procedure but its whole intent: a baseline honestly beaten is a result, and a model picked out of a shortlist with no baseline behind it is not, however carefully the shortlist was built.

What do all seven steps look like run end to end?

Here is the whole thing on the ten invented months, one step at a time, with nothing skipped and nothing added.

Step one, the question. How much does the Vasant unit move in a month? The answer wanted is a size in per cent, not a yes or a no. The distinction between a size and a label decides everything downstream. A question wanting a label is scored differently, and that is covered separately.

Step two, the record. Ten months, each carrying the monthly change of the Nakshatra unit and of the Vasant unit. Both invented. The ten are held in the order they happened, and that order is fixed here, before any fitting, to give check two something to read. Ten months make a small sampleThe handful of cases actually in hand, as against the much larger set the conclusion is meant to describe. Ten months is a small one, and what small does to a conclusion was built earlier and is taken as known here. and the write up says so.

Step three, the baseline. Guess 2.00 per cent, the average of the ten outcomes, for every single month. Squared misses 893. Typical miss 9.4499 per cent. Average miss 7.50 per cent. Scored and written down, and nothing has been fitted yet.

Step four, the fit. One input. The line comes out as 0.5000 plus 1.5000 times the Nakshatra unit. How it was found is covered separately and is not repeated here.

Step five, the same measure. Squared misses 218. Typical miss 4.6690 per cent. Average miss 3.60 per cent. Same three measures, same record, same order. R squared is one minus 218 over 893, or 0.7559, and adjusted R squared is 0.7254.

Step six, the checks. Misses against the input: no tilt and no bend, passed. Misses against the clock: three above the line then four below, and the lag one figure is 0.4862, failed. Near copies among the inputs: one input, not applicable.

Step seven, written down. All six items from the table above, with the failed check written in the same size type as the two that passed. The write up is the finished result, and the last residualThe standard name for one of these misses: the gap between what the outcome actually did in a month and what the fitted line said it would do. Reading them one at a time is covered separately. line is the one a reader will be grateful for.

Where does this go wrong in the room, and what does it cost?

An analyst who does this weekly, a lender scoring an applicant against a book of past applicants, an investor sizing how much of one thing moves with another: all three are running the same seven steps, usually without writing them down. Five failures account for almost all of it, and every one of them is an honest mistake made by somebody competent.

1. Scoring the baseline after the fit. The number is now a target instead of a question, and the baseline that gets chosen is the one that loses. Costs the analyst the ability to claim the comparison was fair.
2. Comparing the two on different measures. A baseline quoted as an average miss against a model quoted as R squared is not a comparison. Costs the analyst the arithmetic, because the two do not reduce to each other in the reader's head.
3. Reporting the fit statistic without the leftover in the outcome's own units. 0.7559 with no 4.6690 beside it is a number nobody can push back on. Costs the reader the ability to disagree, which is the same thing as costing the analyst the review.
4. Running only the checks the model passes. The shape check passed here and got written down; the clock check failed and, in the failure below, never got run at all. Costs the analyst the one finding that mattered.
5. Trimming or reordering the record after seeing the first result. Sorting ten months by size for a tidier chart destroys check two outright, and dropping the worst month improves every figure in the write up for no reason at all. Costs the analyst the record.

All five are made in good faith, and every one of them is prevented by the order rather than by the intention: doing the steps in this order makes four of the five impossible and makes the fifth visible.