Interaction Terms: When Two Variables Depend on Each Other

An interaction term is one input multiplied by another, added to a fit so that the first input's slope is allowed to change with the level of the second. On ten invented months an on or off marker carries nothing on its own and, once it is allowed to change the slope, splits the single figure of 1.50 into 1.72 when the marker is off and 0.40 when it is on.

Picture a stall on a pavement outside one office building. Every extra hour it stays open buys some extra takings, and somebody works out the average: one more hour is worth about four hundred rupees. The four hundred rupees is a real number, honestly computed. The same four hundred rupees describes no day of the week. On a Tuesday, when the building is full, the extra hour is worth a great deal more than four hundred rupees. On a Sunday, when the building is empty, it is worth almost nothing. The hour has not changed. The worth of the hour has changed, and the day of the week changed it.

A slope that changes with a second condition is the subject, and the arithmetic version of it is called an interaction term. Two invented traded unitsAn invented label for a thing that carries a price somebody quotes. Neither the kind of thing nor the obligation behind it affects any of the arithmetic., the Nakshatra unit and the Vasant unit, carry the arithmetic below. Each posts one monthly changeThe distance a price travelled over a calendar month, expressed against where that month started, so it comes out as a percentage. One figure covers a whole month. per month, and ten months of both are already on the table from notes that come earlier.

One line fitted through the ten paired observationsTwo readings that belong to the same occasion, so they can be set beside each other. The pairing carries the information, so breaking it leaves the reading meaningless. gave a slope of 1.5000, meaning that a movement of one point in the Nakshatra unit went with 1.50 per cent in the Vasant unit. The 1.5000 turns out to be an average of two numbers nowhere near each other, and the column that separates them was sitting in the record the whole time looking useless.

Deciding which terms to keep when several are on offer, by any rule at all, is covered separately.

What is an interaction term, in plain words?

An interaction term is nothing exotic and it involves no new arithmetic. Two columns of numbers are already in hand. A third column is made by multiplying them together, row by row, and all three go to exactly the same fitting method as before. The method does not change, the assumptions behind it do not change, and the only thing that is different is the number of columns it was given.

The third column buys one specific freedom. Without it, the fit is forced to report a single slope for the first input: one number, applying to every row in the record, whatever else was going on. With it, the fit is allowed to report a slope that shifts depending on where the second input stands. The pavement stall gets to say four hundred rupees on a Sunday and eleven hundred on a Tuesday, instead of being made to say seven hundred and fifty every day of the week.

Notice what the product column looks like when the second input is a plain yes or no. A yes or no enters a fit as a one or a zero, and multiplying by zero gives zero. So in every row where the answer is no, the product column reads zero and contributes nothing whatsoever. In every row where the answer is yes, the product column is one times the input, an exact copy of the input. The product column is therefore the input, present in half the record and absent from the other half. The shape of that column is exactly the shape of a question: does the input behave differently in the two halves?

What is the Sharad marker, and why does its zero correlation matter?

Alongside the ten months of both traded units, somebody recorded one more thing: a plain yes or no against each month. Call it the Sharad marker, another name invented for these notes. The marker is on in months 3, 5, 7, 8 and 10, and off in the other five. The arithmetic below works the same whatever the marker stands for, so what it records does not matter. Filling in a story would only tempt a reader to believe the numbers for the wrong reason.

One property of this marker has to be established before anything else happens. Without it, everything that follows can be waved away. The Sharad marker's correlation with the Nakshatra unit is exactly zero, so the two inputs carry no information about each other at all. That is not a rounding to zero. The correlation is zero, and the reason fits in one line: the five months where the marker is off have an average input of exactly 1.00 per cent, and so do the five months where it is on, and so does the whole record.

Why is that worth a paragraph of its own? Because it closes off the first objection a careful reader raises. When two inputs both look important, the usual suspicion is that they are the same information wearing two hats, and that whatever one appears to explain, the other could have explained instead. Here that suspicion has nowhere to land. The marker says nothing about where the input stands, so it cannot be quietly standing in for the input. Whatever the marker turns out to do in the arithmetic below, it is not doing it by overlapping with the thing beside it.

What does the marker look like tested on its own?

The natural first move is the cautious one. A new input is in hand, so it goes into the fit as an ordinary third column, no multiplying, and the question is whether the fit got better. Adding it that way is not naive. Testing a new column beside the existing one is standard procedure, and a careful analyst is supposed to do it before adding anything.

Here is what comes back. Plain R squared rises from 0.7559 to 0.7845. The rise looks encouraging and proves nothing at all. Plain R squared cannot fall when a column is added: the fit can always weight the new column at zero and reproduce exactly what it did before. A rise was the only thing available. The adjusted figure docks the score once for each column the fit was handed, so it is the figure that can move both ways. The adjusted figure falls, from 0.7254 down to 0.7230.

There is a third number, and it is the most damning of the three. The Nakshatra unit's slope does not move. Not a little, not in the fourth decimal place: it stays at 1.5000 exactly, before and after. Adding the marker changed nothing about what the input was said to be doing. The marker took the fit a coefficient of minus 3.2000 all to itself, shifted every reading down by the same amount whenever it was on, and left the relationship it was meant to illuminate completely untouched.

On this evidence the marker is a useless input, and dropping it is the correct call, made correctly, on the correct number. The verdict is not a trap but an honest description of what those three figures say. Multiplying the marker into the input does not overturn the three figures; it asks a different question of the same column.

What happens when the same marker is allowed to change the slope?

Now multiply the marker into the input and hand the fit a fourth column. Nothing has been collected. No new month was recorded, no second data source was found, nobody went and looked anything up. The ten values in the new column were produced by multiplying two columns that were already sitting on the table.

Plain R squared goes to 0.8658. Again, a rise on its own proves little. The sum of squared residualsAdd up how far the fitted line missed each reading, after squaring each miss so the misses above and below do not cancel. Built in notes that come earlier and used here purely as a total to compare. is the more concrete way to see it: the misses total 218 with the input alone and 119.80 once the product term is in, so 45.05 per cent of the squared missing was removed by a column made out of two columns already in hand. And the adjusted figure, the one that can fall, goes up to 0.7988.

The input that looked useless is the best thing in the fit, and the only thing that changed is that it was allowed to act on the other input rather than beside it. Read the pair of adjusted figures together and the reversal is complete: down to 0.7230 when the marker sits alongside the input, up to 0.7988 when it multiplies it. Same ten ones and zeros, same ten months, same fitting method, opposite verdicts.

What was the single slope of 1.50 hiding?

The scoreboard says the product term earned its place. The scoreboard does not yet say what the product term found. To see that, drop the combined fit for a moment and do something cruder: split the ten months into the five where the marker is off and the five where it is on, and fit a separate line to each group.

In the five marker-off months, the Vasant unit moves 1.7200 per cent for every one point of the Nakshatra unit, starting from 1.8800 when the input is at zero. In the five marker-on months it moves 0.4000 for the same one point, starting from 0.0000. The two slopes are more than four times apart. The single figure of 1.5000 that the opening fit reported is not a compromise between two similar numbers; it is an average of two numbers that are describing quite different behaviour.

The 1.5000 is exactly an average. The arithmetic closes without any rounding, so the average is worth watching close up. Both groups hold five months, so the weights are not the counts of months. The weights are how far the input itself moved inside each group, measured as the total squared distance from that group's average input: 250 in the marker-off months and 50 in the marker-on months. The two weights total 300, the whole movement of the input across all ten months. Multiply through and 250 times 1.7200 is 430, 50 times 0.4000 is 20, the two total 450, and 450 divided by 300 is 1.5000 exactly.

The reason that closes so cleanly is the property established three blocks ago. Because both groups sit at the same average input of 1.00 per cent, there is no leftover between-group term to account for, and the whole relationship divides neatly into the two within-group pieces. The zero correlation is not just a defence against an objection; it is what makes the single slope a clean weighted average of the two.

Drawn on a scatterA picture with one dot per occasion, placed by its two readings: how far along for the first, how far up for the second. Nothing about the dot records when it happened. of the ten months, the split shows up as two lines with clearly different steepness, and the distance between them changes across the picture. The changing distance is the interaction made visible, and one feature of it is easy to miss: the two lines cross. To the right of the crossing the marker-off line runs higher; to the left of it the marker-on line does. The signed distance between them, marker-off reading less marker-on reading, climbs steadily from minus 10.00 at an input of minus 9.00 per cent, through zero at about minus 1.42, to 16.40 at an input of 11.00.

What do the three fits look like side by side?

Here are the three fits on one table, in the order anyone would actually run them, so the whole story sits in one place. Every figure below was worked out from the ten paired months and the ten ones and zeros, and nothing was carried across from anywhere else.

The last column, read downward, gives the whole account in four numbers. The slope holds at 1.5000 while a column is added beside it, holds again, and then splits into two the moment that column is allowed to multiply it. The adjusted column, read downward, tells the same story in the opposite order: worse, then better than either.

The two group fits behind that split are 1.7200 from a starting point of 1.8800 when the marker is off, and 0.4000 from a starting point of 0.0000 when it is on. Turned into readings that can be said out loud: at a Nakshatra change of 6.00 per cent, the marker-off line puts the Vasant unit at 12.20 per cent, the marker-on line puts it at 2.40 per cent, and the single line that was reported before any of this puts it at 9.50 per cent. The marker is worth 9.80 percentage pointA plain unit of difference between two percentages. Going from 2 per cent to 3 per cent is one percentage point. Saying it rose by fifty per cent says something quite different.s at that level of the input and only 1.88 at an input of zero, and the single line is wrong for both groups at once, too low for one and far too high for the other.

What does each of the four coefficients actually claim?

Write the fitted model out in full and there are four numbers to read. The model says the Vasant unit equals 1.8800, plus 1.7200 times the input, less 1.8800 times the marker, less 1.3200 times the input times the marker. Each of those four says something quite specific, and three of them say it only under a condition that is not printed anywhere near the number.

The 1.8800 at the front is the reading when the input is at zero and the marker is off. The 1.7200 is the slope when the marker is off, and only then. The minus 1.3200 is how much that slope changes when the marker turns on: 1.7200 less 1.3200 is 0.4000, the marker-on slope. The coefficient is a statement about steepness and nothing else. All three of those are reasonably safe to read.

The fourth is the dangerous one. The marker coefficient of minus 1.8800 gives the worth of the marker when the input is at exactly zero, and at no other level of the input. Once a product term is in the fit, a plain coefficient stops being an overall effect and becomes a reading taken at one specific value of the other input. Treating that reading as an overall effect is the commonest mistake anyone makes on this subject. At an input of 6.00 per cent the marker is worth 9.80 percentage points, which is more than five times the size and the opposite direction of minus 1.8800.

When is a product term the wrong move?

A product term can always be added. Two columns can always be multiplied together, the fit will always accept the result, and plain R squared will always rise. Nothing in the arithmetic stops it. The discipline has to come from somewhere else. Three situations are worth naming.

The first is having no reason to expect the effect to differ. If nothing about the situation suggested that the input should behave differently in the two groups, and the product term was built in the course of trying things, then the adjusted figure is the only thing standing between the analyst and a decoration. With enough pairs tried, something will always improve the fit whatever the record actually holds, so an interaction found by trying every available pair is a found pattern rather than a finding. What turns it back into a finding is a reason stated before the fit was run, so the fit is a test of that reason rather than a search for one.

The second is a group too small to carry a line of its own. Splitting ten months five and five, as the worked case here does, is already thin. Splitting them seven and three would leave a slope resting on three paired readings: three occasions rather than three pieces of independent confirmation. A line through three points will be produced by the arithmetic whether or not anything is there to find. The split becomes arithmetic rather than evidence, and the honest response is to state the count beside the number every time it is quoted.

The third is when the two inputs are near copies of one another. Near copies are a different problem with a different name, covered separately, and reaching for a product term will not address it. The two situations are worth telling apart: in the worked case the two inputs share exactly nothing, and that is what left the interaction as the only available account of what happened.

How does somebody read a fit that already contains one?

Most of the time the analyst is not the one building the product term. Somebody hands over a fitted model with four numbers in it and asks what it says. The working routine is five checks long.

First, the slope gets said out loud twice, once for each level of the other input, and both sentences are checked for sense. Here that gives two sentences: the Vasant unit moves 1.7200 for a point when the marker is off, and 0.4000 when it is on. If one of the two sentences describes something absurd, the model has said so and the trouble is found in ten seconds. Second, a plain coefficient out of a fit that contains that same input inside a product is never quoted without the level of the other input stated in the same breath. Minus 1.8800 alone is not a claim; minus 1.8800 at an input of zero is.

Third, one of the two slopes rests entirely on the count of observations in the smaller group, so that count has to be asked for. Fourth, whether anyone expected the interaction before it was fitted has to be asked. Expecting it first is the difference between a test and a search. Fifth, the plain figure rises whatever gets added and never distinguishes a good addition from a pointless one, so the adjusted figure is the one to quote. Four of those five checks are questions about where a number came from rather than about the number itself. Reading a fit mostly consists of asking exactly those.

The household version of this is not far away. A person who has been told that an extra hour of overtime is worth six hundred rupees on average, and who arranges their week around that figure, is working from a single slope. If the real position is nine hundred rupees on weekdays and two hundred at the weekend, then the average was never wrong and was never useful either. They needed the condition, and the condition was never printed next to the number.

One last property of the ten months matters for the notes that follow. The ten months are recorded in time orderArranged as they happened, first month first. Sorting them by size would make some pictures tidier and would destroy the only evidence that reading them in sequence can provide. and are never rearranged. Every sum above would come out identically on a shuffled record, so nothing needed the order. Worth noticing in itself: addition does not care what came first, so not one figure above can see whether the misses arrived in a pattern.

And a note on how far any of this reaches. Ten months is a very small sampleThe occasions actually in hand, as against all the occasions there might have been. Everything in this guide is computed on the ten in hand and describes those ten., and splitting it into two groups of five makes each slope rest on five paired readings. The spreadHow widely a set of readings sits about its own average. Built in notes that come earlier and used here only to say how far the input moved inside each group. of the input inside each group is what gave the two slopes whatever standing they have, and the marker-on group had a fifth as much of it. Everything above describes these ten months and makes no claim about an eleventh. A single point of difference between two slopes is a fact about a record, not a forecast about anything.