API vs CSV File: One Record, Two Deliveries

An application programming interface (API) and a comma separated values file (CSV) are two deliveries of one record, not two records. The Neelbagh stall returns arrive as 32 rows either way. The file arrives once and whole; the address arrives in pieces, and how many pieces is a setting somebody else chose. The checking differs, not what arrives.

The comparison is four sentences long, and everything below makes each one impossible to argue with. The material is the market office file already introduced: an invented covered market at Neelbagh, a market office writing one row for each stall in each of four months, 32 such rows in the file it hands over, and a takings column with a handful of faults already sitting in it. The office will hand that record over as a file, or answer for it at an address. Both are on the table, and both are introduced separately.

Three things hold the comparison up, and none of them is a matter of taste. First, the record itself, settled separately and used here rather than rebuilt. Second, the two doorways as already introduced: a text file with a header rowThe first line of a text file. It holds the column names and no data of its own, which is what lets every line under it be read as columns rather than as a sentence. on top and a separator between fields, and an address that answers with a nested reply broken into batches. Third, counting. Every reading below is a count of deliveries, a count of rows, or one average with the count underneath it named in the same breath.

Are these two different records, or one record handed over twice?

Getting this wrong poisons everything after it, so start here. The file and the address carry the same record, and every figure computable from one is computable from the other. Flattening the address answer back out gives 32 rows. Opening the file gives 32 rows. In both, 31 of the takings cells carry a number and one is empty. In both, those 31 numbers add to Rs 21,40,099/-. In both, dividing by 31 gives Rs 69,035.45/-.

Here is the everyday version. An electricity board sends a paper bill through the door once a month, and it also keeps a counter at the office where a clerk will read an account out to the customer. Nobody thinks the paper bill and the clerk are describing two different accounts. The account has two doorways, and any argument with the board is never about which doorway is telling the truth. The argument is about which reading was taken, and when.

Why insist on this before anything else? Because a reader who half suspects the two doorways hold different data will read every criterion below as a question about quality, and will end up asking which one is more reliable. The question has no subject, so it has no answer. Reliability is a property of the record and the record is the same one; everything the two doorways differ on is a property of the delivery.

How does each one arrive, and how many times?

The file arrives once. The description ends there. The file lands whole, all 32 rows of it, and nothing about the arrival needs a decision from the analyst.

The address arrives in as many pieces as its batch size decides, and the first genuinely useful idea sits right here. A batch size of 10 means the answer comes back ten rows at a time, so 32 rows need four deliveries holding 10, 10, 10 and 2. A batch size of 4 means eight deliveries. A batch size of 32 means one, and at that setting the address behaves exactly like the file. The batch size is a setting held by whoever publishes the answer, it can be changed without notice, and so the number of deliveries is a fact about their configuration rather than about the record.

What shape does each one arrive in, and what does that shape cost to check?

A file arrives flat, one row for each stall in each month. The grainThe answer to the question, what does one row of this table describe. Stated as a plain sentence, it settles what a row count is actually a count of. is one stall in one month. A flat table has no way to say anything once, so the stall identifier and the stall name are repeated on every row that stall appears in. Kadamba Idli is written out four times. Chandan Tea is written out four times. The repetition is not waste and it is not a fault; it is what makes each row readable on its own.

An address answers nested. Every stall gets written down a single time, with its months hanging underneath. Nothing is repeated. Saving the repetition reads nicely and counts badly. There is no row to count until the reply has been flattened. The practical consequence is narrow and it is the only one worth carrying away: a flat delivery can be counted the moment it lands, and a nested delivery has to be flattened first and counted on both sides of the flattening.

# one delivery from the address, batch 1 of 4, at a batch size of 10
GET  /neelbagh/stall-returns?batch=1&batch_size=10
{ "batch": 1, "batch_size": 10, "rows_in_this_batch": 10,
  "stalls": [
    { "stall_id": "NB-01", "stall_name": "Kadamba Idli",
      "months": [ {"month": 1, "takings_rupees": 42000},
                   {"month": 2, "takings_rupees": 44000} ] },
    { "stall_id": "NB-02", "stall_name": "Chandan Tea",
      "months": [ {"month": 1, "takings_rupees": 31000},
                   {"month": 2, "takings_rupees": 33000} ] }
  ] }

The shape of a single delivery, written with a path that has no host behind it. Two stalls are shown; the real delivery at this setting holds ten rows once flattened.

The answer volunteers a field, rows_in_this_batch. The field is the delivery stating how much of itself it is, and it is the cheapest guard either doorway offers. The file has no equivalent. The file is all of itself, and the count is whatever the reader returns. Both are countable. Only one of them counts itself out loud.

When does each one go out of date, and how would that be known?

A file is a photograph. The file was true at the instant it was written and it has been drifting quietly ever since. Whether somebody wrote the date on the envelope is a matter of office habit, not of the format, and a file that does not carry its own vintage will still open cleanly and give a confident figure. A photograph has no mechanism for noticing that the room has changed, so the file cannot go stale in a way it can report.

An address answers now, and that sounds like the fix until the cost appears. Answering now is exactly why it cannot report what changed since the last time it was asked. There is one instance sitting in this record already: the Harit Greens return for month 2 was filed on day 5 reading Rs 36,400/-, then revised on day 19 reading Rs 39,700/-. Asking the address on day 10 returns the first figure with no hint that a revision is coming. Asking it on day 20 returns, depending on how the office handled it, both rows or only the second. Neither answer mentions that anything moved.

Neither doorway carries provenanceThe written trail behind a number: where it came from, who touched it, and what was done to it before it arrived. It is kept beside a figure because nothing about the figure itself records it. by itself, and both need it written down beside them. One sentence does it: what was asked, where it was asked, on which day, and how many rows came back. Two of those four are free from the address and none of them are free from the file. The file, of the two, is the one people forget to date.

What does each one cost to check?

The cost of checking is the criterion that decides most real cases, and it is the one people notice last. The Neelbagh file prints on one screen, so every fault in it can be pointed at with a finger. Thirty two rows. The duplicated Harit Greens month sits at rows 11 and 12. The empty takings cell sits at row 14. The office code for no return received sits at row 26. Two rows both reading Rs 4,80,000/- sit at rows 23 and 24. All of that can be seen at once, argued about in a meeting, and marked up in pencil.

A batched answer must be reassembled before it can be checked at all, and until it is reassembled, the figure in hand depends entirely on where the reading stopped. The point is not a warning about carelessness. The point is arithmetic, and here is the arithmetic.

The ladder repays a slow walk, and it does something the story about batching does not prepare for. From a batch size of 4 to a batch size of 8 the reading rises. From 8 to 10 it does not move at all. From 10 to 16 it falls. From 16 to 32 it jumps to the record figure. At four settings out of five the first delivery reads below what the whole record reads. At exactly one setting it reaches it, and that is the setting where the address stops being an address and becomes the file.

The repeat in the middle of that ladder is the most dangerous thing in the whole comparison, and it deserves naming rather than admiring. At a batch size of 8 the first delivery holds rows 1 to 8. The eight cells add to Rs 3,08,000/- and give Rs 38,500/-. At a batch size of 10 the first delivery holds rows 1 to 10. The ten cells add to Rs 3,85,000/- and give Rs 38,500/- again. The reason is small and specific. Rows 9 and 10 together come to Rs 77,000/-, and over 2 cells that is Rs 38,500/-, exactly what the eight rows above them average. Two extra rows that happen to sit at the running average cannot move it, and that is a fact about this row orderThe order the rows happen to be sitting in. It belongs to the way the record was written out, not to the record itself, so a fact that depends on it will not survive being sorted. rather than a property of batch sizes.

Put it in the language of a market and it stops feeling clever. Eight stalls average Rs 38,500/- a month between them. Two more stalls walk in, and they happen to average Rs 38,500/- too. Of course the ten stall average is Rs 38,500/-. Nothing about the number eight and the number ten made that happen. Shuffle the file so that the two Rs 4,80,000/- rows sit in positions 9 and 10, and the two settings come apart at once.

Where the faults actually sit sharpens the point. At a batch size of 10, every marked line in that file lands in delivery 2 or delivery 3. A reader who opens the first delivery and stops has not seen a single one of the record faults, and nothing about the delivery they are holding suggests there is anything to see. The delivery is a clean, well formed answer to a question they did not quite mean to ask.

How does each one break, and which break is louder?

A file breaks visibly. A row that is not there is not there, and the count says so. Opening the Neelbagh file expecting 32 rows and finding 29, the difference sits in plain sight before anything has been computed. A file that arrived truncated usually announces itself in a worse way still, with a half written last line that stops a reader dead. Loud, ugly, and cheap to fix.

An address breaks quietly, and this is the expensive one. A short read is a well formed answer to a question nobody meant to ask. The delivery is complete, the fields are correct, the numbers are real, and nothing anywhere reports an error. The figure computed from it is wrong for exactly one reason: one delivery out of four was asked for. The quiet break costs more than the loud one because nothing about it looks like a break, so nobody goes looking.

# the one line guard, run on the ten rows printed above
rows   = delivery_rows
usable = [r for r in rows if r["takings_rupees"] is not None]
figure = sum(r["takings_rupees"] for r in usable) / len(usable)
print(len(rows), "rows in,", len(usable), "usable,", figure)
# 10 rows in, 10 usable, 38500.0

The row count and the count divided by, printed in the same breath as the figure, on the ten rows of the first delivery shown earlier.

The whole fix is one line, and notice that it is the same fix for both doorways. Print the row count and the denominatorThe number a total is divided by. It decides what an average is an average of, which is why two people can divide the same honest total and still disagree about the answer. in the same sentence as the figure. Against the file it records that 32 arrived and 31 were usable. Against the address it records that 10 arrived when 32 were expected. The quiet break is made loud, in one line, for free.

So what does the whole comparison look like on one grid?

Five criteria, two doorways, one reading in each cell. Everything above compresses into this, and the shape of it is the argument: the two columns differ on every row about handling and checking, and on no row about the record.

How is the choice made?

Two lines, and a lender, an analyst or a household budget keeper all use the same two. The file suits a record small enough to print, where pointing at a row matters. The address suits a record too large to hand over in one piece, or one that changes often enough that a photograph would be stale before it was opened.

The whole decision really is those two lines, and it is worth noticing what is not in it. Nothing about which is more modern. Nothing about which is more professional. A household checking its own electricity bills for a year has twelve rows and should absolutely be handed a file. A lender reading stall returns across four hundred markets, refreshed weekly, cannot be handed a file at all and should be given an address. The size and the freshness of the record decide, and nothing else does.

What sits either side of this comparison

The Neelbagh market, its stall record, the market office keeping it, the day book beside it and every stall named on it, Kadamba Idli, Chandan Tea and Harit Greens among them, are invented, and the example address reaches no host.
Educational material. Not advice on any investment, tax, budget or market position.