Consumer price index

You cannot read what you cannot measure, and measuring "the general price level" turns out to be genuinely hard: hard enough that reasonable statisticians build several different indices that disagree, and knowing why they disagree is a real analytical skill. Start with the workhorse: the Consumer Price Index (CPI).

How is a CPI built?

Construction, step by step. A CPI answers one question: what does it cost, today, to buy the basket of goods and services a typical household buys, compared with a base period? Building it takes four ingredients:

1. A basket. Statisticians run a large consumer expenditure survey to find out what households actually spend on (the shares going to food, housing, transport, healthcare, recreation, and so on) and fix a representative basket. In the US the BLS (Bureau of Labor Statistics) does this; in India the NSO/MOSPI does. The basket is meant to be held roughly fixed so that the index measures price change, not spending change. That fixed-basket design is the root of the biases below.
2. Weights. Each category gets a weight equal to its share of household spending. This is where indices differ most: US CPI gives shelter (housing) about a third of the basket; India's CPI gives food and beverages about 46% (see CPI vs WPI in India). The weights are the single biggest structural fact about any price index: they decide which prices matter.
3. Prices. Field staff (and increasingly web-scraping) collect prices for thousands of specific items in specific outlets, month after month.
4. Aggregation. The price changes are combined using the weights into a single index number (say, base year = 100), and the inflation rate is the percentage change in that index, usually year-over-year (this month's index versus the same month a year ago), sometimes month-over-month.

What are the four biases in a CPI?

A CPI built on a fixed basket is technically a Laspeyres-type index (it prices a base-period basket at current prices). That fixed-basket choice is defensible, since it answers a clean question ("what does the same basket cost now?"), but it introduces four well-understood biases, all of which tend to make a naive CPI overstate true inflation. These are not trivia; they are the reason the US and the RBI use the indices they do, and the reason a sophisticated reader treats a CPI print as an estimate with a known lean:

• Substitution bias. When beef gets expensive, households buy more chicken; when one brand's price jumps, they switch to another. A fixed basket cannot capture this: it keeps pricing the old, now-relatively-expensive item at its old weight, so it overstates the cost of living (real households substitute away from what got dear, softening the true blow). This is the largest single bias and the central reason the US Fed prefers PCE (see PCE vs CPI), which uses a formula that captures substitution.
• Quality-change bias. When this year's car has airbags and lane-assist that last year's lacked, part of its higher price buys more car, not pure inflation. Separating "the price went up" from "the product got better" is genuinely hard; if unaddressed, quality improvements masquerade as inflation. Statisticians fight this with hedonic adjustment (statistically pricing the quality attributes and stripping them out), most visibly in electronics and cars, which is why the measured price of a laptop can fall for years even as the sticker price holds, because you get vastly more computer per dollar.
• New-product bias. New goods (the smartphone, the flat-screen TV, the GLP-1 drug) enter the basket years after they appear and usually after their steepest price falls and quality gains are over, so the index misses the early consumer-surplus gains and can overstate inflation for the categories they belong to.
• Outlet-substitution bias. When shoppers shift from full-price stores to discounters, warehouse clubs, and online, they pay less for the same good, but the index, sampling traditional outlets, may miss the saving and overstate the price actually paid.

How much does CPI overstate inflation?

The famous quantification: the Boskin Commission (US, 1996) estimated that these biases together caused the US CPI to overstate true inflation by roughly 1.1 percentage points a year, an enormous figure when you remember that CPI indexes Social Security payments, tax brackets, and trillions in contracts, so an overstatement is a slow, automatic fiscal leak. The BLS has since adopted fixes (more frequent weight updates, geometric-mean formulas at the lowest level, hedonics) that shrank the bias, but a residual overstatement of a few tenths is still the consensus. The analyst's takeaway: every CPI is a considered estimate with a known upward lean and a specific basket; "the inflation rate" is not a fact of nature but the output of a hundred methodological choices, and knowing which choices a given index makes is what lets you compare US CPI to India CPI to PCE without being fooled.