Author
Price fluctuations have been an economic phenomenon since ancient times. Escalating prices led Roman Emperor Diocletian to enact price controls in a.d. 301, but his “Edict on Maximum Prices” failed to restore financial order. In England during the 16th and early 17th centuries, an inflow of plundered treasure from the New World contributed to 4 percent annual price inflation. The United States experienced sharp price increases during the Revolutionary War—when Congress’ enthusiastic printing of money to finance the war triggered rapid depreciation of the Continental dollar—and during the Civil War. The Great Depression and many other periods in U.S. history have seen the opposite phenomenon—deflation, or declining prices.
Attempts to measure inflation rates—how much prices rise or fall over time—also go back a long way. Economists experimented with index formulas for gauging the average price of various commodities in the 19th century, and the first official indexes measuring broad prices and their changes were developed by government agencies in the early 1900s as a means of settling wage disputes.
Over the past century, government entities such as the U.S. Bureau of Labor Statistics (BLS)—keeper of the Consumer Price Index—have strived to develop more accurate measures of changes in consumer prices, often under intense scrutiny from stakeholders in price fluctuations. Few government statistics have generated as much debate over the decades as price indexes, which have as-sumed an increasingly important role in the economy.
Conceived as tools for adjusting wages and business contracts to current prices, over time price indexes developed into cost-of-living escalators for government programs and crucial indicators of economic performance. Economists began to watch closely the rate of change of price indexes—the inflation rate. A marked and persistent rise in inflation usually presages higher interest rates, which could in turn reduce investment by businesses, slowing economic growth for a time. When inflation approaches zero or becomes negative, it raises the specter of sustained deflation.
No wonder, then, that price indexes are closely watched and often questioned. Economists in particular frequently tussle over the finer points of computing price indexes, said Jack Triplett, an economist with the Brookings Institution who has done extensive research on price measures. “There are great demands that economists make on the accuracy of the CPI, demands that they don’t make on other data they use,” he said. The same could be said of the Personal Consumption Expenditures (PCE) price index, an alternative inflation measure published by the Bureau of Economic Analysis (BEA).
But getting an accurate reading of the overall change in consumer prices isn’t as easy as it would first appear; all kinds of methodological problems bedevil designers and overseers of price indexes. Among the challenges: ensuring that the index represents the consumption patterns of the U.S. population, accounting for the inclination of consumers to substitute items (say, ground turkey for hamburger) in response to relative price changes and adjusting prices to reflect quality improvements and the introduction of new products.
Over the decades, government agencies have been prodded to address these issues and improve price indexes. The nearly constant criticism came initially from trade unions and politicians, and later from economists. During World War II, trade unions insisted that the BLS’s cost-of-living index (predecessor of the CPI) grossly underestimated the rate of wartime inflation. In the 1990s, a congressional commission appointed to review the CPI took the opposite tack, maintaining that the index overestimated the inflation rate.
The science of gauging consumer inflation has made great strides over the past century; current measures are vastly superior to early efforts to track price changes, and the fundamentals of inflation measurement are no longer in dispute. But price indexes remain works in progress; economists continue to sweat the technical details in an effort to improve them further.
Can you believe the prices these days?
Statisticians developed the first gauges of changing price levels over 200 years ago; during the American Revolution, the Massachusetts Legislature developed a rudimentary price index used to adjust soldiers’ wages as the dollar declined in value. The index comprised an average of the prices of four staple commodities: Indian corn, beef, sheep wool and finished leather.1
But the science of measuring price trends wasn’t developed until the late 19th century, when economists and statisticians came to grips with issues such as index formulas, weights (relative expenditures on different items) and sampling error. In the 1870s, the German econometrician Étienne Laspeyres invented the index formula for determining price changes that is at the heart of the CPI. About the same time, British economist William Stanley Jevons advocated the creation of a government-authorized “tabular standard of value” that could be used to update contract prices by measuring average variations in the purchasing power of gold.
In the United States, the early 1900s were a period of massive labor unrest. Prices for most goods were on the rise after a long period of stable or slowly declining prices, and workers in a range of industries were agitating for higher pay.2 Public and private employers wanted a reliable gauge of living expenses for use in wage negotiations, but none existed at the time. The only official price indexes were for selected wholesale commodities sold in a limited number of cities.
In 1904, the federal Bureau of Labor (forerunner of the BLS) published a monthly index of retail food prices gleaned from 800 merchants in large industrial centers. The index, covering the past 13 years, priced 30 principal food items and weighted them according to average consumption. Within a few years, the food price index reflected data gathered from over 1,000 retail establishments in 40 states. The bureau also collected wage data in the surveyed cities.
For the first time, changes in workers’ pay could be compared with changes in the price of food. But labor groups and some politicians blasted the index, charging that it was politically motivated and that it failed to reflect the diminished purchasing power of workers. “It will take more than [the food index] to convince the housewives of the nation that wages have increased in proportion to the increase in prices,” declared the International Association of Machinists.3
World War I provided the impetus for a much more ambitious indexing project—a set of new retail indexes covering an array of domestic items in addition to food. As in previous conflicts, price increases accelerated during the war, particularly in shipbuilding centers. Intent on setting equitable wages for factory workers vital to the war effort, the National War Labor Board in 1918 called upon the BLS to produce nationwide data on the “cost of living”; changes over time in this index would indicate how much household income would have to change to maintain roughly the same standard of living. The stated goal of this exercise: “[I]nsure the subsistence of the worker and his family in health and reasonable comfort.”4
President Woodrow Wilson allocated the bureau $300,000 ($4.4 million in today’s dollars) to conduct a national study of prices and household expenditures. Over the next two years, BLS agents fanned out across the country to collect prices for about 145 consumer products and services. Price takers carefully specified items to make pricing of identical or similar items easier in future surveys and surveyed about 12,000 working-class families in 42 states to gather information about income and consumption patterns.
In 1919, the bureau released the first comprehensive set of cost-of-living indexes for 31 major industrial and shipbuilding centers. Thereafter, updated indexes were issued semiannually for individual cities (Washington, D.C., was added in 1921) and the nation as a whole. In the Roaring ’20s, an era of rising incomes and economic growth, the indexes quickly became standard tools for negotiating wage increases.
“Hidden” price increases
In the 1930s, widespread financial hardship prompted revisions to the BLS’s cost-of-living index, which was still based on consumer purchases during the First World War. The BLS updated and enlarged the index’s basket of goods to better reflect spending by wage-earning households and conducted a new family expenditure survey from 1934 to 1936. The new survey showed a marked rise in the U.S. standard of living since the war, despite the privations of the Great Depression. Workers and their families were buying more ready-made clothing, heating oil and refrigerators, eating more fruits and vegetables, and spending more money at the beauty shop and on driving vacations.
Price controls and rationing during World War II brought more changes to the cost-of-living index—scarce items such as refrigerators, automobiles and tires were removed, for example—and provoked heated opposition by trade union leaders who distrusted the BLS’s methodology.
In spite of federal price controls, prices started rising in 1942 as factories switched from production of goods demanded by consumers to the manufacture of ships, tanks, munitions and other war materiel. After the War Labor Board tied wage increases to the cost-of-living index in what became known as the “Little Steel Agreement,” labor unions attacked the index, charging that it failed to capture the full rise in living costs for industrial workers. In an effort to refute the BLS’s figures, a number of unions collected their own retail price data.
Controversy over the index contributed to labor unrest during the war; ignoring no-strike pledges by union leaders, workers in a number of industries staged a series of wildcat strikes in 1943 and 1944. Toward the end of the war, the War Labor Board created a presidential committee to investigate the unions’ charges. Before it could rule, labor representatives on the committee, including George Meany of the American Federation of Labor (AFL), issued their own blistering critique of the BLS’s price index. In their 1944 report, the labor leaders estimated that the actual rise in the cost of living from 1941 to 1943 was almost twice the 23 percent reported by the BLS (see chart).
As Triplett and Marshall Reinsdorf of the BEA note in a 2006 paper, the unions ascribed the “hidden” price increases to skewed BLS sampling that omitted items not subject to price controls and a decline in the quality of items such as shoes due to wartime shortages of material and labor.5 In a speech, Meany questioned the scientific validity of the cost-of-living index and accused the bureau of “obsequiously” going along with an alleged Roosevelt administration plan to freeze wages.6
The BLS vigorously defended the index, supported by experts involved in the review process who estimated that the index underestimated wartime price increases by only 3 percent to 4 percent. But in a nod to its critics, the bureau changed the name of the index in 1945. Henceforth it would be known as the Consumer Price Index—an acknowledgment that the measure was not a true cost-of-living index because it didn’t fully capture changes in product quality or consumer substitution of items when one becomes cheaper (or its price rises less) than another.
Unions remained suspicious of the CPI into the early 1950s. The AFL and the Congress of Industrial Organizations continued to push for improvements in the index to allow it to serve as a broad measure of living costs, while more radical labor groups damned it as a flawed, politicized instrument of wage suppression.
Finger on the pulse
Before World War II, price indexes were used chiefly as yardsticks for adjusting wages, rents, royalties and other contracts to the cost of living. In the postwar economy, they took on an additional function as a barometer of the general level of inflation, a key indicator of economic performance.
After price controls were phased out in the late 1940s, the inflation rate accelerated; the federal government became increasingly anxious about rising prices and turned to the CPI and BLS wholesale price indexes as a guide to monetary policy. Economists at universities and private research institutions also wanted a reliable gauge of changes in the overall price level.
In 1951, the economics arm of the U.S. Department of Commerce introduced a new price index measure designed specifically as a macroeconomic indicator. The “implicit price deflator,” based on data on personal consumption in the national accounts, was broader in scope than the CPI and used a different index formula to measure average change in the price of consumer goods and services. The deflator would evolve into today’s PCE price index.
The growing importance of the CPI to economic analysis exposed the measure to renewed criticism—leveled not by labor groups, but by professional economists. A 1961 review of the CPI and other federal price indexes commissioned by the Eisenhower administration delved into statistical problems that had long simmered in academic circles. One issue was how to account for changes in product quality; if a washing machine costs the same as last year’s model but performs better, its price has fallen in real terms. Another was how to treat the cost of shelter, which can be viewed as an investment as well as a living expense.
“It isn’t that people way back then didn’t understand what the issues were,” Triplett said. “It’s just that these were things that were hard to resolve. People knew that the quality change problem was serious; they just didn’t know what to do about it.”
A panel of experts chaired by University of Chicago economist George Stigler exhorted the BLS to research potential solutions to these problems, including using the rental equivalence method (estimating market rents for owner-occupied homes) to measure changes in shelter expenses and sampling items more frequently to reflect purchases of new or improved products.
The Stigler Commission’s recommendations were barely noticed by the public, but they prompted intensive research on price indexes that over time greatly improved the CPI and other price indexes.
Just WIN, baby
Taking the measure of inflation took on fresh urgency in the late 1960s and 1970s, when inflation threatened to spiral out of control. Energy prices soared, contributing to annual inflation rates above 4 percent in the early 1970s. “Stagflation”—high inflation coupled with slow economic growth—gripped the nation, spurring the Nixon administration to impose price controls and President Gerald Ford to launch Whip Inflation Now (WIN), a much-lampooned initiative to foster energy conservation and cut consumer spending.
Fighting inflation was job one for policymakers, who sought more precise intelligence on price movements and their interplay with economic output and employment. The BLS and other agencies broadened the scope of their price indexes and developed new ones to obtain a closer reading of inflationary trends and their impact on consumers.
In 1978, the BLS split the CPI into two measures, each representing the buying habits of distinct populations. A new CPI for all urban consumers (CPI-U) expanded the index’s geographic reach beyond large cities to smaller urban areas and added previously excluded groups of consumers such as salaried employees, part-time workers, the unemployed and retirees. This is the measure of “headline,” or overall, inflation that is reported most widely each month.
The traditional CPI dating to World War I continued as the CPI for urban wage earners and clerical workers (CPI-W), with weights reflecting the consumption patterns (more gasoline purchased by commuters, for example) of wage earners.
Research by economist Robert Gordon of Northwestern University gave rise to the CPI excluding food and energy, a measure of “core” inflation that first appeared in the annual Economic Report of the President in 1980. Gordon and many other economists believed that fluctuating prices for energy and food in global markets obscured the elusive inflation “signal” that should inform fiscal and monetary policy. In particular, oil prices fixed by producers in the Middle East were viewed as artificial distortions of the inflation rate, says John Greenlees, chief of the BLS’s Division of Price and Index Number Research.
“Part of the idea behind a core index was to get rid of volatility or noise in the index,” he said, “and part of it was to try to limit the measure to price changes that really reflected [inflationary forces] within the U.S. as opposed to something that’s just arbitrarily set by Arab states.”
The inflationary 1970s also saw major revisions to the PCE implicit price deflator, which was primarily used for macroeconomic analysis and forecasting. In 1976, the BEA changed the formula of the deflator to focus on pure price change instead of measuring changes in both prices and quantities of items purchased by households. This modification made the PCE a full-fledged measure of consumer inflation akin to but distinct from the CPI. (For more on the PCE and how it differs from the CPI, see sidebar.)
Greenspan speaks; Boskin weighs in
Sharp increases in interest rates in the late 1970s reined in inflation, ushering in the Great Moderation—an extended period of modest business cycles and low inflation. Even though the stagflation beast had been tamed, price indexes remained key indicators of economic performance, carefully watched by the Fed and other policymakers. And the CPI became more and more embedded in government operations as a cost-of-living escalator. Social Security benefits had been indexed to the CPI since the early 1970s; beginning in 1985, the CPI-U was used to make adjustments to federal income tax brackets, exemptions and deductions.
Over the next 30 years, the BLS and the BEA significantly improved their indexes by gathering more timely price and expenditure data and by revising methods of calculating price changes to reduce—if not entirely eliminate—statistical biases.
The BLS had resisted for over 20 years the Stigler Commission’s recommendation that it revamp its method of calculating changes in shelter costs. The Bureau had stuck with an asset-based approach to measuring homeowner costs—tracking house prices. But in the early 1980s, after a period of rapid home appreciation, the agency warmed to the idea—endorsed by most economists of the day—that including home prices in the CPI distorted the index.
In 1983, the BLS switched to the rental equivalence method of measuring homeowner costs: CPI price gatherers consider how much a house would rent for if the owner rented it to someone else.
A major impetus for further changes in the CPI came from the Boskin Commission, an advisory group appointed by the U.S. Senate Finance Committee in 1995 to study the CPI.
Federal Reserve Chairman Alan Greenspan had caused a furor by stating in testimony to the committee that aspects of the CPI’s construction “point in the direction of an overstatement of increases in the cost of living.”7
The report of the commission, chaired by Stanford University economist Michael Boskin, supported Greenspan’s assertion. In a turnabout from past criticism of the CPI that it downplayed inflation, the Boskin Commission found that the current index overestimated the rise in living costs by just over 1 percentage point per year.
The implications of this were enormous, provoking intense interest in the Boskin Report from economists, politicians and journalists around the world. If the CPI overstated inflation, then economic output and productivity had grown more than previously believed in the 1990s. Real median income had risen more than the official CPI indicated. And—an eye opener for seniors and taxpayers—Social Security cost-of-living adjustments were too high and should be trimmed. The Boskin Commission’s finding also held great import for interest rate policy at the Federal Reserve.
The commission recommended major revisions to the CPI that would largely correct the upward bias and make the measure a more accurate cost-of-living index, that is, an index that accounted to a greater degree for factors such as quality change, consumer substitution and technological innovation.
Reconstruction projects
The Boskin Commission found that more than half of the CPI’s upward bias was due to the failure of the index to keep up with new products coming into the market and quality improvements in existing products.
Because household expenditure surveys were conducted only once every 10 years, the CPI missed the introduction of new products such as the microwave, VCR and cellular phone. By the time they entered the index (cell phones finally made the cut in 1998), they were much less expensive, but this price drop was missed by the index. To catch new products earlier, the BLS switched to a continuous survey process in which the CPI’s expenditure weights are updated roughly every two years.
To address the quality change issue, the BLS expanded its use of “hedonic regression modeling,” a technique developed in the 1930s to measure the impact of quality improvement (or decline) on the price of a product. If the price of an item rose, the portion of the increase that reflected improved quality didn’t count as inflation in the CPI. The BLS had been using hedonic models to track the prices of clothing since the late 1980s; in response to the Boskin Report, the Bureau began applying the method to appliances and electronics, including TVs and computers.
Other changes to the CPI tackled the vexing problem of consumer substitution, which the Boskin Report found also contributed to the index’s overstatement of inflation. The BLS had never found a solution to this issue, which had been raised by the Stigler Commission as well. The CPI’s method of figuring monthly price changes couldn’t accommodate the readiness of shoppers to substitute away from items that increased in relative price.
In 1999, the BLS partially solved the problem by modifying the Laspeyres formula used to measure price change at the basic item level (apples in Milwaukee or steaks in New York) in the CPI. The new calculation assumed that consumers responded to relative price hikes by buying similar, cheaper items; as a result, overall prices rose less than they did under the old formula.
Nevertheless, the CPI was still subject to bias from “upper level” substitution—consumer trade-offs among broader item categories such as rice and pasta. To address this shortcoming, the bureau in 2002 added yet another measure of price change to its family of indexes—the chained CPI for all urban consumers. Intended primarily a research tool, the C-CPI-U relies on a different price-change formula that better accounts for consumer substitution among all item classes.
The Boskin Commission didn’t examine the PCE index. Since its inception, the BEA’s price deflator has largely avoided the scrutiny directed at the CPI—perhaps because the PCE has never served as a cost-of-living index. The CPI “affects more people’s lives,” observed Clinton McCully, a lead researcher in the Consumption Branch of the BEA, which publishes the PCE. “A lot of money is riding on the CPI that’s not riding on the PCE—things like Social Security adjustments [and commercial] contracts.”
But like its more famous cousin, the PCE also was revamped in the mid-1990s. Seeking to hone the index as a macroeconomic indicator, the BEA reworked the PCE into an index whose formula, like the C-CPI-U, accounts for broad month-to-month changes in consumption patterns. In 1995, the BEA added a PCE index that excluded food and energy prices, analogous to the CPI “core” price index.
Chasing the dream
Tremendous progress has been made in measuring inflation; thanks to more than 100 years of research and development, today’s price indexes are far more accurate than the first food and commodity price indexes. But the work begun in the 1800s by pioneers such as Laspeyres and Jevons and carried forward into the 21st century by teams of researchers at the BLS and the BEA is not yet finished.
Economists and statisticians still struggle with technical problems that dogged researchers in the 1930s. For example, new products trickle quite slowly into the CPI, despite the BLS’s efforts to continuously update its basket of goods. It can take up to four years for the latest tablet computer to be fully represented in the national index as price takers make their rounds of stores across the country. So, aggregate monthly price changes for that item are missed. (This type of omission also affects the PCE, because 75 percent to 80 percent of personal expenditures tracked by the PCE consist of CPI price data.)
Another challenge is fully accounting for quality change in certain markets such as computer software, health care and the airline industry. Medical technology, for instance, is in constant flux, with new treatments quickly supplanting the old. If a hospital charges more for hip replacement surgery, how much of the price increase is due to better quality—improved techniques and more advanced materials that reduce pain and speed healing? Difficult to tell, considering that complete recovery may take years.
Similarly, cinema admissions and airfares defy hedonic analysis; the BLS doesn’t try to compare the quality of new movies to last year’s or put a value on reduced leg space in jetliners. “There are just things we miss,” Greenlees said. “It’s not because we don’t want to adjust for quality change … it’s really that there’s no known way to make these kinds of adjustments.”
Researchers constantly tweak indexes and experiment with new ones in an ongoing effort to more precisely measure price change. In 2009, the BEA modified the “core” PCE index to include previously omitted meals and beverages purchased away from home. The Cleveland Federal Reserve computes median and trimmed mean versions of the CPI—alternate measures of consumer inflation that eliminate the high and low extremes of prices for all items.
The latest wrinkle in price indexes is the emergence of “web scraping,” the practice of crunching price data gathered daily from the Internet. The Billion Prices Project, an index published by the Massachusetts Institute of Technology, tracks prices of over a half million products and services sold online in seven countries. The limitations of such web indexes are obvious—many products and services aren’t sold online. But by harnessing the web’s instantaneous access and expansive coverage, they have the potential to make inflation measures more timely and accurate.
In his 1887 essay, “Remedies for Fluctuations of General Prices,” British economist Alfred Marshall wrote that “an absolutely perfect standard of purchasing power is not only unattainable but even unthinkable.”8 Marshall may be proven right, but that possibility doesn’t deter economists and statisticians from chasing the dream. “The CPI is a little like cosmology; there’s always a new problem to explore,” Triplett said. “You’re always looking to make changes, and in doing research, that opens up another question sometimes.”
Endnotes
1 Thomas M. Humphrey. 1979. “The Concept of Indexation in the History of Economic Thought.” In Federal Reserve Readings on Inflation, Federal Reserve Bank of New York, 241.
2 Hayes, Matthew. 2011. “The Social History of Quantifying Inflation: A Sociological Critique.” Journal of Economic Issues 45 (1), 101.
3 Goldberg, Joseph, and William T. Moye. 1985. The First Hundred Years of the Bureau of Labor Statistics. Bulletin 2235, Bureau of Labor Statistics, 138.
4 Ibid., 103-04.
5 Reinsdorf, Marshall, and Jack E. Triplett. 2006. “A Review of Reviews: Ninety Years of Professional Thinking About the Consumer Price Index.” In Price Index Concepts and Measurement, National Bureau of Economic Research, 12.
6 Goldberg, 156.
7 Greenspan, Alan. 1995. Testimony before the Committee on Finance, U.S. Senate, March 13.
8 Humphrey, 246.
