Skip to main content

Student Essay Contest Winner: The Federal Reserve of the Next 100 Years: The Promise of "Big Data"

September 16, 2014

Student Essay Contest Winner: The Federal Reserve of the Next 100 Years: The Promise of "Big Data"

The Federal Reserve of the Next 100 Years: The Promise of "Big Data"

Peter Otness
Edina High School
Edina, Minn.

Over the next 100 years, “big data” will revolutionize the Federal Reserve’s conduct of monetary policy. Currently, Federal Reserve policy is based on economic data that at any given time are “only partially known, as key information on spending, production, and prices becomes available only with a lag.”1  As a result, policymakers may be forced to “act on the basis of misleading information.”2  In the future, the availability of vast amounts of data, along with the computing power to interpret and analyze it—so-called big data3—will allow the Federal Reserve to react more quickly and effectively to changes in the U.S. economy. While there will still be uncertainties regarding the timing and magnitude of the economy’s response to Federal Reserve policy, lags and misleading information will no longer be significant impediments to policymakers.

The impact of lags and gaps in economic data can be seen in the lead-up to the financial crisis of 2008-09. On Sept. 16, 2008, the day after the collapse of Lehman Brothers, the Federal Open Market Committee kept its target for the federal funds rate at 2 percent.4  While the transcript of that meeting shows considerable uncertainty on the part of FOMC members about what was currently happening in key sectors of the economy,5  the FOMC ultimately concluded that the “current stance of monetary policy is consistent with a gradual strengthening of economic growth” beginning in 2009.6  The Fed’s economists also projected a stabilization of the housing market.7  As we now know, the U.S. economy was on the edge of a precipice. While the preliminary August 2008 payroll report released 11 days before the meeting showed a decline of 84,000 jobs,8  revised numbers for this period showed a decline three times as large.9  Far from stabilizing, the downturn in the housing market was accelerating, with rapidly declining prices and rising mortgage delinquencies.10  Initial estimates of fourth quarter 2008 GDP were of a contraction at a rate of 3.8 percent per year.11  Later estimates for this period show that the actual decline was at a rate of 8.9 percent per year.12   

In September 2008, information that could have provided an accurate, up-to-the-minute assessment of the economy did exist: It consisted of the many transactions occurring in every sector of the economy, recorded in real time in the computer networks and accounting systems of private sector companies and government agencies. Access to this real-time information on payroll tax payments, unemployment filings and average hours worked would have provided the FOMC with insight into the actual, not perceived, employment situation. Information on daily retail sales and prices would have revealed spending and growth trends. Information on mortgage payment delinquencies, which were rising rapidly, would have corrected any impression of a housing market recovery. However, while the information existed, the means to collect, analyze and provide it to FOMC policymakers, on a real-time basis, did not. Their response to the deteriorating economy was hampered by the substantial “recognition lag”13  to which the economic information they needed was subject. A key to better economic policy, especially critical in times of financial crisis and dislocation, is access to this information in real time and on a larger scale, coupled with analytical tools to enable policymakers to interpret it quickly and accurately. This is what big data offers.14 

The promise of big data in improving economic policy can already be seen in the Billion Prices Project, which tracks prices in the United States and other countries on a daily basis by using “web scraping” techniques to gather, from publicly available sources, the prices of certain identified goods.15  In comparison, the traditional consumer price index is still determined from data manually (and more expensively) collected for approximately 80,000 items,16  with CPI data for a particular month available after a lag of approximately two weeks (chained CPI is not final until more than a year later).17  Over the past five years, BPP data have closely tracked the CPI.18  The BPP detected not only drops in prices that occurred as soon as two days after the collapse of Lehman Brothers, but also the price recovery that began in January 2009, well before the same information became available through the CPI.19 

However, big data will not automatically lead to better economic reports or forecasts. Correct interpretation and modeling of data by economists and statisticians will still be necessary. Big data has enormous potential, but without careful analysis and modeling, the information it provides may be inaccurate. An example of a large data set that nonetheless produces an apparently flawed result is the monthly ADP private sector employment forecast. While ADP’s report is based on 23 million payroll records from over 400,000 employers,20  in the past six months it has varied by an average of 65,000 jobs per month, or 35 percent, from the authoritative final monthly numbers provided by the Bureau of Labor Statistics.21  It may be that the ADP sample is not broad enough or is not properly modeled. In any case, the discrepancy illustrates the caution necessary in using data derived from a large sample that may appear to have produced a reliable result. Economic data are inherently noisy, and providing good reports and forecasts to policymakers requires separating out “the signal from the noise.”22  The availability of big data does not remove the need for common sense, economic theory or careful research design.23  There will still be a need for the discretion of experienced FOMC officials in making economic policy.

The Federal Reserve should seek real-time access to government and private-company data on economic activity and should invest in the human and computing power necessary to fully utilize these data.24  Access to databases will need to be subject to carefully designed protections for proprietary business information and consumer privacy. The types of relevant data are virtually limitless, but some of the more important would be daily sales information from bricks-and-mortar and online retailers, payroll tax payments, energy use, and mortgage and credit card payments and delinquencies. In the Ninth District alone, information from companies such as Target, UnitedHealth Group, Best Buy and U.S. Bancorp, and from government sources such as the Minnesota Department of Revenue, would provide valuable real-time economic data.

In September 2008, a Federal Reserve economist wondered whether he could take the current retail sales report at face value, noting that “we’ve been head-faked a number of times by the retail sales data, which are subject to some pretty substantial revisions.”25  With comprehensive, accurate real-time data, he would have had the answer to his own question. The information provided by big data will lead to better policymaking by the Federal Reserve of the next 100 years.


1 Board of Governors of the Federal Reserve System. 2005. The Federal Reserve System: Purposes and Functions, 9th ed. Washington, D.C.: Board of Governors of the Federal Reserve System, p. 18. Accessed March 16, 2014.

2 Ibid.

3 “Big data generally is defined as a collection of large datasets that cannot be analyzed with normal statistical methods.” Sara Royster. 2013. “Working with Big Data.” Bureau of Labor Statistics, Occupational Outlook Quarterly (Fall, p. 3). Accessed March 16, 2014.

4 Board of Governors of the Federal Reserve System. 2008. Minutes of the Federal Open Market Committee. Sept. 16, p. 8. Accessed March 16, 2014..

5 Board of Governors of the Federal Reserve System. 2008. Transcript of Meeting of the Federal Open Market Committee. Sept. 16. Mr. Dudley: “It takes a while for us to get the reports from the clearing bank (p. 7).” Mr. Stockton : “[S]ome of the 0.4 percentage point increase in the unemployment rate last month could be statistical noise (p.20).” Mr. Lockhart: “Anecdotal reports ... support the view that the economy is quite weak but not deteriorating markedly. ... I am also starting to hear some reports that housing markets feel as though they are beginning to stabilize (p. 28).” 78-79: Mr. Warsh: “We are not trying to monitor the broader economy, which we might not be able to measure too much (pp. 78-79).” Accessed March 16, 2014.

6 Board of Governors of the Federal Reserve System. 2008. Minutes of the Federal Open Market Committee. Sept. 16, p. 8.

7 Board of Governors of the Federal Reserve System. 2008. Transcript of Meeting of the Federal Open Market Committee. Sept. 16, p. 20.

8 Bureau of Labor Statistics. 2008. The Employment Situation: August 2008. Accessed March 16, 2014, at

9 Bureau of Labor Statistics. 2014. Employment, Hours and Earnings from the Current Employment Statistics Survey (National). Accessed March 16, 2014, at

10 Federal Reserve Bank of St. Louis. 2014. “S&P Case-Shiller 20-City Home Price Index.” Accessed March 16, 2014. “Delinquency Rate on Single-Family Residential Mortgages , Booked in Domestic Offices, Top 100 Banks Ranked by Assets.” Accessed March 16, 2014.

11 Bureau of Economic Analysis. 2009. “Gross Domestic Product: Fourth Quarter 2008 (Advance).” Accessed March 16, 2014.

12 Eugene P. Seskin and Shelly Smith. 2011. “Annual Revision of the National Income and Product Accounts.” Survey of Current Business 91 (August, p.12). Accessed March 16, 2014. Bureau of Economic Analysis. 2011. FAQ: “How Did the Recent GDP Revisions Change the Picture of the 2007-2009 Recession and the Recovery?” Aug. 5. Accessed March 16, 2014, at

13 Lloyd B. Thomas. 2011. The Financial Crisis and Federal Reserve Policy. New York: Palgrave Macmillan, p. 203.

14 Uran Einav and Jonathan Levin. 2013. “The Data Revolution and Economic Analysis.” Prepared for the NBER Innovation Policy and the Economy Conference. April 23, p. 4. Accessed March 16, 2014..

15 Alberto Cavallo. 2012. “The Billion Prices Project: Building Economic Indicators from Online Data.” Presented at the Meeting of the Group of Experts on Consumer Price Indices, United Nations Economic Commission for Europe, Geneva, May 31, p. 3. Accessed March 16, 2014l.

16 Bureau of Labor Statistics. 2013. FAQ: “How Are CPI Prices Collected and Reviewed?” Aug. 15. Accessed March 16, 2014, at

17 Bureau of Labor Statistics. 2014. Release Calendar: Schedule of Releases for the CPI. Accessed March 16, 2014 at Bureau of Labor Statistics. 2007. “The Consumer Price Index (Updated 06/2007).” Chap. 17 in BLS Handbook of Methods, p. 5. Accessed March 16, 2014.

18 Einav and Levin, p. 10.

19 Alberto Cavallo. 2010. “MIT Sloan Professors Publish Real-Time Inflation Rates Around the World in ‘Billion Prices Project.’” Billion Prices Project @ MIT. Nov. 8. Accessed March 16, 2014.

20 ADP Research Institute.  National Employment Trends, Report FAQs. Accessed March 16, 2014. Harry J. Enten. 2013. “How Good Is ADP at Forecasting the Monthly BLS Jobs Reports?Guardian (London, May 4). Accessed March 16, 2014.

21 Calculations by author. Data from ADP Research Institute. National Employment Trends, National Employment Report. Accessed March 16, 2014. Bureau of Labor Statistics. Current Employment Statistics Highlights: February 2014 (March 7, 2014). Accessed March 16, 2014 at

Employment statistics highlights

Large Image

22 Nate Silver. 2012. The Signal and the Noise: Why So Many Predictions Fail—But Some Don’t. New York: Penguin Press.

23 Einav and Levin, p. 26.

24 The Federal Reserve is already taking steps to “enhance the Board’s data environment,” creating in 2013 a new Office of the Chief Data Officer and including a redesign of “data governance and management processes” as one of six priorities identified in its 2012-2015 Strategic Framework. These actions were taken in recognition of the “growing quantity of data and an increased need to share data more broadly.” Board of Governors of the Federal Reserve System. 2013. Strategic Framework 2012-2015, p. 14. Accessed March 16, 2014.

25 Board of Governors of the Federal Reserve System. 2008. Transcript of Meeting of the Federal Open Market Committee. Sept. 16, p. 19.