U.S. Bureau of Labor Statistics

Consumer Price Index

Overview of the 1998 revision of the Consumer Price Index

The current revision of the Consumer Price Index encompasses changes ranging from reselecting areas, items, and outlets, to new systems for data collecting and processing

The Consumer Price Index (CPI) is the principal source of information concerning trends in consumer prices and inflation in the United States, and is one of the Nation’s most important economic indicators. The measure is used extensively for economic analysis and policy formulation in both the public and private sectors, and for escalation of contract amounts and other payments among individuals and organizations. The CPI also has a significant impact on the finances of the Federal Government. It is used to adjust payments to Social Security recipients, to Federal and military retirees, and for a number of entitlement programs such as food stamps and school lunches. An increase in the CPI increases Federal statutory obligations for these payments and programs. In addition, individual income tax brackets and personal exemptions are adjusted for inflation using the CPI. In this case, an increase in the CPI results in lower tax revenues. It is estimated, for example, that in fiscal year 1996, each 1-percent increase in the index produced a $5.7 billion increase in outlays and a $2.5 billion decline in revenues.¹
To maintain the accuracy of the CPI, an updating of the index is undertaken approximately every 10 years. The most fundamental and visible activity in each of these CPI revisions is the introduction of a new "market basket," or set of expenditure weights attached to the categories of goods and services comprising the CPI. Because the next market basket introduction will occur in January 1998, the current revision effort is usually identified as the 1998 CPI revision. Revisions of the CPI are comprehensive, multiyear efforts, however, and the current revision is planned to be completed over a 6-year period ending in 2000. The projects and changes encompassed in the current revision—the sixth major revision in the CPI’s history—range from the reselection and reclassification of areas, items, and outlets, to the development of new systems for data collection and processing. This article provides a general description of those projects and changes, and directs the reader to additional articles in this issue for more in-depth treatment of several topics.

About revisions

Context.The CPI is a measure of price change for a fixed market basket of goods and services of constant quantity and quality purchased for consumption.² Consumers change their purchasing patterns in response to a myriad of factors, including relative price changes, real income changes, the introduction of new product distribution patterns and marketing techniques, population and other demographic changes, and changes in consumer preferences. The CPI samples and weights are updated during revisions to reflect these marketplace changes, because without such updates the index would cease to be an accurate measure of current price inflation. The technological improvements undertaken during CPI revisions also contribute to improved accuracy by making possible more and faster data collection, with fewer errors.
It is important to note, however, that numerous methodological improvements in the CPI have taken place outside the revision framework. Among the most prominent examples of these are the annual adjustment for changes in the quality of new cars after model changeovers introduced in 1967, the shift to flow-of-services measures of the cost of owner-occupied housing in the early 1980s, and the implementation of regression-based methods for quality adjustment of apparel prices starting in 1991. Exhibit 1 presents a chronology of some of these changes. Also outside the context of decennial revisions, the CPI program develops experimental indexes of consumer prices corresponding to population subgroups or using alternative aggregation formulas. One example of such an experimental index is the CPI-E, an index based on the expenditure patterns of consumer units with reference persons aged 62 or older.³ Another index, currently under development, uses a geometric mean formula to average the prices of items within CPI product categories.⁴ Recently, the Bureau has begun an evaluation effort to determine whether this geometric mean formula should be adopted, in full or in part, in the official CPI.
This current revision occurs at a time of heightened public scrutiny of the CPI. As mentioned above, CPI movements have a major influence on Federal revenues and expenditures. Largely for this reason, issues relating to the accuracy of the CPI have gained the attention of observers both inside and outside of government.⁵ Most notably, following hearings in the spring of 1995, the Senate Finance Committee appointed an advisory commission to study the CPI and make recommendations for methodological improvements.⁶

One source of upward bias that has been of particular concern since its discovery by BLS researchers was a technical problem that tied the weight of a CPI sample item to its expected price change. This flaw was effectively eliminated by the 1995 and 1996 changes to CPI sample rotation and substitution procedures and to the functional form used to calculate changes in the cost of shelter for homeowners.⁷ Numerous issues remain, however, all generally resulting from the dynamic nature of U.S. consumer markets—new goods, new types of outlets, increases or decreases in product quality, and consumer substitution behavior—and the difficulty of completely reflecting such changes in the CPI. It is generally recognized that the identification of solutions to most of these issues will require much additional research. The 1998 revision, therefore, is not designed to be a solution to, or even BLS response to, issues of potential bias. Nevertheless, some components of the revision are relevant to those issues.
Substitution bias, for example, should be mitigated by the updating of CPI weights and samples. The CPI, as a fixed-weighted index, generally overstates changes in what economists call a true cost-of-living index, because consumers can substitute toward categories of goods and services whose relative prices have fallen. Elimination of this bias would require the use of an alternative formula for aggregating individual item indexes. The BLS produces experimental U.S.-level indexes using such "superlative" formulas,⁸ and research to enhance the accuracy of these indexes is continuing. The data requirements of superlative formulas limit the potential for their use in the official monthly CPI. It is reasonable to expect, however, that the incorporation of more up-to-date expenditure weights in 1998 will reduce the potential gap between the official CPI and an index that fully reflects current consumer spending patterns and responses. The updating of the CPI geographic and housing samples should work in the same direction.
The process of incorporating new goods and new outlets into the CPI will be enhanced by several aspects of the revision. First, the redefinition of item categories is designed to facilitate the introduction of wholly new types of goods and services that might not have fit neatly into the existing CPI item structure. Second, new item and outlet samples will be introduced in the geographic areas that are new to the CPI, and in several item categories that have been extensively redefined. The most important of the latter categories is the hospital services component of the CPI, where a new structuring of the index is specifically designed to better reflect rapidly changing technology and treatment patterns. Third, as part of the revision, the CPI’s sample rotation procedures are being redesigned to permit the accelerated introduction of new items and outlets in those product markets characterized by the most rapid change.

History.The CPI was developed during World War I to meet the need of the Federal Government in establishing cost-of-living adjustments for workers in shipbuilding centers. Rapid increases in prices made such an index essential for calculating these cost-of-living adjustments. Regular publication of a national index began in 1921, based on the prices of 145 selected items in 32 industrial cities and expenditure patterns corresponding to the 1917–19 period.⁹ Since that time, the Bureau has updated, or revised, the CPI five times. Exhibit 2 shows the dates for each of the past revisions, and the period of time during which expenditure patterns were collected to provide the weights for the goods and services that comprise the market basket. Consumer spending patterns during 1982–84, for example, have formed the basis for the CPI weighting structure since January 1987. Effective with publication of CPI data for January 1998, movements in the index will be based on 1993–95 consumer expenditures.
Although reselecting and reweighting the items in the CPI sample is fundamental to the revision process, each historical revision has brought important methodological innovations that improved the accuracy and representativeness of the index. The 1940 revision, for example, in addition to updating expenditure weights because people’s buying patterns had changed substantially since the 1917–19 period, introduced the concept of a sample of cities and items as well as the principle that the priced items could be used to impute the price movement of similar non-priced items. Based on this, the Bureau was able to publish price indexes for categories of items, rather than for separate distinguished items.
During World War II, when many commodities were scarce and goods were rationed, the index weights were adjusted temporarily to reflect those shortages. The first comprehensive postwar revision was completed in January 1953, using weights from an expenditure survey conducted in 1950. Both medium-sized and small cities were added to the city sample to make the index representative of prices paid by all urban wage earner and clerical worker families.¹⁰
After the 1953 revision, it became apparent that the CPI needed to be revised every decade.¹¹ As a result, the next revision was completed in 1964. For the first time, the expenditure weights included single persons as well as families. In addition, computer processing was used for the first time in the 1964 revision.¹²
The 1978 CPI revision implemented a variety of fundamental changes, many of which were stimulated by the 1961 report of the Price Statistics Review Committee headed by future Nobel Prize winner George J. Stigler.¹³ The revision introduced a quarterly survey approach to the collection of consumer expenditures as well as numerous improvements and innovations in pricing for the CPI. Additionally, 1978 saw the introduction of a second index of consumer prices, a more broadly-based CPI for All Urban Consumers, or CPI-U. This index differed from the CPI for Urban Wage Earners and Clerical Workers (CPI-W) by including the buying patterns of all urban households regardless of the consumer units’ occupational status.
The sampling process of the CPI also moved to a more comprehensive and sophisticated statistical basis in 1978. A new store-specific approach to the item selection process, called disaggregation, was introduced. Perhaps the most significant innovation in the 1978 revision was the introduction of the Point-of-Purchase Survey (POPS). In the POPS, consumer units were interviewed in each geographic area covered in the CPI. Respondents identified the actual stores and other retail outlets in which they shopped and specified the amount they spent for a category of items.
The most recent revision was completed in 1987 and updated the CPI to 1982–84 expenditure patterns. Unlike previous revisions in which all new areas and items were replaced at the same time, the 1987 revision introduced the concept of "rolling-in." New areas, items, and outlet samples were gradually introduced over a period of years using existing updating procedures.¹⁴ The use of systematic sample rotation was also broadened.¹⁵ Finally, the 1987 revision introduced the use of an advanced model for CPI sample allocation. The model uses nonlinear programming techniques to determine the distributions of item and outlet selections across all item and geographic strata and to minimize overall price change sampling variance subject to certain budgetary and operational constraints.

The 1998 revision

The most basic aspect of this revision will be the incorporation of a new set of expenditure weights. Consumer Expenditure Survey data from 1993–95 will be used to calculate a new expenditure weight for each item strata category in every CPI index area. These new market baskets—new geographic area samples, new item structure, and new expenditure weights—will take effect with the index for January 1998. At the same time, many of the samples underlying the CPI will be replaced. These samples include geographic areas, items selected for pricing, and outlets in which items are priced.

Geographic sample.The geographic sample selection process uses stratified sampling to represent the U.S. urban population. Eighty-seven geographic areas, known as primary sampling units, were selected for the revised CPI based on the 1990 decennial census. These units replace the current primary sampling units, which are representative of the 1980 U.S. population distribution. Thirty primary sampling units where prices are currently collected, including Anchorage and Honolulu, were selected with certainty. Another 21 units in the current design were also selected again. Thirty-six new primary sampling units were selected to replace units in the current CPI geographic sample. These 36 new areas will have new outlet and item pricing samples introduced into the revised index for January 1998. As part of the reselection process, the present three metropolitan-area size categories will be consolidated into two. Details on the redesigned CPI geographic sample and the procedures used to select that sample are in "The redesign of the CPI geographic sample."

Item structure.As the geographic sample must reflect today’s population, the CPI market basket needs to be updated to represent current consumption patterns. The item strata (the groupings of items into homogeneous categories for publication) will be redefined and the way in which they are aggregated will change based on shifts taking place in the marketplace. For example, the present seven major groups of goods and services will be restructured into eight major groups with the creation of the ‘education and communication’ group. That new group includes components previously included in the ‘housing’ and ‘other goods and services’groups. The new item structure, along with a summary of the assumptions underlying the development of the new structure, are in "Changing the item structure of the Consumer Price Index."
The selection of commodities and services samples during the revision will benefit from the results of an expanded and re-estimated CPI sample optimization model. As compared with the models used in earlier years, the new design will lead to the selection of relatively fewer outlets and more items per outlet. Also, there will be a broad shift in relative sample size away from the food and beverages category and toward the other major groups. These changes occur largely because updated and improved variance estimates were used in the estimation of the optimization model, and additional sample share constraints were imposed.¹⁶

Publication strategy.The selection of new geographic areas and modernization of the item structure will necessarily result in changes in the number and mix of published CPI series. In addition, during the revision, the publication criteria and procedures have been reviewed extensively. BLS will develop a variance-based publication strategy in which the level of detail published is based on the variance surrounding its item index estimates.
As in the past, the Bureau will publish overlap indexes based on both the new and the old item structure and expenditure weight for several months beginning in February 1998. These overlap indexes will permit users to see first-hand the revision’s effect on the published rate of inflation. Also, effective with the index for January 1999, the CPI will change from a 1982–84 = 100 to a 1993–95 = 100 reference base. Details on the new CPI publication system and structure are in "Publication strategy for the 1998 revised Consumer Price Index."

Housing.The housing portion of the CPI revision has two fundamental components, both of which will be implemented with the index for January 1999:the shift to an improved estimation method for homeowner shelter costs, and the selection of a new housing unit sample using the 1990 decennial census. The new estimator for owners’ equivalent rent will be based on the reweighting of the same rental observations that are used for the residential rent index. In the current CPI estimator, implicit rents for a sample of owner-occupied units are estimated by matching those units to specific rental units. The new method will not require selection of an owner-occupied sample.
This new sample will provide a current set of rental housing units that, as noted above, will be priced to calculate indexes for both residential rent and owners’ equivalent rent. The decennial census provides information that will be used to select the ‘housing’ segments within geographic areas. This selection process utilizes stratification criteria to locate segments that represent the housing stock throughout the geographic area, and also determines the sampling rates that will be used to derive the appropriate total sample of housing units. Housing units will be selected to support the initial revised sample, as well as the shadow samples that will be rotated into the CPI in later years to reduce respondent burden and refresh the sample. In addition, new housing units constructed since the decennial census will be brought into the CPI housing sample through an augmentation process. Additional information on the changes to the housing component of the CPI is in "Revision of the CPI housing sample and estimators."

Hospital services.The medical care component of the CPI also will be extensively revised to reflect the dramatic changes taking place in this sector of the economy, particularly in the delivery of hospital services. Effective with the index for January 1997, the three item strata within the hospital and related services category will be consolidated into two, with only a single hospital services stratum. Pricing procedures are also being fundamentally revised, from pricing individual items (such as a unit of blood or a hospital inpatient day) to pricing the combined sets of goods and services provided on selected patient bills. These changes support the collection of transaction prices and facilitate the measurement of price change in an area where the products and services available are undergoing significant change. A full description of these revisions is in "Revision of the CPI hospital services component."

Other enhancements.Several technological enhancements are planned for the CPI revision as well. We will modernize the processing systems for both the Consumer Expenditure Survey and the housing sample of rental prices. Each of these complex computer processing systems has reached the end of its current life-cycle, and will be modernized to stay current with technology. This revision will also build on the Point-of-Purchase Survey and significantly improve it by utilizing computer-assisted telephone data collection. Utilization of a shorter interview and edits at the point of collection should lead to faster data turnaround and consequently more up-to-date information on the distribution of consumer spending across outlets. Based on the improved survey methodology, BLS can introduce new outlets and items into the CPI more rapidly and efficiently. Moreover, the new Point-of-Purchase Survey sample design will permit a shift to sample rotation by category rather than by geographic area, and thereby facilitate accelerated sample rotation in product areas where markets are most dynamic. The article, "New methodology for selecting CPI outlet samples," describes this new methodology and its implementation.
A final, crucial element in the current revision is the conversion to electronic systems of all commodities-and-services and housing data collection and transmission. A fully-implemented computer-assisted data collection (CADC) system for the CPI will significantly improve the overall quality of CPI data. While quality improvement is the primary justification for CADC, electronic data collection and transmission will probably provide long-term savings through a reduction of mail, paper and printing costs, and a transfer of some edit and review functions from the BLS national office to the local data collector.
CADC instruments will help to assure consistent application of survey rules and data review decisions at the point of collection. The results will be increased accuracy of collected data, greater conformance with collection procedures, a higher percentage of successfully completed collection schedules, and an increase in the yield of usable quotes per release of the index. Electronic data transmission also will allow collection of prices closer to the end of the month, improving coverage of price changes during the reference period.
The initial CADC system implementation, Version I, will be a fully reliable system that meets current data collection standards and will be a foundation upon which future enhancements can be built with confidence. This version will include all functionality necessary for housing. There will be basic edits and checks for collected data. In addition, a survey information and control system will be developed that will provide up-to-date information on the status and location of all housing data collection activities. Implementation of Version I will begin in 1997 and will be completed in 1998.
Strategically, we will build on our computer data collection experience with the housing data before undertaking the pricing and initiation of commodities and services in retail establishments. In Version II, there will be more functionality for initiation, and edits and checks will be expanded. The survey information and control system will likewise expand to control the larger volume of data collected for commodities and services. After Version II is operating smoothly, a part of the commodity analyst review will be incorporated into the CADC system at the point of collection. This will provide automatic checks for the comparability of item substitutions and transfer more functions from the national office staff to the regional staff.

THE MOST SIGNIFICANT MILESTONES in the overall revision schedule are presented in exhibit 3. The conclusion of the revision in 2000 does not mean the end of the CPI improvement process. BLS will continue to enhance its program of experimental indexes and research on CPI measurement issues. It is also reasonable to expect that the recommendations of the Senate Finance Committee’s Advisory Commission, as well as the results of other studies of the CPI by outside researchers, will lead to additions in the list of CPI improvements outside the revision context. Nevertheless, the decennial revision process remains the central method by which the BLS maintains the quality of the CPI and thereby ensures the index’s status as the most accurate and timely measure of changes in consumer prices.

Acknowledgment:

The authors are grateful to Marybeth Tschetter for her comments, suggestions and assistance during the preparation of this paper and the others in this volume. Sylvia Leaver, Frank Ptacek, and Janet Williams also provided helpful comments on the paper. Finally, the authors would like to acknowledge the contributions of John Marcoot, who played a fundamental role in the design and execution of the 1998 CPI Revision as well as earlier revisions.

Footnotes

¹Office of Management and Budget estimate, as reported in BLS budget documents.

²Handbook of Methods, Bulletin 2285 (Bureau of Labor Statistics, 1988) "Chapter 19. The Consumer Price Index."

³For a description of the construction of, and recent movements in, the CPI-E, see Kenneth Stewart and Joseph Pavalone, "Experimental CPI for Americans 62 years of age and older," CPI Detailed Report, April 1996.

⁴Theoretical and empirical comparisons of the geometric mean formula to the arithmetic mean formula used in the CPI are presented in Brent Moulton and Karin Smedley, "A Comparison of Estimators for Elementary Aggregates of the CPI," Bureau of Labor Statistics Working Paper, June 1995.

⁵For general reviews of these issues see, for example, Dennis Fixler, "The Consumer Price Index: underlying concepts and caveats," Monthly Labor Review, December 1993, pp. 3–12; Is the Growth of the CPI a Biased Measure of Changes in the Cost of Living? (Congressional Budget Office, October 1994); Report for the House Budget Committee (Bureau of Labor Statistics, 1995); Matthew Shapiro and David Wilcox, "Mismeasurement in the Consumer Price Index:An Evaluation," NBER Macroeconomics Annual 1996 (Cambridge, MA, National Bureau of Economic Research, forthcoming); and Brent R. Moulton, "Bias in the Consumer Price Index: What is the Evidence?" Journal of Economic Perspectives, Fall 1996, pp. 159–77.

⁶See Consumer Price Index, Hearings before the Committee on Finance, U.S. Senate, U.S. Government Printing Office, 1995; and Toward a More Accurate Measure of the Cost of Living, Final Report to the Senate Finance Committee from the Advisory Commission to Study the Consumer Price Index, December 4, 1996.

⁷For details on these changes, see Steve Henderson and Karin Smedley, "Improvements in estimating the shelter indexes in the CPI," CPI Detailed Report, October 1994, pp. 5–6; Ken Stewart, "Improving CPI sample rotation procedures," CPI Detailed Report, October 1994, pp. 7–8; Ken Stewart, "Extending the improvement in CPI sample rotation procedures," CPI Detailed Report, June 1996, pp. 9–10; and Kenneth J. Stewart, "Improving CPI item substitution procedures," CPI Detailed Report, July 1996, pp. 8–9.

⁸See, for example, Ana Aizcorbe and Patrick Jackman, "The commodity substitution effect in CPI data, 1982–91," Monthly Labor Review, December 1993, pp. 25–33.

⁹The Consumer Price Index:History and Techniques, Bulletin 1517 (Bureau of Labor Statistics), p. 2; and The First Hundred Years of the Bureau of Labor Statistics (Bureau of Labor Statistics, September 1985) pp. 94–97.

¹⁰Handbook of Methods, Bulletin 2285 (Bureau of Labor Statistics, 1992), p. 154.

¹¹Sidney Jaffe, "The Consumer Price Index—Technical Questions and Practical Answers," Paper presented to the American Statistical Association, Dec. 30, 1959.

¹²John Marcoot, "Revision of Consumer Price Index is now underway," Monthly Labor Review, April 1985, p. 28.

¹³Government Price Statistics, U.S. Congress, Joint Economic Committee, 87th Cong., 1st Sess., January 24, 1961.

¹⁴The rolling-in approach effected significant cost and time savings. Rolling-in the new areas into the CPI allowed more time to train field representatives. More importantly, using the existing procedures for introducing new outlet samples on a systematic basis precluded the need to maintain dual operations, one for the current CPI and one for the revised index.

¹⁵Marcoot, "Revision of Consumer Price Index."

¹⁶S.G. Leaver, W. H. Johnson, R.M. Baskin, S. Scarlett, and R. Morse, "Commodities and Services Sample Redesign for the 1998 Consumer Price Index Revision," Proceedings of the Survey Research Methods Section, American Statistical Association, forthcoming.

John S. Greenlees is Assistant Commissioner for Consumer Prices and Price Indexes and Charles C. Mason is an economist in the Division of Consumer Prices and Price Indexes, Bureau of Labor Statistics.

Last Modified Date: October 16, 2001