The current revision of the Consumer Price Index encompasses changes ranging from reselecting areas, items, and outlets, to new systems for data collecting and processing
The Consumer Price
Index (CPI) is the principal source of information concerning
trends in consumer prices and inflation in the United States, and
is one of the Nations most important economic indicators.
The measure is used extensively for economic analysis and policy
formulation in both the public and private sectors, and for
escalation of contract amounts and other payments among
individuals and organizations. The CPI also has a significant
impact on the finances of the Federal Government. It is used to
adjust payments to Social Security recipients, to Federal and
military retirees, and for a number of entitlement programs such
as food stamps and school lunches. An increase in the CPI
increases Federal statutory obligations for these payments and
programs. In addition, individual income tax brackets and
personal exemptions are adjusted for inflation using the CPI. In
this case, an increase in the CPI results in lower tax revenues.
It is estimated, for example, that in fiscal year 1996, each
1-percent increase in the index produced a $5.7 billion increase
in outlays and a $2.5 billion decline in revenues.1
To maintain the accuracy of the CPI, an updating of the index is undertaken approximately every 10 years. The most fundamental and visible activity in each of these CPI revisions is the introduction of a new "market basket," or set of expenditure weights attached to the categories of goods and services comprising the CPI. Because the next market basket introduction will occur in January 1998, the current revision effort is usually identified as the 1998 CPI revision. Revisions of the CPI are comprehensive, multiyear efforts, however, and the current revision is planned to be completed over a 6-year period ending in 2000. The projects and changes encompassed in the current revisionthe sixth major revision in the CPIs historyrange from the reselection and reclassification of areas, items, and outlets, to the development of new systems for data collection and processing. This article provides a general description of those projects and changes, and directs the reader to additional articles in this issue for more in-depth treatment of several topics.
Context.The CPI is a measure of price
change for a fixed market basket of goods and services of
constant quantity and quality purchased for consumption.2 Consumers change their purchasing
patterns in response to a myriad of factors, including relative
price changes, real income changes, the introduction of new
product distribution patterns and marketing techniques,
population and other demographic changes, and changes in consumer
preferences. The CPI samples and weights are updated during
revisions to reflect these marketplace changes, because without
such updates the index would cease to be an accurate measure of
current price inflation. The technological improvements
undertaken during CPI revisions also contribute to improved
accuracy by making possible more and faster data collection, with
It is important to note, however, that numerous methodological improvements in the CPI have taken place outside the revision framework. Among the most prominent examples of these are the annual adjustment for changes in the quality of new cars after model changeovers introduced in 1967, the shift to flow-of-services measures of the cost of owner-occupied housing in the early 1980s, and the implementation of regression-based methods for quality adjustment of apparel prices starting in 1991. Exhibit 1 presents a chronology of some of these changes. Also outside the context of decennial revisions, the CPI program develops experimental indexes of consumer prices corresponding to population subgroups or using alternative aggregation formulas. One example of such an experimental index is the CPI-E, an index based on the expenditure patterns of consumer units with reference persons aged 62 or older.3 Another index, currently under development, uses a geometric mean formula to average the prices of items within CPI product categories.4 Recently, the Bureau has begun an evaluation effort to determine whether this geometric mean formula should be adopted, in full or in part, in the official CPI.
This current revision occurs at a time of heightened public scrutiny of the CPI. As mentioned above, CPI movements have a major influence on Federal revenues and expenditures. Largely for this reason, issues relating to the accuracy of the CPI have gained the attention of observers both inside and outside of government.5 Most notably, following hearings in the spring of 1995, the Senate Finance Committee appointed an advisory commission to study the CPI and make recommendations for methodological improvements.6
One source of
upward bias that has been of particular concern since its
discovery by BLS researchers was a technical problem that tied
the weight of a CPI sample item to its expected price change.
This flaw was effectively eliminated by the 1995 and 1996 changes
to CPI sample rotation and substitution procedures and to the
functional form used to calculate changes in the cost of shelter
for homeowners.7 Numerous issues
remain, however, all generally resulting from the dynamic nature
of U.S. consumer marketsnew goods, new types of outlets,
increases or decreases in product quality, and consumer
substitution behaviorand the difficulty of completely
reflecting such changes in the CPI. It is generally recognized
that the identification of solutions to most of these issues will
require much additional research. The 1998 revision, therefore,
is not designed to be a solution to, or even BLS response to,
issues of potential bias. Nevertheless, some components of the
revision are relevant to those issues.
Substitution bias, for example, should be mitigated by the updating of CPI weights and samples. The CPI, as a fixed-weighted index, generally overstates changes in what economists call a true cost-of-living index, because consumers can substitute toward categories of goods and services whose relative prices have fallen. Elimination of this bias would require the use of an alternative formula for aggregating individual item indexes. The BLS produces experimental U.S.-level indexes using such "superlative" formulas,8 and research to enhance the accuracy of these indexes is continuing. The data requirements of superlative formulas limit the potential for their use in the official monthly CPI. It is reasonable to expect, however, that the incorporation of more up-to-date expenditure weights in 1998 will reduce the potential gap between the official CPI and an index that fully reflects current consumer spending patterns and responses. The updating of the CPI geographic and housing samples should work in the same direction.
The process of incorporating new goods and new outlets into the CPI will be enhanced by several aspects of the revision. First, the redefinition of item categories is designed to facilitate the introduction of wholly new types of goods and services that might not have fit neatly into the existing CPI item structure. Second, new item and outlet samples will be introduced in the geographic areas that are new to the CPI, and in several item categories that have been extensively redefined. The most important of the latter categories is the hospital services component of the CPI, where a new structuring of the index is specifically designed to better reflect rapidly changing technology and treatment patterns. Third, as part of the revision, the CPIs sample rotation procedures are being redesigned to permit the accelerated introduction of new items and outlets in those product markets characterized by the most rapid change.
CPI was developed during World War I to meet the need of the
Federal Government in establishing cost-of-living adjustments for
workers in shipbuilding centers. Rapid increases in prices made
such an index essential for calculating these cost-of-living
adjustments. Regular publication of a national index began in
1921, based on the prices of 145 selected items in 32 industrial
cities and expenditure patterns corresponding to the 191719
period.9 Since that time, the Bureau
has updated, or revised, the CPI five times. Exhibit
2 shows the dates for each of the past revisions, and the
period of time during which expenditure patterns were collected
to provide the weights for the goods and services that comprise
the market basket. Consumer spending patterns during
198284, for example, have formed the basis for the CPI
weighting structure since January 1987. Effective with
publication of CPI data for January 1998, movements in the index
will be based on 199395 consumer expenditures.
Although reselecting and reweighting the items in the CPI sample is fundamental to the revision process, each historical revision has brought important methodological innovations that improved the accuracy and representativeness of the index. The 1940 revision, for example, in addition to updating expenditure weights because peoples buying patterns had changed substantially since the 191719 period, introduced the concept of a sample of cities and items as well as the principle that the priced items could be used to impute the price movement of similar non-priced items. Based on this, the Bureau was able to publish price indexes for categories of items, rather than for separate distinguished items.
During World War II, when many commodities were scarce and goods were rationed, the index weights were adjusted temporarily to reflect those shortages. The first comprehensive postwar revision was completed in January 1953, using weights from an expenditure survey conducted in 1950. Both medium-sized and small cities were added to the city sample to make the index representative of prices paid by all urban wage earner and clerical worker families.10
After the 1953 revision, it became apparent that the CPI needed to be revised every decade.11 As a result, the next revision was completed in 1964. For the first time, the expenditure weights included single persons as well as families. In addition, computer processing was used for the first time in the 1964 revision.12
The 1978 CPI revision implemented a variety of fundamental changes, many of which were stimulated by the 1961 report of the Price Statistics Review Committee headed by future Nobel Prize winner George J. Stigler.13 The revision introduced a quarterly survey approach to the collection of consumer expenditures as well as numerous improvements and innovations in pricing for the CPI. Additionally, 1978 saw the introduction of a second index of consumer prices, a more broadly-based CPI for All Urban Consumers, or CPI-U. This index differed from the CPI for Urban Wage Earners and Clerical Workers (CPI-W) by including the buying patterns of all urban households regardless of the consumer units occupational status.
The sampling process of the CPI also moved to a more comprehensive and sophisticated statistical basis in 1978. A new store-specific approach to the item selection process, called disaggregation, was introduced. Perhaps the most significant innovation in the 1978 revision was the introduction of the Point-of-Purchase Survey (POPS). In the POPS, consumer units were interviewed in each geographic area covered in the CPI. Respondents identified the actual stores and other retail outlets in which they shopped and specified the amount they spent for a category of items.
The most recent revision was completed in 1987 and updated the CPI to 198284 expenditure patterns. Unlike previous revisions in which all new areas and items were replaced at the same time, the 1987 revision introduced the concept of "rolling-in." New areas, items, and outlet samples were gradually introduced over a period of years using existing updating procedures.14 The use of systematic sample rotation was also broadened.15 Finally, the 1987 revision introduced the use of an advanced model for CPI sample allocation. The model uses nonlinear programming techniques to determine the distributions of item and outlet selections across all item and geographic strata and to minimize overall price change sampling variance subject to certain budgetary and operational constraints.
The most basic aspect of this revision will be the incorporation of a new set of expenditure weights. Consumer Expenditure Survey data from 199395 will be used to calculate a new expenditure weight for each item strata category in every CPI index area. These new market basketsnew geographic area samples, new item structure, and new expenditure weightswill take effect with the index for January 1998. At the same time, many of the samples underlying the CPI will be replaced. These samples include geographic areas, items selected for pricing, and outlets in which items are priced.
Geographic sample.The geographic sample selection process uses stratified sampling to represent the U.S. urban population. Eighty-seven geographic areas, known as primary sampling units, were selected for the revised CPI based on the 1990 decennial census. These units replace the current primary sampling units, which are representative of the 1980 U.S. population distribution. Thirty primary sampling units where prices are currently collected, including Anchorage and Honolulu, were selected with certainty. Another 21 units in the current design were also selected again. Thirty-six new primary sampling units were selected to replace units in the current CPI geographic sample. These 36 new areas will have new outlet and item pricing samples introduced into the revised index for January 1998. As part of the reselection process, the present three metropolitan-area size categories will be consolidated into two. Details on the redesigned CPI geographic sample and the procedures used to select that sample are in "The redesign of the CPI geographic sample."
Item structure.As the geographic sample
must reflect todays population, the CPI market basket needs
to be updated to represent current consumption patterns. The item
strata (the groupings of items into homogeneous categories for
publication) will be redefined and the way in which they are
aggregated will change based on shifts taking place in the
marketplace. For example, the present seven major groups of goods
and services will be restructured into eight major groups with
the creation of the education and communication
group. That new group includes components previously included in
the housing and other goods and
servicesgroups. The new item structure, along with a
summary of the assumptions underlying the development of the new
structure, are in "Changing the item
structure of the Consumer Price Index."
The selection of commodities and services samples during the revision will benefit from the results of an expanded and re-estimated CPI sample optimization model. As compared with the models used in earlier years, the new design will lead to the selection of relatively fewer outlets and more items per outlet. Also, there will be a broad shift in relative sample size away from the food and beverages category and toward the other major groups. These changes occur largely because updated and improved variance estimates were used in the estimation of the optimization model, and additional sample share constraints were imposed.16
Publication strategy.The selection of new
geographic areas and modernization of the item structure will
necessarily result in changes in the number and mix of published
CPI series. In addition, during the revision, the publication
criteria and procedures have been reviewed extensively. BLS will
develop a variance-based publication strategy in which the level
of detail published is based on the variance surrounding its item
As in the past, the Bureau will publish overlap indexes based on both the new and the old item structure and expenditure weight for several months beginning in February 1998. These overlap indexes will permit users to see first-hand the revisions effect on the published rate of inflation. Also, effective with the index for January 1999, the CPI will change from a 198284 = 100 to a 199395 = 100 reference base. Details on the new CPI publication system and structure are in "Publication strategy for the 1998 revised Consumer Price Index."
Housing.The housing portion of the CPI
revision has two fundamental components, both of which will be
implemented with the index for January 1999:the shift
to an improved estimation method for homeowner shelter costs, and
the selection of a new housing unit sample using the 1990
decennial census. The new estimator for owners equivalent
rent will be based on the reweighting of the same rental
observations that are used for the residential rent index. In the
current CPI estimator, implicit rents for a sample of
owner-occupied units are estimated by matching those units to
specific rental units. The new method will not require selection
of an owner-occupied sample.
This new sample will provide a current set of rental housing units that, as noted above, will be priced to calculate indexes for both residential rent and owners equivalent rent. The decennial census provides information that will be used to select the housing segments within geographic areas. This selection process utilizes stratification criteria to locate segments that represent the housing stock throughout the geographic area, and also determines the sampling rates that will be used to derive the appropriate total sample of housing units. Housing units will be selected to support the initial revised sample, as well as the shadow samples that will be rotated into the CPI in later years to reduce respondent burden and refresh the sample. In addition, new housing units constructed since the decennial census will be brought into the CPI housing sample through an augmentation process. Additional information on the changes to the housing component of the CPI is in "Revision of the CPI housing sample and estimators."
Hospital services.The medical care component of the CPI also will be extensively revised to reflect the dramatic changes taking place in this sector of the economy, particularly in the delivery of hospital services. Effective with the index for January 1997, the three item strata within the hospital and related services category will be consolidated into two, with only a single hospital services stratum. Pricing procedures are also being fundamentally revised, from pricing individual items (such as a unit of blood or a hospital inpatient day) to pricing the combined sets of goods and services provided on selected patient bills. These changes support the collection of transaction prices and facilitate the measurement of price change in an area where the products and services available are undergoing significant change. A full description of these revisions is in "Revision of the CPI hospital services component."
Other enhancements.Several technological
enhancements are planned for the CPI revision as well. We will
modernize the processing systems for both the Consumer
Expenditure Survey and the housing sample of rental prices. Each
of these complex computer processing systems has reached the end
of its current life-cycle, and will be modernized to stay current
with technology. This revision will also build on the
Point-of-Purchase Survey and significantly improve it by
utilizing computer-assisted telephone data collection.
Utilization of a shorter interview and edits at the point of
collection should lead to faster data turnaround and consequently
more up-to-date information on the distribution of consumer
spending across outlets. Based on the improved survey
methodology, BLS can introduce new outlets and items into the CPI
more rapidly and efficiently. Moreover, the new Point-of-Purchase
Survey sample design will permit a shift to sample rotation by
category rather than by geographic area, and thereby facilitate
accelerated sample rotation in product areas where markets are
most dynamic. The article, "New
methodology for selecting CPI outlet samples," describes
this new methodology and its implementation.
A final, crucial element in the current revision is the conversion to electronic systems of all commodities-and-services and housing data collection and transmission. A fully-implemented computer-assisted data collection (CADC) system for the CPI will significantly improve the overall quality of CPI data. While quality improvement is the primary justification for CADC, electronic data collection and transmission will probably provide long-term savings through a reduction of mail, paper and printing costs, and a transfer of some edit and review functions from the BLS national office to the local data collector.
CADC instruments will help to assure consistent application of survey rules and data review decisions at the point of collection. The results will be increased accuracy of collected data, greater conformance with collection procedures, a higher percentage of successfully completed collection schedules, and an increase in the yield of usable quotes per release of the index. Electronic data transmission also will allow collection of prices closer to the end of the month, improving coverage of price changes during the reference period.
The initial CADC system implementation, Version I, will be a fully reliable system that meets current data collection standards and will be a foundation upon which future enhancements can be built with confidence. This version will include all functionality necessary for housing. There will be basic edits and checks for collected data. In addition, a survey information and control system will be developed that will provide up-to-date information on the status and location of all housing data collection activities. Implementation of Version I will begin in 1997 and will be completed in 1998.
Strategically, we will build on our computer data collection experience with the housing data before undertaking the pricing and initiation of commodities and services in retail establishments. In Version II, there will be more functionality for initiation, and edits and checks will be expanded. The survey information and control system will likewise expand to control the larger volume of data collected for commodities and services. After Version II is operating smoothly, a part of the commodity analyst review will be incorporated into the CADC system at the point of collection. This will provide automatic checks for the comparability of item substitutions and transfer more functions from the national office staff to the regional staff.
THE MOST SIGNIFICANT MILESTONES in the overall revision schedule are presented in exhibit 3. The conclusion of the revision in 2000 does not mean the end of the CPI improvement process. BLS will continue to enhance its program of experimental indexes and research on CPI measurement issues. It is also reasonable to expect that the recommendations of the Senate Finance Committees Advisory Commission, as well as the results of other studies of the CPI by outside researchers, will lead to additions in the list of CPI improvements outside the revision context. Nevertheless, the decennial revision process remains the central method by which the BLS maintains the quality of the CPI and thereby ensures the indexs status as the most accurate and timely measure of changes in consumer prices.
|Exhibit 1. Improvements to the Consumer Price Index|
|Rent samples augmented with units built after 1960.|
|Quality adjustment of new automobile prices||
|New automobile prices adjusted for quality differences after model changeovers.|
|Introduced a systematic replacement of outlets between major revisions.|
|Changed homeowners' component from cost of purchase to value of rental services for CPI-U.|
|Return from sale price imputation||
|Introduced procedure to eliminate downward bias for items discontinued by outlets that went out of index with discounted prices.|
|Changed CPI-W homeowners' component to value of services.|
|Enhanced seasonal products methodology||
|Enhanced methodology used for seasonal items by expanding the number of price quotations to select products from alternate seasons and eliminate under-representation of such items.|
|Quality adjustment of used car prices||
|Prices of used cars adjusted for differences in quality after model changeovers.|
|Aging bias correction||
|Rental values adjusted for aging of the housing stock.|
|Imputation procedures for new cars and trucks||
|Price changes for noncomparable new models are imputed using only the constant-quality price changes for comparable model changeovers.|
|Quality adjustment of apparel prices||
|Regression models used to adjust apparel prices for changes in quality when new clothing lines are introduced, and eliminate bias due to linking product substitutions into the CPI.|
|Discount air fares||
|Substitution rules modified to expand pricing of discount airline fares.|
|Increase in the number of outlets from which prices are collected to replace sample lost through sample attrition.|
|New models imputation||
|Refined imputation methods used when introducing products into the CPI.|
|Hotels & motels||
|Samples for hotels and motels quadrupled to reduce variances related to seasonal pricing.|
|Procedures for seasonal adjustment revised to eliminate residual seasonality effects.|
|Quality adjustment for gasoline||
|Treat "reformulated" gasoline as a quality change and adjust the price to reflect quality difference. Impact of the change estimated.|
|Introduced new procedures that allow generic drugs to be priced when a brand drug loses its patent.|
|Food at home base period prices||
|Introduced seasoning procedures to eliminate upward bias in setting of base period prices of newly initiated items.|
|Modified imputation of homeowners' implicit rent to eliminate the upward drift property of the current estimator.|
|Composite estimator used in housing||
|Replaced current composite estimator with a 6-month chain estimator. Under-reporting of 1-month rent changes had resulted in missing price change in residential rent and homeowners' equivalent rent. Old estimator also produced higher variances.|
|Commodities and services base period prices||
|Extended food-at-home seasoning procedures to remainder of commodities and services series. Base period prices left unchanged in most noncomparable substitutions.|
|Exhibit 2. Previous CPI revisions|
|Release of revised CPI||Expenditure base period||Notable innovations|
|1940||1934-36||Introduced the concept of a sample of cities and items, and the principle of imputation|
|1953||1950||Expanded population coverage to represent all urban wage earner and clerical worker families|
|1964||1960-61||Expanded population coverage to represent individuals as well as families; introduced computer processing|
|1978||1972-73||Expanded population coverage to represent all urban consumers; improved methodology for construction of outlet sample frame; introduced probability sampling techniques into the selection of the item and outlet samples|
|1987||1982-84||Expanded scope of systematic outlet rotation; introduced advanced sample allocation model|
|Exhibit 3. Major milestones in the 1998 CPI revision|
|Introduce revised hospital services item structure and sample||February 1997 (index for January 1997)|
|Introduce new geographic sample and item structure and update expenditure weights to 1993-95||February 1998 (index for January 1998)|
|Begin pricing of new housing sample using computer-assisted data collection||June 1998|
|Introduce new housing sample and estimator into CPI||February 1999 (index for January 1999)|
|Rebase CPI to 199395 =100||February 1999 (index for January 1999)|
|Introduce computer-assisted data collection for commodities and services sample||Summer 1999|
|Begin shift from area sample rotation to item category rotation using telephone Point-of-Purchase Survey||Early 1999|
|Enter redesigned Consumer Expenditure Survey processing system into full production||October 1999|
The authors are grateful to Marybeth Tschetter for her comments, suggestions and assistance during the preparation of this paper and the others in this volume. Sylvia Leaver, Frank Ptacek, and Janet Williams also provided helpful comments on the paper. Finally, the authors would like to acknowledge the contributions of John Marcoot, who played a fundamental role in the design and execution of the 1998 CPI Revision as well as earlier revisions.
1Office of Management and Budget estimate, as reported in BLS budget documents.
2Handbook of Methods, Bulletin 2285 (Bureau of Labor Statistics, 1988) "Chapter 19. The Consumer Price Index."
3For a description of the construction of, and recent movements in, the CPI-E, see Kenneth Stewart and Joseph Pavalone, "Experimental CPI for Americans 62 years of age and older," CPI Detailed Report, April 1996.
4Theoretical and empirical comparisons of the geometric mean formula to the arithmetic mean formula used in the CPI are presented in Brent Moulton and Karin Smedley, "A Comparison of Estimators for Elementary Aggregates of the CPI," Bureau of Labor Statistics Working Paper, June 1995.
5For general reviews of these issues see, for example, Dennis Fixler, "The Consumer Price Index: underlying concepts and caveats," Monthly Labor Review, December 1993, pp. 312; Is the Growth of the CPI a Biased Measure of Changes in the Cost of Living? (Congressional Budget Office, October 1994); Report for the House Budget Committee (Bureau of Labor Statistics, 1995); Matthew Shapiro and David Wilcox, "Mismeasurement in the Consumer Price Index:An Evaluation," NBER Macroeconomics Annual 1996 (Cambridge, MA, National Bureau of Economic Research, forthcoming); and Brent R. Moulton, "Bias in the Consumer Price Index: What is the Evidence?" Journal of Economic Perspectives, Fall 1996, pp. 15977.
6See Consumer Price Index, Hearings before the Committee on Finance, U.S. Senate, U.S. Government Printing Office, 1995; and Toward a More Accurate Measure of the Cost of Living, Final Report to the Senate Finance Committee from the Advisory Commission to Study the Consumer Price Index, December 4, 1996.
7For details on these changes, see Steve Henderson and Karin Smedley, "Improvements in estimating the shelter indexes in the CPI," CPI Detailed Report, October 1994, pp. 56; Ken Stewart, "Improving CPI sample rotation procedures," CPI Detailed Report, October 1994, pp. 78; Ken Stewart, "Extending the improvement in CPI sample rotation procedures," CPI Detailed Report, June 1996, pp. 910; and Kenneth J. Stewart, "Improving CPI item substitution procedures," CPI Detailed Report, July 1996, pp. 89.
8See, for example, Ana Aizcorbe and Patrick Jackman, "The commodity substitution effect in CPI data, 198291," Monthly Labor Review, December 1993, pp. 2533.
9The Consumer Price Index:History and Techniques, Bulletin 1517 (Bureau of Labor Statistics), p. 2; and The First Hundred Years of the Bureau of Labor Statistics (Bureau of Labor Statistics, September 1985) pp. 9497.
10Handbook of Methods, Bulletin 2285 (Bureau of Labor Statistics, 1992), p. 154.
11Sidney Jaffe, "The Consumer Price IndexTechnical Questions and Practical Answers," Paper presented to the American Statistical Association, Dec. 30, 1959.
12John Marcoot, "Revision of Consumer Price Index is now underway," Monthly Labor Review, April 1985, p. 28.
13Government Price Statistics, U.S. Congress, Joint Economic Committee, 87th Cong., 1st Sess., January 24, 1961.
14The rolling-in approach effected significant cost and time savings. Rolling-in the new areas into the CPI allowed more time to train field representatives. More importantly, using the existing procedures for introducing new outlet samples on a systematic basis precluded the need to maintain dual operations, one for the current CPI and one for the revised index.
15Marcoot, "Revision of Consumer Price Index."
16S.G. Leaver, W. H. Johnson, R.M. Baskin, S. Scarlett, and R. Morse, "Commodities and Services Sample Redesign for the 1998 Consumer Price Index Revision," Proceedings of the Survey Research Methods Section, American Statistical Association, forthcoming.
John S. Greenlees is Assistant Commissioner for Consumer Prices and Price Indexes and Charles C. Mason is an economist in the Division of Consumer Prices and Price Indexes, Bureau of Labor Statistics.
Last Modified Date: October 16, 2001