U.S. Bureau of Labor Statistics U.S. Bureau of Labor StatisticsThe Bureau of Labor Statistics (BLS) began providing discussion of the employment outlook soon after the end of World War II in order to offer career information to veterans reentering the civilian workforce. The first set of formal numerical projections, however, was published in 1960. Since then, BLS has developed long-term projections of likely employment patterns in the U.S. economy. Projections cover the future size and composition of the labor force, aggregate economic growth, detailed estimates of industry production, and industry and occupational employment. The resulting data serve a variety of users who need information about expected patterns of economic growth and the effects these patterns are expected to have on employment. Data users include individuals seeking career guidance and organizations and individuals that offer career guidance resources. In addition, policymakers, community planners, and educational authorities, who need information for long-term policy planning purposes, make use of BLS employment projections. BLS projections also are used by States in preparing State and local area projections.
Since the early 1970s, projections have been prepared on a 2-year cycle. Until 1997, BLS developed projections in which the target year always ended in a zero or a five. Projections were prepared every other year, resulting in at least two—and sometimes three—sets of projections being prepared for the same target year. As a result, projection horizons were as short as 10 years—or as long as 15 years. Beginning with the 1996–2006 projections, which were published in 1997, BLS began developing projections for a 10-year period, still on a 2-year cycle.
Over the years, the projections used to develop the employment projections have undergone many changes, as new data series became available and as economic and statistical tools improved. Since the late 1970s, however, the basic methodology has remained largely the same. Procedures have centered around projections of an interindustry or input-output model that determines job requirements associated with production needs, and the National Employment Matrix, which depicts the distribution of employment by industry and occupation. Projecting employment in industry and occupational detail requires projections of the total economy and its sectors. BLS develops its projections in a series of six steps, each of which is based on separate procedures and models, and on related assumptions. These six steps examine:
These components provide the analytical framework needed to develop detailed employment projections. BLS analysts solve each component sequentially. Each step includes several iterations to ensure internal consistency as assumptions and results are reviewed and revised.
BLS projects the future supply of labor by applying labor force participation rate projections made by BLS to population projections produced by the Census Bureau. The Census Bureau carries out long-term population projections of the resident U.S. population. The projection of the resident population is based on the current size and composition of the population and includes assumptions about future fertility, mortality, and net international migration. The conversion from the resident population concept of the Census to the civilian noninstitutional population concept of the BLS Current Population Survey (CPS) takes place in three steps. First, the population of children under 16 years is subtracted from the total resident population. Then, the population of the Armed Forces, by age, gender, race, and ethnic categories, is also subtracted. Finally, the institutional population is subtracted from the civilian population for all the different categories.
For 136 age, gender, race and ethnic groups, BLS maintains a data base of annual averages of CPS labor force participation rates. BLS analysts examine trends and past behavior of participation rates for all categories. First, the historical participation rates for these groups are smoothed. Second, the smoothed data are transformed into logits, or natural log of the odds ratio.1 Finally, the logits of the participation rates are extrapolated linearly by regressing against time and then extending the fitted series to or beyond the target year. When the series are transformed back into participation rates, the projected path is nonlinear.
In addition, projected labor force participation rates are reviewed for consistency. The time-path, cross-section in the target year, and cohort patterns of participation are all reviewed and, if necessary, modified. Projected labor force participation rates are applied to the population projections, producing labor force projections for each of the different age, gender, race, and ethnicity groups. Groups then are summed to obtain the total civilian labor force, which becomes an input to the next stage of the projections.
The second stage of the BLS projections process develops projections of the gross domestic product (GDP) and the major categories of demand and income. The results of this stage provide aggregate measures that are consistent with each other and with the various assumptions and conditions of the projections. Values generated for each demand sector are then used in the next stage: developing detailed commodity purchases for personal consumption, business investment, foreign trade, and government.
Recent projections are based on a macroeconomic model developed by St. Louis-based Macroeconomic Advisers, LLC (MA). This model has 744 variables that describe the U.S. economy. The 744 variables can be divided into: 134 behavioral equations, 409 identities, and 201 exogenous variables, including key assumptions, such as monetary policy, fiscal policy, energy prices and supply, world economic growth, and demographic changes. Additionally, BLS takes a long-term view by assuming a long-run full-employment economy where unemployment is frictional and not a consequence of deficient demand.
Besides being governed by general assumptions, usually projections are approached with specific goals or targets. Goals used to assess the behavior of a given set of projections include rate of growth and demand composition of real GDP, rate of growth of labor productivity, rate of inflation, and the unemployment rate. Many solution rounds may be necessary to arrive at a balanced set of assumptions, which yield a defensible set of results. When the aggregate economic projection is final, the components of GDP are then supplied to the commodity component of the projections process.
The macroeconomic model provides forecasts of final demand sectors, including personal consumption expenditures, gross private fixed investment, change in private inventories, exports and imports of goods and services, Federal and State and local governments, and more-detailed sectors within many of these categories. The next step in the projections process is to further disaggregate the results from the macro model into detailed categories and then into the types of commodities purchased by each of the categories. The sectoring plan of categories and commodities is chosen to be as detailed as possible but such that the data are supported by both the National Income and Product Accounts2 and the Input-Output Accounts3, both published by the Bureau of Economic Analysis (BEA).
Personal consumption expenditures (PCE) are projected in the MA model at an aggregate level. The Houthakker Taylor model4 is used to project consumption expenditures for 88 detailed national income and product account categories over the projection period. These 88 category estimates then are chain weighted,5 to the level of total PCE from the macro model and adjusted as necessary to insure consistency between aggregate PCE and the detailed estimates. A bridge table based on the most recent Benchmark Input-Output Accounts then is used to distribute consumption spending for each of the 88 categories among roughly 200 producing industries for the projection period.
Gross private domestic investment is initially projected by the MA model for private investment in equipment and software (PIES), nonresidential and residential structures, and business inventories. The PIES categories are estimated in greater detail using a system of regression equations that set GDP, capital stock, and the cost of capital as explanatory variables. In all, projections are made for 28 categories of private investment in equipment and software. Estimates then are aggregated to the level of the macroeconomic model control and adjusted as necessary, to insure consistency between the macroeconomic model aggregate and the detailed estimates. Business inventories, on a commodity basis, are extrapolated, based on lagged values of commodity output. These, also, are aggregated and adjusted to conform to the macroeconomic model aggregate of the change in inventories. The controls for nonresidential and residential structures are taken directly from the macro model.
Foreign trade is initially projected by the MA macro model for export goods and services, and import goods and services. Adjustments are made to the initial MA model forecast to account for re-exports and re-imports and move the data from a NIPA based estimate to an I-O basis. Distributional models are used to allocate the adjusted forecasted macro model data to a commodity basis. Other factors also are considered, including energy forecasts, existing and expected shares of the domestic market, expected world economic conditions, and known trade agreements.
Government demand is projected by the MA model for three major government categories: Federal defense, Federal nondefense, and State and local government. Projections for each major category include estimates for two expenditure categories: consumption and gross investment. These are further disaggregated based upon past trends and expected government political and policy changes. Finally, each of the six expenditure categories is allocated to detailed commodity sectors, such as electric utilities or hospitals, based on relationships from BEA's benchmark I-O accounts.
Components of final demand, personal consumption expenditures, gross private domestic investment, foreign trade, and government, are then compiled into a final demand matrix. The process yields a matrix comprising about 200 rows of commodity sectors and 200 columns of final demand categories.6 The resulting detailed distribution of GDP provides the demand component of an interindustry model of the U.S. economy.
The creation of an input-output model is the next stage in developing BLS projections.7 Each industry within the economy relies on other industries to supply inputs—intermediate products or services—for further processing. By definition, GDP reflects only sales to final purchasers, such as car buyers. Intermediate material inputs, such as the steel incorporated into cars, are not explicitly reflected in the GDP estimates. An input-output model provides a means to derive an industry-level estimate of the output and employment needed to produce a given level of GDP.
Input-output tables are the area of greatest difficulty in data development. The United States, like many other countries, does not produce detailed annual input-output tables; its benchmark tables are released every 5 years and analyze the survey period from about 5 years earlier. To develop a historical detailed set of I-O tables, BLS compiles a time series of industry and commodity output estimates. The series are based on a variety of primary source data, such as annual surveys of manufacturers, of wholesale trade, and of service, as well as other annual and monthly surveys conducted by the Census Bureau. Additional data sources include estimates of revenue and various output that come from the Energy Information Agency, the Department of Agriculture, and the Internal Revenue Service. BLS assembles these data sources into a set of industry output measures that are consistent, both over time and with the benchmark input-output table.
The BLS input-output model consists of two basic matrices for each year, a "use" table and a "make" table. Both tables are expressed in coefficient form. The "use" table, the principal one, shows the use of commodities by each industry as inputs into its production process. In coefficient form, each column of this table shows the pattern of commodity inputs per dollar of industry output. The "make" table shows the commodity output of each industry. The table allocates commodity output to the industry in which it is the primary commodity output and to those industries where it is secondary. This table shows the industry distribution of production for each commodity.
Generally, BLS updates the latest benchmark input-output tables (currently 2002) to other years by taking the estimates for the total commodity and industry output and balancing the "make" table, allowing some flexibility in the commodity output estimate. The industry and commodity output totals from the "make" table and the estimated final demand matrices then are used to overwrite portions of the benchmark "use" table. The rest of the "use" table, intermediate and value added, is then estimated by applying a balancing procedure, commonly referred to as RAS.
Projecting the "use" table must take into account the changes in the input pattern or the way in which goods are produced or services are provided by each industry. In general, two types of changes in these input patterns are made in developing a future input-output table: those made to the inputs of a specific industry (such as the changes in inputs in the publishing industry) and those made to the inputs of a specific commodity in all or most industries (such as increased use of business services across a wide spectrum of industries). Unlike the "use" table, the relationships in the "make" table generally are held constant or changed very little over the projections decade.
When projected values of the "use" and "make" relationships are available, the projection of commodity demand developed in preceding steps is converted into a projection of domestic industry output using the relationships obtained from the U.S. Department of Commerce's Bureau of Economic Analysis (BEA). These relationships are summarized briefly below.
g = D(I - BD)-1e
where,
g = vector of domestic industry output by sector
B = "use" table in coefficient form
D = "make" table in coefficient form
I = identity matrix
e = vector of final demand by commodity sector.
In sum, matrix multiplying the inverse of the coefficient forms of the "make" and "use" tables by a vector of final demand commodity distribution, as represented by "e" above, yields industry outputs.
The detailed industry output from the previous stage then is used to derive the industry employment necessary to produce the given level of output. To arrive at the employment estimate, EPP combines data from two BLS sources: (1) the Current Employment Statistics (CES), an establishment survey, offers data for nonagricultural wage and salary employment, production worker employment, and weekly hours; and (2) the Current Population Survey (CPS), a household survey, provides information regarding agricultural employment, self-employed and unpaid family worker jobs and hours, and private household workers.
The BLS models industry employment as a function of industry output, wages, prices, and time. The Employment Projections Program (EPP) measures total employment as a count of jobs, not a count of individual workers. This concept is different from that used by another BLS measure familiar to many readers: CPS total employment as a count of the number of workers. The EPP total-employment concept also is different from the CES total employment-measure: whereas the CES measure also is a count of jobs, it covers nonfarm payroll jobs only, while the EPP measure includes all jobs. Industry employment then is projected using the estimated historical relationship between the variables. Industry employment is projected in both numbers of jobs and hours worked, both for wage and salary workers and for self-employed and unpaid family workers. Projections are developed according to the following procedure implemented for each industry.
Through a system of equations, employment for wage and salary workers is solved independently over the projections decade for each industry. The individual industry estimates of employment must be consistent with the total employment level derived from the macroeconomic solution. The employment equation relates an industry's labor demand (total hours) to its output, its wage rate relative to its output price, and a trend variable to capture technological change within that industry. A separate set of equations, describingaverage weekly hours for each industry, are estimated as a function of time and the unemployment rate. The equations then are used to predict average weekly hours over the projections decade. An identity relating average weekly hours, total hours, and employment yields a count of wage and salary jobs by industry.
The number of self-employed and unpaid family workers is derived by first extrapolating the ratio of the self-employed to the total employment for each industry. This equation is a function of time and the unemployment rate. The extrapolated ratio is used to derive the level of self-employed and unpaid family workers given the number of wage and salary jobs in each industry. Total hours for self-employed and unpaid family workers are calculated by applying the estimated annual average weekly hours to the employment levels for each industry. Finally, total hours for each industry are derived by summing hours for wage and salary workers and for self-employed and unpaid family worker hours.
Together with the industry output projections, employment results provide a measure of labor productivity. BLS analysts examine the implied growth rates in projected productivity for consistency with historical trends. At the same time, analysts attempt to identify industries that may deviate from past behavior because of changes in technology or other factors. Where appropriate, changes to the employment estimates are made by modifying either the employment demand itself or the results from earlier steps in the projections process.
The final estimates of the projected employment for about 200 industries are then used as inputs to determine the occupational employment over the projections decade.
The technique for developing the occupational employment projections is based on an industry-occupation matrix showing the distribution of employment for nearly 300 industries and for 750 detailed occupations. For all except a few industries, information on the occupational distribution of wage and salary workers by industry (staffing patterns) is derived from the BLS Occupational Employment Statistics (OES) survey. The OES program surveys about 200,000 establishments per panel (every 6 months), taking 3 years to fully collect the sample of 1.2 million establishments. About 800 detailed occupations from the Standard Occupational Classification (SOC) are surveyed across more than 450 industries. The industry classifications correspond to the sector, which are 3, 4, and 5-digit North American Industry Classification System (NAICS) industrial groups. In developing the base-year matrix, occupations that have fewer than 5,000 workers generally are aggregated into similar larger occupations or appropriate residuals. Also, industries with staffing patterns that are comparable to the residual employing fewer than 50,000 workers are aggregated into residuals within the same 3-digit NAICS group.
In some industries, adjustments are made to OES survey staffing patterns, because some occupations are not listed separately in the survey questionnaire but are included in a residual category. To develop economy-wide employment estimates for these occupations, data are disaggregated from OES survey residuals. Data from the decennial census are used for these adjustments.
Adjustments also are made to staffing patterns derived from sources other than the OES survey. For example, the occupational classifications used to group Federal Government workers are more detailed than those used in the matrix. Similarly, estimates of occupational employment for workers who are self-employed, unpaid family workers, and in the private household industry and agriculture, except agricultural services, come from the household-based Current Population Survey and must be adjusted to make them comparable to the occupational classification used in the matrix.
In coefficient terms, the industry-occupation matrix represents industry staffing patterns, where each column represents the occupational distribution of employment in a specific industry. The change in occupational requirements is jointly determined by shifts in these coefficients and by the structure of industry employment developed in the preceding step. Because staffing patterns of industries may change over time, the projection method must account for shifts through a series of steps. First, historical data are reviewed to identify trends. Next, factors underlying these trends are then identified through analytical studies of specific industries and occupations, technological change, and a variety of other economic data. Finally, projected staffing patterns are produced, based on judgments of how the pattern is expected to change in the future. Numerous factors underlying these changes include technological developments affecting production and products, innovations in the ways business is conducted, modifications of organizational patterns, responses to government policies, and decisions to add new products and services or to stop offering existing ones.
Some expected trends may not be evident in the historical data. For example, an analysis of the past would not point toward the impact of radio frequency identification (RFID) on staffing patterns for cashiers because this technology has not been used widely in most industries. However, as more stores track their inventory and purchases through RFID, this technology may have a significant impact on cashiers, especially in industries in which RFID is easiest to implement.
The projected change in a specific occupation's share of industry employment may be small, moderate, or large; the magnitude of the percentage change reflects the judgment of the BLS analyst who studies that occupation. Documentation8 is released with the projections that details the analysis of occupations in which changes to the base-year coefficients were made.
When projected staffing patterns are available, they are used to allocate each industry's projected employment to detailed occupations. These estimates then can be summed across industries to yield total employment for each detailed occupation as follows:
o = Sl
where,
o = vector of wage and salary employment by occupation
l = vector of wage and salary employment by industry
S = staffing pattern matrix in which each column contains the allocation of industry employment to occupations in percent terms.
Estimates described above relate only to wage and salary employees. Other classes of workers, primarily the self-employed, are analyzed separately. The analyses of these other workers then are combined with that of wage and salary workers to produce a projection of total occupational demand for the United States. Increases in occupational employment create job openings due to economic growth. If employment is projected to decline, that occupation has no openings due to growth.
In addition to projecting employment change by occupation, BLS projects replacement needs. The replacement needs are combined with economic growth openings to derive total job openings over the projection decade. To calculate job openings due to replacement needs, BLS analyzes historical data from the CPS on occupational employment and calculates replacement rates by age group. These historical rates are applied to occupational age-distribution data in the base year to estimate replacement needs for the future. The projected replacement needs assume that workers will continue to retire and otherwise exit an occupation at ages similar to those which have been observed in the recent past. The result is occupation-specific replacement needs that capture the impact of demographic, but not behavioral, changes. For a full discussion of how replacement needs are estimated, see the technical documentation9 on the Employment Projections Program Web site.
BLS also provides information about education or training requirements for each of the occupations for which the office publishes projections data.10 This allows occupations to be grouped in order to create estimates of the education or training needs for the labor force as a whole and estimates of the outlook for occupations with various types of education or training needs. In addition, educational attainment data for each occupation are presented to show the level of education achieved by current workers. Education and training categories include: First professional degree, doctoral degree, master's degree, bachelor-or-higher degree plus work experience, bachelor's degree, associate degree, postsecondary vocational award, work experience in a related occupation, long-term on-the-job training, moderate-term on-the-job training, and short-term on-the-job training. Definitions for each of the 11 categories can be found in the Occupational Variable Data Definitions technical document.11
An important element of the projection system is its comprehensive structure. To ensure internal consistency and reasonableness of this large structure, the BLS projections process encompasses detailed review and analysis of the results at each stage. For example, the close relationship between changes in staffing patterns in the occupational model to changes in technology is an important factor in determining industry labor productivity. Specialists in many different areas from inside and outside the projection group review all of the relevant results from their particular perspective. In short, final results reflect innumerable interactions among BLS analysts, who focus on particular sectors in the model. Through this review, the projection process at BLS converges into an internally consistent set of employment projections across all industries and occupations.
BLS employment projections are developed with a number of underlying assumptions, both explicit and implicit. Projections are developed using statistical and econometric models combined with subjective analysis. All analytical projections implicitly assume that relationships exhibited in the past will continue to hold over the projected period. Statistical and econometric models formally project historical relationships on a mathematical basis. Subjective analysis projects current and historical behavior into the future based on analogous past experience. The efficacy of the projections relies both on the understanding of history, and the expectation that the past can be extrapolated into the future.
The following assumptions underlie BLS employment projections.
In addition to the assumptions listed above, the component processes of the projections may incorporate specific assumptions or exogenous inputs. For example, the labor force model uses the Census Bureau population projections to derive the labor force level, by applying a projection of labor force participation rates. These assumptions are discussed in the relevant sections of this chapter.
The BLS employment projections should be understood to be a projection and not a forecast. The distinction involves emphasis on purpose and results. Projections focus on longer term underlying trends based on a set of assumptions, while forecasts focus more on predicting the actual outcome in the near term. The assumptions that underlie projections are usually designed to provide a neutral backdrop that allows focused analysis of the long term trends. For example, this means that BLS does not forecast business cycle activity, but rather is concerned with the long term growth path of the aggregate economy. Because the purpose of a forecast is prediction, the forecast user will be interested in the actual forecast values. A projection, however, supplies the user with a plausible scenario in which to understand the ramifications of the long-term trends.
Finally, the unexpected will occur and have unknown influences. There will be unanticipated events, whether changes in technology, war, disaster, human understanding, or social dynamics. In this context, BLS employment projections should be considered as likely outcomes based on specified assumptions, and not definitive outcomes.
Projections are released online and in the Monthly Labor Review and the Occupational Outlook Quarterly. The Monthly Labor Review typically includes an overview article and an article on each of the major projections components: labor force, aggregate economy, industry output and employment, and occupational employment and job openings. The Quarterly publishes articles related to career preparation, such as occupational profiles, jobseeking information, and understanding wage and benefits data. Part of each biennial projection study is the release of the Occupational Outlook Handbook that contains extensive information about hundreds of occupations. In addition to outlook data for each occupation, this publication includes information on the nature of the work, training requirements, working conditions, and earnings. The Handbook is used as a primary source of information for people choosing a career and is available in many career centers of high schools and colleges, as well as in libraries.12The Career Guide to Industries13 is a separate publication detailing the projections of employment by industry. Detailed information is given including the types of goods and services provided, occupations employed, the earnings, and the outlook for each industry.
The BLS projection process does not end at publication. BLS is constantly working to improve the accuracy of projections. To ensure that projections are reliable and of the highest quality, BLS typically evaluates its projections after the target years have passed. Projections of the labor force, industry employment, and occupational employment are evaluated, using metrics that provide measures of accuracy. These metrics were developed from a review of methods used by BLS and other agencies in evaluating projections.
Evaluations benefit BLS and external users. Identifying sources of errors helps BLS improve the models used in developing the employment projections. And publishing the results allows users to gauge the accuracy of statements about future economic conditions, industry activity, and employment growth. The most recent evaluation articles include:
(Note: Projections are biennial and normally appear in the November issue of the Monthly Labor Review in odd-numbered years. New BLS bulletins related to the projections are released shortly thereafter.)
Howard N. Fullerton, Jr. "Evaluating BLS projections to 2000," Monthly Labor Review, October 2003, pp. 3-12.
Andrew Alpert and Jill Auyer "Evaluating the BLS 1988–2000 employment projections," Monthly Labor Review, October 2003, pp 13-37.
H.O. Stekler and Rupin Thomas, "Evaluating BLS labor force, employment, and occupation projections for 2000," Monthly Labor Review, July 2005, pp. 46-56.
Roger J. Moncarz, Michael G. Wolf, and Benjamin Wright, "Service-providing occupations, offshoring, and the labor market ," Monthly Labor Review, December 2009, pp. 71-86.
Information about the Census Bureau's U.S. Population Projections is on the Internet at www.census.gov/population/www/projections/index.html.
Mitra Toossi, "A new look at long-term labor force projections to 2050 ," Monthly Labor Review, November 2006, pp. 19-39.
U.S. Department of Labor, Bureau of Labor Statistics. "Charting the projections," Occupational Outlook Quarterly, Winter 2009-10.
U.S. Department of Labor, Bureau of Labor Statistics. Occupational Outlook Handbook, BLS Bulletin 2800, January 2010.
U.S. Department of Labor, Bureau of Labor Statistics. Career Guide to Industries.
U.S. Department of Labor, Bureau of Labor Statistics. Monthly Labor Review, November 2009 (issue devoted to the 2008–18 projections).
[1] For more information on labor force methodology, see Paul F. Velleman, "Definition and Comparison of Robust Nonlinear Data Smoothing Algorithms," Journal of the American Statistical Association, September 1980, Volume 75, Number 372, Theory and Methods Section, pp. 609-615.
[2] For more detailed discussion on concepts and methods of the U.S. National Income and Product Accounts, see the publication on the Internet at www.bea.gov/national/pdf/NIPAhandbookch1-4.pdf.
[3] For more detailed discussion on concepts and methods of the U.S. Input-Output Accounts, see the publication on the Internet at www.bea.gov/papers/pdf/IOmanual_092906.pdf.
[4] Houthakker, H. S., and Lester D. Taylor, "Consumer Demand in the United States: Analyses and Projections," 1970 Harvard University Press, Cambridge, MA.
[5] The U.S. National Income and Product Accounts have adopted a chain-weighted Fisher Index to calculate real aggregates. Because BLS data is based on the Bureau of Economic Analysis NIPA and I-O accounts, real projections data is also measured in chain-weighted dollars. Due to the mathematical properties of chain weighting, for a particular year, details do not necessarily add to their higher level aggregates.
[6] Categories may vary from one projection study to the next, depending on data availability.
[7] For detailed information regarding I-O Analysis, see Ronald E. Miller and Peter D. Blair, Input-Output Analysis: Foundations and Extensions (Prentice-Hall, Inc., Englewood Cliffs, NJ, 1985), 276-294.
[8] www.bls.gov/emp/ep_table_502.htm.
[9] www.bls.gov/emp/ep_replacements.htm
[10] For a more-detailed discussion of the education and training categories, see the Measures of Education and Training page on the BLS Web site at www.bls.gov/emp/ep_education_tech.htm.
[11] www.bls.gov/emp/ep_nem_definitions.htm#education
[12] With the publication of previous projections, BLS also released the Occupational Projections and Training Data bulletin, which presented detailed statistics on employment, occupational separations, and training classification systems. Although this bulletin is no longer available in print format, all information formerly included in the bulletin is available on the employment projections Web site. For further information, "see" www.bls.gov/emp/optd/.
[14] www.bls.gov/opub/mlr/2003/10/art2full.pdf