U.S. Bureau of Labor Statistics

State and Metro Area Employment, Hours, & Earnings

Reliability of State and Area estimates

The Current Employment Statistics (CES) survey, like other sample surveys, is subject to two types of error—sampling and nonsampling. The magnitude of sampling error, or variance, is directly related to the size of the sample and the percentage of universe coverage achieved by the sample. Measurements of error associated with sample estimates for Statewide industries are provided in the below table.

Estimates for states and areas are produced using two methods. The majority of state and area estimates are produced using direct sample based estimation. However, published area and industry combinations (domains) that do not have a large enough sample to support estimation using only sample responses have been estimated using a small domain model.

Variance of the CES estimates. The sampling errors provided in this technical note have been estimated using the Repeatedly Grouped Balanced Half Samples (RGBHS) method, a modification of the Balanced Half Samples (BHS) technique used for the National estimates (see the Variance Estimation section at www.bls.gov/web/empsit/cestn2.htm). This modified technique involves the repeated calculation of BHS estimates R times. After each calculation, the sample units are randomly assigned to two groups in order to produce new BHS estimates. The final estimate of variance is an average of R BHS estimates. This method is used in calculating the sample errors for each published basic and summary cell. Under normal circumstances, where the average response rates are met and there are no reporting anomalies, the sampling errors represent a good indication of the magnitude of error in the estimates from sampling only. They do not represent components of error in the estimates that are due to reasons other than sampling; nonsampling error is captured in the benchmark revision.

The CES State and Area benchmark revision. To help control both sampling and nonsampling error, CES estimates are benchmarked annually to universe employment counts. These counts are derived primarily from employment data reported on unemployment insurance (UI) tax reports that nearly all employers are required to file with state employment security agencies. Benchmark levels replace the original sample-based estimates usually from April of the previous benchmark year to September of the current benchmark year for each month.

Once the new September level for the current benchmark level is made, the appropriate sample links are applied to the new level, and the estimates are recalculated from October forward. These sample links capture the over-the-month change of sample estimates, and are calculated by dividing employment reported by survey respondents for that month by employment reported by those same respondents for the previous month. The links used during the benchmark process may differ slightly from those used to derive original estimates, because they include data from respondents who reported too late for inclusion in the previously published estimates.

Appropriate uses of sampling variances in CES. Variance statistics are useful for comparison purposes, but they do have some limitations. Variances reflect the error component of the estimates that is due to surveying only a subset of the population, rather than conducting a complete count of the entire population. However, they do not reflect nonsampling errors, such as reporting errors, bias due to nonresponse, or the model error for series denoted as model-based. The overall performance of the all-employee estimates is still measured in terms of the benchmark revisions. Benchmark revisions are discussed in the last section of this technical note. For other data types published by the CES program, average hourly earnings and average weekly hours, variances are the best measure of accuracy. The variances of the over-the-month change estimates are very useful in determining when changes are significant at some level of confidence.

Illustration of the use of table 1. Table 1 provides a reference for standard errors of CES State and area employment estimates. The standard errors of differences between estimates in 2 non-overlapping industries are calculated as

The errors are presented as standard errors.

Table 1 provides a reference for the standard errors of 1-, 3-, and 12-month changes in employment. The errors are presented as standard errors of the changes.

Example:

Suppose that the over-the-month change in manufacturing employment from January to February in a given state is 1,500. The table indicates that the standard error for a 1-month change in employment for manufacturing is 1,200. The interval estimate of the over-the-month change in employment that will include the true over-the-month change with 90-percent confidence is calculated:

1,500 +/- 1.645(1,200) = 1,500 +/- 1,974 = [-474, 3,474]

The true value of the over-the-month change is in the interval -474 to 3,474. Because this interval includes 0 (no change), the change of 1,500 shown is not significant at the 90-percent confidence level. Alternatively, the estimated change of 1,500 does not exceed 1,974 (1.645 x 1,200); therefore, one could conclude from these data that the change is not significant at the 90-percent confidence level.