Negotiating 2004 labor exchange employment performance goals

U.S. Department of Labor, Employment and Training Administration Classification: WIA/Performance Measures Correspondence symbol: OWT Date: June 3, 2004

TRAINING AND EMPLOYMENT GUIDANCE LETTER No. 22-03

TO: ALL STATE WORKFORCE LIAISONS
ALL STATE WORKFORCE AGENCIES

FROM: EMILY STOVER DeROCCO
Assistant Secretary

SUBJECT: Negotiating PY 2004 Expected Levels of Performance for the Labor

Exchange Entered Employment Rate (EER) and Employment Retention Rate (ERR) Performance Measures

1. Purpose. To provide guidelines for negotiating state performance goals for the labor
   exchange component of the One-Stop service delivery system for Program Year 2004.

2. References. Training and Employment Information Notice 13-00, “Consultation Paper on
   Labor Exchange Performance Measurement System;” Training and Employment Guidance Letter

1-02, “Transmittal of ET Handbook No. 406: Reporting Instructions for ETA 9002 and VETS 200 Reports.”

3. Background. On May 31, 2001, the Department of Labor published in the Federal Register
   final performance measures for the public labor exchange. The four labor exchange performance measures are Entered Employment Rate (EER), Employment Retention Rate (ERR), Job Seeker Customer Satisfaction and Employer Customer Satisfaction. The labor exchange performance measures serve as one of three components of the new labor exchange performance measurement system. The other two components are a set of revised reporting requirements for labor exchange services and performance outcomes and procedures for establishing performance goals. On July 1, 2002, states began using revised reporting requirements to collect and report on the new measures.

4. Levels of Performance. Expected levels of performance for individual states for the
   measures will be negotiated for Program Year 2004. Regional Offices will negotiate performance levels with the State Workforce Agencies for two of the four labor exchange measures: Entered Employment Rate (EER) and Employment Retention Rate (ERR).

Section 5 below describes the framework for setting expected levels for the EER and ERR measures.

Rescissions: None stated
Expiration date: Continuing

States will not negotiate Job Seeker and Employer Customer Satisfaction scores as part of this process. States will continue to report on the Job Seeker and Employer Customer Satisfaction measures using the American Customer Satisfaction Index (ACSI) methodology. Please note that the reporting of customer satisfaction is an annual score based on the four quarters of the program year, not on a rolling-four-quarter basis. Program Year 2004 weights for the Wagner- Peyser labor exchange and WIA customer satisfaction measures will be issued in separate guidance. The required response rate for Job Seeker and Employer Customer Satisfaction remains 70%.

5. Framework for Reaching Agreement on State Performance Levels. The following
   elements provide a uniform framework for setting performance targets. The primary tools states and regional offices will use in the negotiation are described in this section. The negotiation process for performance levels must be completed by July 31, 2004. A recommended timeline for the negotiation process is included in Attachment 1.

DOL has developed tools and information to assist in the process of negotiating expected levels of performance:

A. Government Performance and Results Act (GPRA) PY 2004 Goals. States should review the established PY 2004 Government Performance and Results Act targets established for the labor exchange component of the One-Stop system. The GPRA indicators are national targets and will serve as the starting point from which states will negotiate performance levels based on state-specific circumstances. For the EER and ERR measures, the PY 2004 GPRA goals are:

• 58% of job seekers registered with the public labor exchange will enter

employment with a new or different employer by the end of the second quarter following registration.

• 72% of job seekers who entered employment with a new or different employer will
  continue to be employed two quarters after initial entry into employment with a new or different employer.

B. PY 2002 Labor Exchange Performance Outcomes. State quarterly reports for the quarter ending March 31, 2003, recorded an entered employment rate of 61% for job seekers registered in the first two quarters of PY 2002 (7/1/2002-12/31/2002). The first set of state-reported employment retention rates appear in the quarterly report for the quarter ending March 31, 2004 (submitted by May 15, 2004). Entered employment rate results for job seekers that registered in the first three quarters of PY 2002 (7/1/2002- 3/31/2003) are also available in this quarterly report.

C. ETA/VETS PY 2001 Pilot Study. Prior to the availability of actual EER outcome data under the new labor exchange performance measures, ETA and the Veterans’ Employment and Training Service (VETS) undertook a pilot study to gauge performance under the measures. Seven states participated in the pilot, which focused on the collection of PY 2001 labor exchange registrant information and the matching of UI wage records to obtain a proxy for performance under the new measures. Attachment 2 contains a brief analysis of the results of the study, which indicated an average entered

employment rate of 63% for all registrants and an average employment retention rate of 76% for the same population. Because the study’s focus was kept intentionally small, pilot results should not be considered as representative of potential performance across all States.

6. Other Input to Consider. States may also consider the following product and supplementary
   factors.

Internal Benchmarking Methodology Using Data Envelopment Analysis (DEA).

In 2002, ETA and the Labor Exchange Performance Measurement Workgroup commissioned a study to establish an econometric model that would provide an adjustment factor for economic conditions when considering performance levels for the EER and ERR. Using a regression analysis, the study analyzed the impact of several variables, including the unemployment rate, the number of jobs available and the size of the labor force, to see whether there is a positive or negative correlation between the variables and the entered employment rate. The study showed, for example, that there is no positive correlation among any of the economic variables and the entered employment rate. For example, a high unemployment rate in a state does not automatically depress the entered employment rate. Rather, the study concluded that internal policy decisions on service delivery tend to have a greater influence on performance outcomes.

ETA, therefore, investigated another approach known as the Data Envelopment Analysis (DEA) model. The model sets the highest achievable performance levels for any type of operating entity within a large grouping of similar entities by using actual data on resource levels and outcomes observed for those operational units, and identifies optimal performance targets. The model can be used as a tool for setting expected levels of performance for the public labor exchange service component of the One-Stop system. The model considers factors such as service delivery strategies to help states assess current effectiveness of their systems and ascertain changes that could be made at the state level to positively impact outcomes.

Program Year input data on resource use, service transactions, service delivery approach, job postings and the outcomes achieved were used to help estimate the potential range for improvement in outcomes for each state. To test the model, ETA used data available at the time, which were ETA 9002 aggregate state data preceding implementation of the new labor exchange performance measurement system. Model-generated analyses for each state have been shared with the respective regional offices for use (optional) during the negotiation process. While we recognize that the information generated by the model is not necessarily comparable to the performance data available under the new measures, the model can be very useful in assessing performance relative to peers.

Other Possible Factors. Other factors to consider in negotiating expected levels of performance are economic conditions such as the rate of job creation/job loss, new business-start-ups, state legislation or policies which might impact performance, etc.

This list is intended to suggest the kinds of information that might be considered in the negotiation process.

7. Action: Negotiation of Expected Levels of Performance

The DOL process for reaching agreement with states on performance levels will include these steps:

• States will utilize the tools and information discussed in “Framework for Reaching
  Agreement on State Performance Levels” to offer expected state levels of performance for the labor exchange entered employment rate and employment retention rate.

• States are asked to submit a proposal to their regional office by June 30, 2004 (also see
  Timeline) containing proposed levels and the data used to arrive at the targets, including the time period from which the data were drawn and if the data are part of a trend or anomalous.

• Regional offices will review and negotiate with states to obtain mutually agreed upon
  expected levels of performance.

• Regional offices will approve and report the negotiated EER and ERR for each of their
  states to the National Office by July 31, 2004.

It will not be necessary to modify the five-year strategic plan. Meeting or exceeding the labor exchange negotiated levels of performance will not be used to award incentives. Conversely, failure to achieve the negotiated levels will not be used to determine sanctions. Continuous improvement, however, in the core employment and workforce information services component of the One-Stop system will have a positive impact on the workforce investment system’s overall performance. Regional offices will monitor state progress toward achievement of the targets and may request a corrective action plan if a state does not appear on track to achieve its negotiated levels.

8. Common Measures. The Employment Service is one of seventeen Department of Labor
   programs implementing the common measures. Further guidance on the implementation of these measures will be issued in the coming months. In the interim, labor exchange levels of performance for the EER and ERR performance indicators will cover the entire Program Year 2004 (July 1, 2004 - June 30, 2005).

9. Inquiries. Questions concerning this issuance should be directed to your regional office.

Attachments:

Attachment 1: Timeline for Negotiation Process
Attachment 2: ETA/VETS PY 2001 Pilot Study Results

Attachment 1: Negotiation timeline

Official attachment: https://www.dol.gov/sites/dolgov/files/ETA/advisories/TEGL/2004/TEGL22-03_Attach1.pdf

Attachment 1: Timeline for Negotiation Process for Setting PY 2004 Levels of Performance for Labor Exchange Performance Measures

Attachment 2: Pilot study results

Official attachment: https://www.dol.gov/sites/dolgov/files/ETA/advisories/TEGL/2004/TEGL22-03_Attach2.pdf

Attachment 2: ETA/VETS PY 2001 Pilot Study Results

Exhibit 1. Entered Employment Rate (EER) and Employment Retention Rate (ERR) by registrant group for PY 2001 pilot states

Exhibit 2. Comparison of first quarter and annual EER and ERR results for PY 2001 pilot states