Negotiating WIA performance goals for program years 2005 and 2006

Employment and Training Administration Advisory System
U.S. Department of Labor
Washington, D.C. 20210
Classification: WIA/Performance Goals
Correspondence symbol: PROTECH
Date: April 14, 2005

TRAINING AND EMPLOYMENT GUIDANCE LETTER NO. 27-04
TO: ALL STATE WORKFORCE AGENCIES; ALL STATE WORKFORCE LIAISONS
FROM: EMILY STOVER DeROCCO, Assistant Secretary
SUBJECT: Negotiating Workforce Investment Act (WIA) Title 1B Performance Goals for Program Years (PY) 2005 and 2006
Rescissions: TEGL 22-02
Expiration date: Continuing

1. Purpose. To inform states of the guidelines for negotiating PY 2005 and PY 2006 WIA title 1B performance and customer satisfaction goals. The negotiated performance goals for these two program years will be included in the WIA and Wagner-Peyser Act State Plans, which are due to ETA by May 31, 2005.

2. References. WIA Section 136; WIA regulations at 20 CFR Part 666 and Part 661; Training and Employment Guidance Letter (TEGL) No. 7-99, “Core and Customer Satisfaction Performance Measures for the Workforce Investment System;” TEGL No. 8-99, “Negotiating Performance Goals and Incentives and Sanctions Process Under Title I of the Workforce Investment Act;” TEGL No. 11-01, “Guidance on Revising Workforce Investment Act State Negotiated Levels of Performance;” TEGL 22-02, “Negotiation of Performance Goals for Program Years Four and Five Under Title I of the Workforce Investment Act;” TEGL No. 15-03, “Common Measures Policy;” TEGL No. 14-04, “Announcing the Soon-to-be-Published Proposed Revisions to the Planning Guidance and Instructions for Submission of the Strategic Five-Year Plan for Title I of the WIA of 1998 and the Wagner-Peyser Act; and Workforce Investment Act Unified Planning Guidance;” Proposed PY 2005, “Planning Guidance and Instructions for Submission of the Strategic Five-Year Plan for Title I of the Workforce Investment Act of 1998 and the Wagner Peyser Act” (http://www.doleta.gov/usworkforce/wia-planning-guidance/standalone.cfm and see 70 Federal Register 6455, February 7, 2005); and TEGL No. 18-04, “Announcing the Soon-to-be-Published Proposed Revisions to Existing Performance Reporting Requirements for the Implementation of Common Measures for Title I of the Workforce Investment Act, the Wagner-Peyser Act (Employment Service/Labor Exchange), the Trade Adjustment Assistance Reform Act, and Title 38, Chapter 41 Job Counseling, Training, and Placement Service (Veterans’ Employment and Training Service).”

3. Background. As states are preparing the WIA state plans and addressing what policies are in place to support common data collection and reporting processes, information management, integrated service delivery, and performance management, they are also preparing to negotiate performance goals for PY 2005 and PY 2006. In order to effectively provide integrated services to meet the needs of business and job seekers, states should negotiate their goals with changes to the provision of services in mind. The workforce investment system is being transformed to a demand-driven system with strategic investments in workforce solutions that result in more individuals being trained for high-skill jobs in high-growth, high-demand industries. The demand-driven approach may require unique approaches to training and service delivery. It may require shorter and targeted training in some cases and longer term training in others, depending on the specific skills and credentials identified as necessary by industry — both at entry level and as individuals progress up and across career lattices. Some workforce solutions may be developed to ensure individuals requiring more comprehensive service strategies are part of the pipeline of workers moving into jobs in high-growth industries such as out-of-school youth, individuals with limited English proficiency, or individuals with disabilities.

The performance accountability system, including negotiated levels of performance, should be aligned with the specific demand-driven strategies identified both at the state and local levels. Additionally, states and local areas should consider options for making the accountability system more flexible, including the use of the waiver provision in WIA section 189(i).

The information in this guidance letter supercedes previously issued guidance related to performance negotiations and clarifies the performance measures that will be utilized for performance accountability purposes for PY 2005 and PY 2006. In general, the process for negotiations will not change from the processes used in the past. However, we believe the upcoming round of negotiations will be improved by the provision of consistent analysis tools across the system. These tools are available as attachments to this guidance, as well as through ETA’s performance Web site (www.doleta.gov/Performance).

This guidance provides a clarification of the measures for which states will be accountable in PY 2005 and PY 2006, descriptions of tools available for use during the negotiations, and an outline of the negotiation process. The negotiations themselves take place between the states and the corresponding ETA regional offices, and specific guidance regarding contacts and timeframes will be provided to the states.

States will not be required to establish PY 2005 performance levels for the Wagner-Peyser Act component of the state plans. Wagner-Peyser Act programs will begin reporting on common measures beginning July 1, 2005, in lieu of the existing labor exchange performance measures. Due to the application of new features for the labor exchange function, such as the concept of program exit, PY 2005 will serve as a baseline year to capture information under the common measures that will aid in negotiations in subsequent program years.

4. Clarification of Accountability. Section 136 of WIA provides clear guidance on what measures of performance will be used to assess the effectiveness of states and local areas in achieving continuous improvement of workforce investment activities funded under title I of WIA. The core indicators of performance for the adult, dislocated worker and older youth populations served as listed in section 136 include:

• Entry into unsubsidized employment;

• Retention in unsubsidized employment six months after entry into the employment;

• Earnings received in unsubsidized employment six months after entry into the employment; and

• Attainment of a recognized credential relating to achievement of educational skills.

The core indicators of performance for the younger youth population served include:

• Attainment of basic skills and, as appropriate, work readiness or occupational skills;
• Attainment of secondary school diplomas and their recognized equivalents; and
• Placement and retention in postsecondary education or advanced training, or placement and retention in military service, employment, or qualified apprenticeships.

Additionally, the law includes that states will be held accountable for collecting and reporting measures of employer and participant customer satisfaction.

These indicators were defined through ETA policy guidance (TEGL 7-99), which prescribed the definitions and methodologies for measuring outcomes against these indicators. During PY 2005 and PY 2006, states and local areas will be held accountable to these indicators; however, the definitions behind the key measures for the adult and dislocated worker populations will be changed in order to implement a set of common performance measures across programs. In the near future, additional guidance on the common measures will be issued to clarify the common measures policy issued in TEGL 15-03. The following table presents a snapshot of the measures for which states will be accountable in PY 2005 and PY 2006.

Negotiating Performance for PY 2005 and PY 2006: Measures and Definition Sources

Changes to existing quarterly and annual reports for WIA will be announced in spring 2005 in order to accommodate the revisions necessary to report performance in PY 2005. In addition to these changes, the revised reporting instructions will include a means for states and locals to begin reporting against the common youth measures, which are:

• Placement in employment or education;
• Attainment of a degree or certificate; and
• Literacy and numeracy gains.

During PY 2005, states will collect and report common measures data for the placement and degree/certificate attainment measures for the entire youth population, older and younger combined. States are encouraged to begin data collection and reporting for the literacy/numeracy gains measure, although this is not required for PY 2005. States will not negotiate levels of performance for these three measures during the planning process that takes place in spring 2005, and performance against these measures will not affect a state’s eligibility for incentives or be a factor in determining whether a state would be subject to sanctions for PY 2005 performance. Data collected on the placement and degree/certificate attainment measures during PY 2005 will serve as a baseline for negotiations on these measures for PY 2006, and all states will be required to begin reporting performance outcomes on the literacy/numeracy gains measure in PY 2006.

5. Reaching Agreement on State Performance Levels. States should use negotiated levels of performance to manage for continuous improvement and enhanced customer satisfaction. The following tools and process guidelines provide a uniform framework for states to set performance goals that demonstrate this commitment. Final performance levels must be negotiated and agreed upon by the state and the ETA Regional Administrator no later than June 30, 2005.

A. Tools for Proposing Levels of Performance

I. Past performance. States should use historical, annual performance information (PY 2001-2003) to inform projected levels of performance for PY 2005 and PY 2006. They should also use recent quarterly performance results to inform the performance path the state is following. The Department anticipates that states will submit proposed levels of performance that reflect continuous improvement and additional experience, and show increases over the prior years’ performance levels. However, performance levels may vary, up or down, based on prior performance and environmental factors that are beyond the state’s control. Various tools and resources are available to examine states’ historical performance data, including state-by-state files of the data (see www.doleta.gov/Performance/results/wia_national_performance.cfm) and the Federal Research and Evaluation Database (www.fred-info.org).

II. Government Performance and Results Act (GPRA) goals. Throughout the performance negotiation process, states should be aware of the GPRA goals the Department has established for PY 2005. These goals will be used by regional offices as benchmarks by which to gauge their states’ proposed performance levels. The GPRA is an important mechanism by which Congress evaluates the success of Federal programs, including

those which are operated by states and localities. The GPRA is also a principal component of the President’s Management Agenda, by which the Administration evaluates programs as part of its goal to integrate budget and performance outcomes. The GPRA performance goals for the Department are listed in Attachment II, and more information is available at http://www.doleta.gov/Performance/goals/gpra.cfm.

III. National comparisons. ETA has utilized states’ previously submitted annual performance data to provide information on the national distribution of performance outcomes. These benchmarks serve as estimates that states and regions can refer to when setting goals to achieve continuous improvement. Attachment III presents the national distribution of performance outcomes for the past three years. (Please note that these tables include estimates of outcomes using common measure definitions for adult and dislocated workers. These estimates were created by using states’ previously submitted Workforce Investment Act Standardized Record Data (WIASRD). State-by-state estimates are available in electronic format from each regional office.)

IV. Estimates of the effects of economic and demographic variables. WIA section 136(b)(3)(A)(iv) (see http://www.doleta.gov/performance/guidance/laws_regs.cfm) and TEGL 8-99 address additional factors, such as differences in economic conditions, characteristics of participants, and services to be provided, that can be considered in the negotiation process. ETA has provided estimates of how various economic and demographic variables may impact outcomes achieved in Attachment IV. However, these estimates are based on national-level data, and do not necessarily reflect the economic conditions and client base unique to a state. Therefore, these estimates should not be considered hard and fast reasons for adjusting performance goals up or down. Instead, they are provided as an example of how a state might analyze its own data in order to propose goals that take into account the characteristics of individuals served and economic conditions in the state.

When negotiating the youth goals, states should be aware that ETA’s new strategic vision for youth services includes a focus on serving the neediest youth, especially out-of-school youth including youth in foster care, youth in the juvenile justice system, children of incarcerated parents, and migrant youth, as well as youth with disabilities. States that transition to serving a higher percentage of these more difficult-to-serve populations may not be able to achieve performance levels as high as in the past. Therefore, when proposing expected levels of performance states should take into account the populations being served since serving a greater percentage of the neediest youth has the potential to impact outcomes. If states plan to serve a greater percentage of the neediest youth, they should provide data that shows how outcomes are impacted by serving this population.

B. Process for Reaching Agreement on State Performance Levels

The process for reaching agreement on state performance levels will include the following steps, as outlined in the attached timeline (Attachment I):

I. After conducting their own analysis of factors that may affect performance, states will propose levels of performance for each of the performance and customer satisfaction indicators for PY 2005 and PY 2006 by submitting these proposed levels to the Regional Administrator serving the state. Proposed performance levels must be included in the state plans due to ETA by May 31, 2005; however, regional offices will work with states to begin the negotiation process prior to that date and in parallel with the planning process in order to ensure that final levels are agreed upon by June 30, 2005.

When submitting the proposed levels, states should provide the following as support for the levels, which is also necessary documentation for the state plans (see Section X, Part D, Item 1):

• The methodology used for developing proposed levels of performance including a description of data sources, calculations, and additional environmental factors (such as those listed in WIA section 136(b)(3)(A)(iv) and previously addressed in TEGL 8-99.)

• How the target levels will promote continuous improvement in state performance.

When submitting the proposed levels for review, states should also include a discussion of how the proposed levels will positively impact customer satisfaction with services received and the extent to which the proposed levels ensure optimal return on investment of Federal funds. (See WIA section 136(b)(3)(A)(iii) and (iv).)

II. The regional office will review the analyses used by the state to develop the proposed performance levels and will work with the state to set mutually agreed upon levels of performance. The regional office will consider the proposed levels in light of previously negotiated goals, past performance results, and national GPRA goals. Regional offices will take into account the environmental factors addressed by the states. Additionally, regional offices will consider the quality of the data presented by states, including its relevance, source, the time period from which it is drawn, and whether the data is part of a trend or anomalous. The negotiation process will focus on whether each performance level appears appropriate in light of statutory criteria and this guidance, and the adequacy of any information states offer to substantiate each level. If regional offices determine through their analysis that a state could increase its proposed performance levels to more fully support continuous improvement and customer satisfaction strategies, they will negotiate with the state to obtain mutually agreed upon performance levels.

III. Once the performance levels are agreed upon, the Regional Administrator will send a letter to the state confirming the agreed upon levels. ETA will incorporate these agreed upon levels into the state plans.

6. Inclusion of Performance Goals in State Plans. At a minimum, states will be required to submit the proposed levels of performance as part of the state plans, which are due to ETA by May 31, 2005. (See 20 CFR 661.230(b)(2) and “Planning Guidance and Instructions for Submission of the Strategic Five-Year Plan for Title I of the Workforce Investment Act of 1998 and the Wagner Peyser Act.”) In the case that final agreement on performance goals is not reached before the plan is submitted for review, the regional office and state staff will continue to negotiate the measures subsequent to May 31, 2005. However, final performance levels must be agreed upon no later than June 30, 2005.

States should note that the content of the state plans, including the proposed levels of performance, is subject to public review and comment requirements.

7. Action Required. States are requested to distribute this information to the appropriate state and local staff.

8. Inquiries. Questions concerning this issuance may be directed to the appropriate regional office.

9. Attachments.

Attachment I: Recommended Timeline for Negotiation Process

Attachment II: GPRA Performance Goals for the Department of Labor

Attachment III: National Distribution of Performance Outcomes, PY 2001-PY 2003
Attachment IV: Economic and Demographic Variables

Attachment V: Time Periods for Reporting Performance in the WIA Annual Reports

Attachment I: Recommended timeline for negotiation process

Official attachment: https://www.dol.gov/sites/dolgov/files/ETA/advisories/TEGL/2005/TEGL27-04_AttachI.pdf

Attachment II: GPRA performance goals for the Department of Labor

Official attachment: https://www.dol.gov/sites/dolgov/files/ETA/advisories/TEGL/2005/TEGL27-04_AttachII.pdf

Please note: Goals for PY 2005 were set based on WIA policy guidance (TEGL 7-99).

Attachment III: National distribution of performance outcomes, PY 2001–PY 2003

Official attachment: https://www.dol.gov/sites/dolgov/files/ETA/advisories/TEGL/2005/TEGL27-04_AttachIII.pdf

Table 1: National averages of performance measure outcomes

1. Common Measure estimates derived from Workforce Investment Act Standardized Record Data (WIASRD).
2. Includes supplemental data.

Table 2: National distribution of performance outcomes

Each group follows PY 01 / PY 02 / PY 03 / Avg. Percent and dollar units apply to every cell in the relevant row.

The source prints FALSE in the Top 10% adult entered-employment average cell.

1. Common Measure estimates derived from Workforce Investment Act Standardized Record Data (WIASRD).
2. Includes supplemental data.

Attachment IV: Economic and demographic variables

Official attachment: https://www.dol.gov/sites/dolgov/files/ETA/advisories/TEGL/2005/TEGL27-04_AttachIV.pdf

Table 1: Relationship between performance outcomes and unemployment rates and customer characteristics

Each value is the effect of a one percentage point increase in the column variable on the row measure. -- is the source's empty marker, not zero. Columns for adults and dislocated workers: unemployment rate; percent age 55 or older; percent not high school graduate; percent low income; percent with disabilities.

For older youth, the columns are unemployment rate; percent age 19; percent not high school graduate; percent basic skills deficient; percent with disabilities.

For younger youth skill and diploma attainment, the columns are unemployment rate; percent age 14 or 15; percent high school dropout; percent basic skills deficient; percent with disabilities. For retention, the third column is percent not high school graduate.

Source note: “Almost all adjustments shown are statistically significant at the 1% level; the remainder is statistically significant at the 5% level.”

Table 1: Relationship Between Performance Outcomes and Unemployment Rates and Customer Characteristics

This table shows estimates of how performance outcomes are affected by changes in unemployment rates and selected customer characteristics. The analysis was conducted using the PY 2003 Workforce Investment Act Standardized Record Data (WIASRD). Each number in the table represents the effect on performance of a one percentage point change in the unemployment rate or in the percentage of customers with a specific characteristic. This type of information can be used during negotiations to inform discussions of the impact of changes in the economic environment and customer characteristics on state performance. States are encouraged to conduct similar analyses using their own state data when proposing performance levels.

Performance measures shown in the table were determined as follows:

• Adults and dislocated workers
  • Entered employment uses the common measures definition and reflects the use of supplemental data.
  • Retention uses the common measures definition and reflects the use of supplemental data.
  • Earnings change uses the TEGL 7-99 adult definition.
  • Credential uses the TEGL 7-99 definition and uses supplemental data to determine employment.
• Older and Younger Youth
  • All measures based on TEGL 7-99 definitions and use supplemental data where appropriate.

Table 1 provides estimated relationships between unemployment rates and performance outcomes. These relationships were estimated using WIASRD information on outcomes for exiters in the most recent year available for each measure¹ and Calendar Year (CY) 2002 state unemployment rates in a simple bivariate regression analysis. The results indicate, for example, that an increase of the national unemployment rate by one percentage point (for example, from 4% to 5%) decreases the national adult entered employment rate by 3.1 percentage points (for example, from 70.0% to 66.9%). This information may be used in conjunction with the information in Table 2 to determine the expected influence of changes in unemployment rates on entered employment rates.

Table 1 also provides estimated relationships between selected customer characteristics and performance outcomes. These estimates were developed in the same way as the estimates for the unemployment rate, using bivariate regressions based on WIASRD information. These estimates indicate, for example, that if the percentage of older youth who are not high school graduates increases by 10 percentage points, then the employment retention rate is expected to decrease by 0.86 percentage points (10 x -0.086). It should be noted, that the estimated relationships between performance outcomes and customer characteristics are generally small enough that only very large changes in customer characteristics will have a material impact on outcomes. However, modest changes in the age categories or education status of youth, especially younger youth, can have a noticeable effect on outcomes.

1 Estimates for entered employment and credential attainment were based on exiters between October 2002 and September 2003. Estimates for retention and earnings change were based on exiters from April 2002 to March 2003. Estimates for younger youth skill attainment and younger youth diploma attainment were based on exiters between April 2003 and March 2004.

States should note that the following parameters and definitions were used to create the estimates for customer characteristics:

• The percentage with disabilities includes both disabilities that are a substantial barrier to employment and other disabilities.

• The percentage in an age group and the percentage with disabilities were based on all WIA exiters (except as noted below for the adult and dislocated worker credential and employment rates and the younger youth diploma rate).

• The percentage who are low income or not high school graduates were calculated for all younger or older youth and for adults and dislocated workers who received intensive or training services (except as noted below for the adult and dislocated worker credential and employment rates and the younger youth diploma rate).

• When adjusting the adult and dislocated worker credential and employment rates, the customer characteristics were based on those who receive training (e.g., the relationship between the adult employment rate and the percent low-income variable is based on the percentage of low-income individuals receiving training services).

• When adjusting the younger youth diploma rate, the customer characteristics were based on younger youth who were not high school graduates, or the equivalent, at registration.

• For younger youth, “not a high school graduate” includes school dropouts and those attending high school.

The estimated adjustments were designed to make adjustments only for a single characteristic.

However, simultaneous adjustments for several characteristics are reasonably accurate with a couple of exceptions:

• Adjustments of older youth outcomes for both the percent not a high school graduate and the percent basic skills deficient leads to a slight over adjustment.

• Adjustments of younger youth outcomes for both age 14 or 15 and education level or both education level and basic skills deficiency will lead to a modest over adjustment.

Table 2: Recent and projected unemployment rates

Projected national rates (%) for CY 2004–2008: 5.5 | 5.3 | 5.2 | 5.1 | 5.1.

Actual rates (%) by CY 2000, CY 2001, CY 2002, CY 2003, and December 2004:

Footnote 2: National average for CY 2004 was 5.5%; annual state data for CY 2004 were not yet available until March 2005. Footnote 3: Seasonally adjusted, preliminary. Footnote 4: National unemployment rate for January 2005 was 5.2%.

Table 2: Recent and Projected Unemployment Rates

Table 2 shows recent and projected unemployment rates for the nation and each state. The values for CY 2003 are representative of when the PY 2003 employment and earnings outcomes occurred. For example, both entered employment for exiters from October 2002 to September

2003 and retention for exiters from April 2002 to March 2003 occurred during CY 2003. Annual data for CY 2004 are not yet available.

The December 2004 numbers are provided as an indication of where states are currently. The national projections are provided to help extrapolate state values to the future. The projections were developed by the Administration to support the FY 2006 budget.

To project a state's unemployment rate in 2007, one might apply the projected national change in unemployment from CY 2003 to CY 2007 to the state's actual 2003 unemployment rate. For example, the national change from CY 2003 to CY 2007 is negative 0.9 percentage points. Thus a state’s CY 2007 unemployment rate might be estimated as the CY 2003 value (5.1) minus 0.9, which yields an unemployment rate of 4.2. Please note that the result of this procedure depends on whether CY 2002 or CY 2003 is used as the base year. Therefore, it can only provide a general guide to future state unemployment rates. States may also have developed forecasts of

unemployment rates, which may be more appropriate to state conditions than the rough estimates that can be generated from the table.

(When using Table 2 to inform the negotiation process, please note that the unemployment rates shown are calendar years, not program years.)

Attachment V: Time periods for reporting performance information

Official attachment: https://www.dol.gov/sites/dolgov/files/ETA/advisories/TEGL/2005/TEGL27-04_AttachV.pdf

WIA Annual Report for PY 2005 and PY 2006.