Negotiating WIA performance goals and incentive and sanction process

Training and Employment Guidance Letter No. 8-99

Official signed 20-page packet: https://www.dol.gov/sites/dolgov/files/ETA/advisories/TEGL/2000/TEGL8-99.pdf

ETA status listing (Canceled, accessed 2026-10-27): https://www.dol.gov/agencies/eta/advisories/training-and-employment-guidance-letter-no-08-99

U. S. Department of Labor
Employment and Training Administration
Washington, D.C. 20210
CLASSIFICATION
WIA/Performance
CORRESPONDENCE SYMBOL
OWS
DATE
March 3, 2000
TRAINING AND EMPLOYMENT GUIDANCE LETTER NO. 8-99
TO: ALL STATE WORKFORCE LIAISONS
ALL STATE WORKER ADJUSTMENT LIAISONS
ALL STATE EMPLOYMENT SECURITY LIAISONS
ALL ONE-STOP CAREER CENTER SYSTEM LEADS
FROM: LENITA JACOBS-SIMMONS
Deputy Assistant Secretary
SUBJECT: Negotiating Performance Goals; and Incentives and Sanctions Process under Title I of the Workforce Investment Act (WIA).

1. Purpose. To inform States of ETA’s implementation guidelines for the Negotiation and Goal
   Setting Process as well as for the Incentive and Sanction Process.
2. References. Workforce Investment Act of 1998 (Pub. L. 105-220), Section 136, Workforce
   Investment Act, Interim Final Rule, 20 CFR Part 666, published at 64 Federal Register 18662 (April 15, 1999).
3. Background. Under the Workforce Investment Act of 1998, States submitting a State Plan under
   Title I, subtitle B must propose expected levels of performance for each of the 17 indicators of performance consisting of the adult, dislocated worker and youth programs respectively and the two customer satisfaction indicators. While States may develop additional performance measures for State use, such measures will not be negotiated with the Department or included in the incentives and sanctions process. In order to ensure an optimal return on the investment of the activities authorized, the Secretary and Governor must reach agreement in the State plan on the levels of performance for each of the first three program years. The requirements of the Government Performance and Results Act (GPRA) will also be a factor in negotiating the expected levels of performance. A critical component of annual reports to Congress will be representations of the extent to which States

have met these negotiated performance levels. This process begins with each State negotiating performance levels with their Regional Office for all measures in each program area (adults, dislocated workers, and youth) and for customer satisfaction and contains a commitment that incentives and sanctions provided for under the law will be based on meeting those negotiated performance levels. This directive provides the requirements for that process, from negotiation between States and ETA’s Regional Offices to subsequent review of levels achieved to determine appropriate incentives and sanctions.

4. Reaching Agreement on State Performance Levels. With the statutory inclusion of
   performance negotiations in WIA comes an expectation that States will use this provision in a pro- active manner to support their efforts for continuous improvement and customer satisfaction. Therefore, ETA has developed requirements that provide a uniform framework for negotiations and give States flexibility in pursuing continuous improvement for each program area. In order to obtain negotiated levels of performance for all 17 measures that demonstrate this commitment, the following negotiation process will be utilized. Please note that early implementing States will be negotiating performance levels for their 2nd and 3rd years of performance. Early implementing States should conform such negotiations to the guidance in this directive. A. Process For Reaching Agreement on State Performance Levels The DOL process for reaching agreement with States on performance levels will include these steps: 1) States will develop baseline data following the instructions within this directive. 2) States will analyze the baseline data and then offer expected State levels of performance, submitted as part of their State Plan, for each of the 17 indicators for each of the first three program years. States are encouraged to share this advance data and work with their Regional Office throughout this process. 3) The information will go to the ETA Regional Office serving the State to review and the Regional Office will negotiate with the State to obtain mutually agreed upon expected levels of performance. 4) The Regional Office will submit the negotiated performance levels1 to the National Office for ETA for inclusion in the State Plan approval process. 5) Actual performance for each program year will then be compared to these negotiated performance levels to determine whether States are eligible for incentives or subject to sanctions. 6) Prior to the fourth program year, States will submit their expected performance levels for the fourth and fifth years of their program following this guidance in the form of plan modifications. 7) ETA will also follow this guidance in the review, negotiation, and submission of the performance levels for program years four and five, although adjustments in the process may occur based upon experience. B. Baselines To the extent available, baselines for each of the 17 measures will be developed by each State and will be a key factor used to determine the level of performance that is negotiated with ETA. Summary definitions of the measures are presented in Attachment A. Baselines are intended to give an indication of the past outcomes of a performance measure. For performance negotiations to be data-driven and reality based, the development of baselines is a critical aspect of the negotiation process. Because WIA authorizes new programs, there is not a perfect match of baseline data available to predict program performance under WIA. Instead, baselines must be based on data from preexisting programs. This situation will change in the coming years once WIA data is accumulated. Initially, developing baselines using JTPA data is more appropriate for some of the core performance measures than for others. Baselines can be developed for the entered employment, employment retention, and earnings gain measures based on wage records because the wage record data, although not used under JTPA, can be accessed retrospectively by most States. To assist States in the development of baseline data and expected performance levels, ETA has developed parameters to be used by all States in their baseline calculations. The Department anticipates that States will submit expected levels of performance that are above the baseline figures computed for the State. The parameters are outlined in Attachment B. The matrix in Attachment B will give States general instructions for each measure and will provide for comparable data nationwide. However, for specific operational definitions and calculation instructions on the core performance measures and the customer satisfaction measures, States should reference the TEGL 7-99, “Core and Customer Satisfaction Measures For the Workforce Investment System.” C. State Expected Levels of Performance States will submit expected State levels of performance as part of the State Plan. States should be prepared to provide support for their expected performance levels by providing information on how baseline performance levels were developed as well as by providing other information States believe may affect performance. States will include in their plan:
   • Baseline performance data and expected levels of performance for each of the 17 measures.
   • The extent to which the expected levels of performance will result in the State attaining a high level of customer satisfaction.
   • How the expected levels of performance compare with projected national averages (see Attachment B for projected national averages), taking into account how factors such as economic conditions, the characteristics of participants when the participants entered the program, and the services to be provided contributed to the development of the expected performance levels?
   • The extent to which such expected levels of performance promote continuous improvement and ensure optimal return on investment of Federal funds. In addition, the methodology for developing the baseline data and expected levels of performance should be included when possible. This should include a description of data sources, calculations, factors (see below for further discussion on factors), and values of factors used for expected levels of performance. For example, if a factor such as unemployment rate is used, the level of unemployment should also be included. ETA has developed projected national average performance levels for each performance measure which should be utilized in developing State expected levels of performance. The national average for each performance measure is included in the matrix in Attachment B. While most of these national averages are based on the experiences of a limited number of early implementation States, ETA has determined these levels to be representative of estimated performance under WIA. Performance levels may vary, up or down, based on environmental factors beyond the control of the State. Examples of possible factors to consider in negotiating expected levels of performance are listed below. This list of factors is not intended to be prescriptive or exhaustive, but to suggest the kinds of information that might be considered in the negotiation process. Differences in Economic Conditions
   • Unemployment rate
   • Rate of job creation/loss
   • New business start-ups
   Characteristics of Participants
   • Indicator of welfare dependency
   • Indicator of educational level
   • Indicator of poor work history
   • Indicator of basic skills deficiency
   • Indicator of disability
   • Indicator of age
   • Creation of “hardest-to-serve” index
   Services to be Provided
   • % WIA Title I to be spent on each category of service (core, intensive, training)
   • Extent of follow-up services planned
   • Extent of experimental/pilot programs
   • Extent of non-Title I training money available
   Other Factors That May Be Considered
   • Community factors, like availability of transportation and daycare
   • Policy-objective factors, like:
   • evidence of application of Malcomb Baldridge criteria
   • pursuit of new or enhanced partnerships
   • piloting of new programs
   D. Negotiation of Expected Levels of Performance
   The Regional Office will review the information contained in the State plan and will compare the expected performance levels with the national averages, baseline information from other States, and the negotiated levels of performance established for other States, taking into account factors including differences in economic conditions, the characteristics of participants when the participants entered the program, and the services to be provided as discussed above. [It is understood that comparative information on negotiated levels may not be fully available during the negotiation process]. A discussion of the appropriateness of each performance level, and the adequacy of any information States offer to substantiate each performance level, will be the core of the negotiation process. The Regional Office will analyze the quality of the data presented by States, including the relevance of the data, the source of the data, the time period from which the data was drawn, and if the data is part of a trend or anomalous. When the Regional Office finalizes its analysis and determines that the State could increase their expected performance levels to support State continuous improvement and customer satisfaction strategies, they will negotiate with the State to obtain mutually agreed upon expected levels of performance. E. Revisions WIA allows for the renegotiation of performance levels if circumstances arise that result in a significant change in the factors used to develop the original performance levels. Since States have limited baseline data to project performance levels for PY 2000, 2001 and 2002, it is anticipated that States may want to renegotiate performance for PY 2001 and 2002 as more baseline data becomes available. DOL will collaborate with the system to develop methods for making such revisions. Such criteria could include significant changes in economic conditions, changes in service mix, or changes in client characteristics.
5. Incentives and Sanctions Policies.
   Under 20 CFR 666.200, a State is eligible to apply for an incentive grant if its performance for the immediately preceding year exceeds:
   • The State’s negotiated levels of performance for the following areas: adult measures, dislocated worker measures, youth measures, and customer satisfaction measures for WIA Title I;
   • The negotiated levels of performance included in plans submitted to the Department of Education for Title II Adult Education and Literacy programs; and
   • The negotiated levels of performance under Title I of the Carl D. Perkins Vocational and Technical Education Act. This guidance addresses the initial requirements for a State to be eligible for the WIA Title I incentives grant awards. Additional guidance will be provided at a future date about the relationship between the Department of Labor and Department of Education in the incentive process, as well as a detailed description of the incentive award application process. A. Evaluating Performance for Incentives and Sanctions The extent to which each State exceeds, meets, or falls below its negotiated performance levels will be examined. For each core performance measure, the percentage by which each State met its negotiated performance level will be calculated (e.g., if a State had an 80% negotiated performance level for the adult entered employment rate and the State’s actual performance was 80%, they would have achieved 100% of their negotiated performance level and receive a score of 100% on that measure. If the State’s actual performance was only 40%, the would have achieved only 50% of their negotiated target level and receive a score of 50% on that measure). This analysis will be clustered by program area (adults, dislocated workers, and youth - both older and younger youth measures) and for customer satisfaction, so that each of the four groups will receive an aggregate performance score calculated from their scores for each measure in each program area and in the customer satisfaction group. The exact aggregation approach to be used will be determined as actual performance information becomes available and will likely include simple or weighted averaging. The resulting score will be the cumulative program area score for each of the four areas. To be eligible to apply for an incentive grant, a State must meet two criteria. A State must achieve at least a 100% cumulative program area score for each of the program areas and for the customer satisfaction group. In addition, a State may not have any of their 17 measures fall below 80% of their negotiated performance levels in order to be eligible to apply for an incentive grant. It is understood that as part of a continuous improvement strategy, States may wish to emphasize improvement on a particular measure (e.g., entered employment) which may have a slightly adverse effect on other measures in the same program area. To allow for these strategies, and to avoid consideration of sanctions for only missing their negotiated goal by a few percentage points, States will not trigger the sanctions process as long as they have achieved 80% of their negotiated performance levels for each measure. Example 1 shows the four performance measures for the adult area, and the extent to which a state exceeded or fell below the negotiated performance levels on each of those measures. Note that the State did not achieve two of the targets. However, since the performance on each of the measures was above the 80% threshold, the State is within the acceptable performance range. This allows States the flexibility to target certain measures within a program area for improvement, without concern for minor decreases in other measures within the program area. The following two examples shows how the cumulative program scores would be used to determine a State’s eligibility for an incentive award. In each example, the bars for each program area represent the cumulative program score. To encourage States to strive to achieve the negotiated level of performance, rather than just to meet the lower acceptable level (80%) allowed, incentive grants will only be available for a State that achieves a cumulative score in each area (adults/youth/dislocated worker/customer satisfaction) that is 100% or greater and where a State does not fall below 80% of the negotiated level on any single measure. Example 2 shows the 100% incentive threshold and cumulative program scores for each program area. A State has a cumulative program score of 115% of the negotiated target for the adult program, 110% for the dislocated workers, 100% for the youth program, and an 110% cumulative score for the two customer satisfaction measures (and did not fall below 80% of negotiated target on any of the measures). The State would be eligible for an incentive award. In Example 3: A State has cumulative program scores of 100% or higher of negotiated target performance levels in all program areas but less than 100% cumulative score on customer satisfaction. The State would not be eligible for an incentive award. Cumulative scores will NOT be averaged across areas; a State cannot make up a deficit in one area by exceeding performance in another. B. Qualifying for Incentive Grants. In order to promote equity and uniformity, only the Federally required core and customer satisfaction measures will be considered. C. Amount of the Incentive Award. Awards will range from $750,000 to $3,000,000 with a proportionate reduction in the minimum and maximum when total available funds are insufficient. D. Incentive Application Process. The final approach to distributing incentive awards is still under development, but may include the following process steps:
   • Performance reports from each State will be submitted by a date specified by the Secretary.
   • The Department of Labor will partner with the Department of Education to review the performance of each State and determine which States are eligible for an incentive award.
   • For those States that are determined eligible, the following criteria may be used to determine the amount each State is eligible to receive:
   • The extent to which goals were exceeded;
   • The extent of improvements from previous year; and
   • The relationship of actual performance levels to actual performance levels in other states
   • These adjusted award amounts will be published and applications for awards will be solicited. E. Failing to Meet the Negotiated Levels. Negotiated levels will be established for each of the 17 performance measures. The lowest acceptable level (lower limit) will be calculated as 80% of the established State negotiated level. States must attain 80% of the target performance level on each measure for performance to be determined acceptable. If a State falls below this threshold of 80% on any of the performance measures, the State may be subject to sanction. Although WIA and this guidance provide some specific requirements for sanctions, the Department will review possible sanctions on a case by case basis. Example 4: A State has exceeded its negotiated target performance on adult entered employment by performing 110%, on employment retention by 105%, on earnings change by 100%, but only achieved 75% of the negotiated performance target for credential attainment. The State may be subject to sanction. Example 5: A State did not achieve 100% of the negotiated performance targets on any of the adult measures. However, the State did achieve at least 80% of the negotiated performance target on each of these measures. The state would not be subject to sanction. F. Unacceptable Performance in Year One The first year a State experiences unacceptable performance, ETA will make technical assistance available. This technical assistance will be provided upon request. One approach to this technical assistance may be helping the State to develop a performance improvement plan with the following components:
   • Statement regarding which performance indicators will be improved through implementation of plan;
   • Analysis of performance problem, including a description of problem solving techniques used to determine the most likely causes of the problem and a description of the most likely causes;
   • Identification of possible actions to improve performance, and a descriptions of what actions the State will take to improve; and
   • A timetable for implementing the selected solutions.
   G. Unacceptable Performance in Two Consecutive Years.
   If performance is unacceptable for two consecutive years on the same performance indicator, monetary sanctions may be imposed. H. Amount of Monetary Sanctions. Sanctions will be determined on a case by case basis, and may range from 1% up to a maximum 5% reduction. I. Sanctions for Failure to Submit Annual Performance Progress Reports. Under 20 CFR 667.300(e), States that are more than 45 days late may be sanctioned up to 1% plus up to an additional 1% for each additional 45 day period of lateness up to a total reduction of 5%.
6. Action. States are to provide these requirements to the appropriate State staff and use this guidance
   to:
   a) develop baseline data and expected performance levels PY2000, PY2001 and PY2002 and submit this information in the State’s WIA State Plan. b) negotiate performance levels to be used in the State Plan approval process.
7. Inquiries. Questions concerning this issuance should be directed to your appropriate Regional
   Office.
8. Attachments. Attachment A: WIA Core and Customer Satisfaction Measures-at-a-Glance;
   Attachment B: Common Approach for Establishing Negotiated Levels of Performance; Attachment C: Summary of Important Terms.

Footnote 1: In the Interim Final Rule, 20 CFR part 666, the outcome of the reaching agreement on State expected levels of performance was termed “adjusted levels of performance.” In this document and in the Final Rule, the outcome of reaching agreement of State expected levels of performance will be termed “negotiated performance level.”

Performance examples printed as bar charts (pages 7 and 9)

The charts compare each measure or program area with an 80% minimum performance line or a 100% incentive line. Their bars have no printed numeric labels; the accompanying letter text supplies the operative figures. Example 1 shows four adult measures, with the two below the negotiated target still above the 80% minimum. Example 2 shows cumulative scores of 115% for adults, 110% for dislocated workers, 100% for youth and 110% for customer satisfaction, meeting the incentive threshold. Example 3 shows the customer satisfaction score below 100%, so the State is ineligible despite the other scores. Example 4 shows adult entered employment at 110%, retention at 105%, earnings change at 100% and credentials at 75%; the State may be sanctioned. Example 5 shows all four adult measures below 100% but each at or above 80%, so the State is not subject to sanction.

Attachment A — Core and customer satisfaction measures at a glance

Each printed fraction gives its numerator first and denominator second; population clauses printed above the fractions are retained below. The two PDF columns are placed in measure-number order.

Attachment A - WIA Core and Customer Satisfaction Measures-at-a-Glance Adult Measures

1. Entered Employment Rate
   Of those who are not employed at registration:

of adults who have entered employment by the

end of the 1st
quarter (Qtr.) after exit

of adults who exit during the Qtr.

2. Employment Retention Rate
   Of those who are employed in 1st
   Qtr. after exit:

of adults who are employed in 3rd

Qtr. after exit

of adults who exit during the Qtr.

3. Earnings Change in Six Months
   Of those who are employed in 1st
   Qtr. after exit:
   [Total Post-Program Earnings (earnings in Qtr 2
   +Qtr 3 after exit)] -[Pre-Program Earnings (earnings
   in Qtr. 2 + Qtr.3 prior to registration)]

of adults who exit during the Qtr.

4. Employment and Credential Rate
   Of those who received training services:

of adults who were employed in the 1st

Qtr. after
exit and received a credential by the end of 3rd Qtr.
after exit

of adults who exit during the Qtr.

Dislocated Worker Measures
5. Entered Employment Rate

of dislocated workers (DW) who have entered

employment by the end of the 1st
Qtr. after exit

of dislocated workers who exit during the Qtr.

6. Employment Retention Rate
   Of those who are employed in the 1st
   Qtr. after
   exit:

of DW who are employed in 3rd

Qtr. after exit

of DW who exit during the Qtr.

7. Earnings Replacement Rate
   Of those who are employed in the 1st
   Qtr. after
   exit:
   Total Post-Program Earnings (earnings in Qtr 2 +
   Qtr 3 after exit)
   Pre-Dislocation Earnings (earnings in Qtr. 2 + Qtr.3
   prior to dislocation)
8. Employment and Credential Rate
   Of those who received training services:

of DW who were employed in the 1st

Qtr. after
exit and received a credential by the end of 3rd Qtr.
after exit

of DW who exit during theQtr.

Older Youth (19-21 years old) Measures
9. Entered Employment Rate
Of those who are not employed at registration and who
are not enrolled in post-secondary education or
advanced training in the first quarter after exit:

of older youth (OY) who have entered

employment by the end of the 1st
Qtr. after exit

of OY who exit during the Qtr.

10. Employment Retention Rate
    Of those who are employed in the first quarter after exit
    and who are not enrolled in post-secondary education
    or advanced training in the third quarter after exit:

of OY who are employed in 3rd

Qtr. after exit

of OY who exit during the Qtr.

11. Earnings Change in Six Months
    Of those who are employed in the first quarter after exit
    and who are not enrolled in post-secondary education
    or advanced training in the third quarter after exit:
    [Total Post-Program Earnings (earnings in Qtr 2
    +Qtr 3 after exit)] -[Pre-Program Earnings (earnings
    in Qtrs 2 + 3 prior to registration)]

of OY who exit during the quarter

12. Credential Rate

of OY who were in employment, post-secondary

education, or advanced training in the first Qtr. after
exit and received a credential by the end of 3rd Qtr.
after exit

of OY who exit during the Qtr.

Younger Youth (14-18 years old) Measures
13. Skill Attainment Rate
Of all in-school youth and any out-of-school youth
assessed to be in need of basic skills, work readiness
skills, and/or occupational skills:
Total # of attained basic skills + # of attained WR
skills + # of attained Occ. skills
Total # of basic skills goals + # of WR skills goals
+ # of Occ. skills goals
14. Diploma or Equivalent Attainment Rate
Of those who register without a diploma or equivalent:

of younger youth (YY) who attained a secondary

school diploma or equivalent by the end of the 1st
Qtr. after exit

of YY who exit during the Qtr. (except those still

in secondary school at exit)
15. Retention Rate

of younger youth found in one of the following

categories in the 3rd
Qtr. after exit:
— post secondary education
— advanced training
— employment
— military service
— qualified apprenticeships

of YY who exit during the Qtr. (except those still

in secondary school at exit)
Across Funding Streams
16. Participant Customer Satisfaction
The weighted average of participant ratings on each
of the 3 questions regarding overall satisfaction
reported on a 0-100 scale.

17. Employer Customer Satisfaction
    The weighted average of employer ratings on each
    of the 3 questions regarding overall satisfaction
    reported on a 0-100 scale.

Attachment B — Common approach for negotiated performance levels

The matrix below retains each measure’s projected average, ETA’s basis for that projection, and its instructions for deriving a state average from JTPA data, in that order. PDF bullet glyphs extracted as “C” or “S” are shown as bullets.

Attachment B
Common Approach for Establishing Negotiated Levels of Performance

1. General Instructions
   States are requested to calculate State averages of past outcomes on the performance indicators where possible. These averages are to be calculated according to the following specifications:
   • Source data:
   - JTPA data for individuals who terminated from JTPA between October 1, 1997 to September 30,
2. This time period was chosen so that all states would have access to the necessary wage record data. Use of a common time period assures consistency among States.
   - Adult measures should be based on terminees from Title II-A.
   - Dislocated worker measures should be based on terminees from Title III, including substate and
   Governor’s Reserve programs.
   - Older youth measures should be based on terminees from Title II-C who were age 19 or older on the
   date of application.
   - Younger youth measures should be based on terminees from Title II-C who were under age 19 on
   the date of application.
   - Wage records data will be required for most measures. Wage record data should be acquired for:
   • The second and third quarters before application for adults and older youth
   • The second and third quarters before the date of dislocation for dislocated workers. NOTE: if the dislocation date is not available, use the application date instead; and
   • The first, second, and third quarters after termination for adults, dislocated workers and older youth.
   • Exclusions:
   • Individuals who received only objective assessment should be excluded from all calculations.
   • Individuals terminated with other termination codes for health/medical or institutionalized should be excluded from all calculations. This includes individuals whose other-termination code is institutionalized or health/medical (codes 1 or 2 in SPIR Item 39, Other terminations).
   • Dislocated workers served exclusively with National Reserve Funds should be excluded from all calculations
   • Entered employment measures for adults and older youth exclude individuals employed at application to JTPA. This data should be based on information collected from the participant and normally included in the JTPA data system in the field “labor force status.” It should not be based on wage records.
   • Inclusions
   • Individuals served by 5%-incentive funded programs are to be included even though they were exempt from JTPA performance standards. Detailed specifications for calculating measures are to be found in TEGL 7-99, Core and Customer Satisfaction Measures for the Workforce Investment System.
3. Matrix showing how projected averages are derived
   The column labeled ‘Projected Average’ presents projected average values for WIA performance indicators based on available data. The column labeled ‘Explanation ...’ describes how the projected averages were calculated. The column labeled ‘How to ...’ describes how to calculate a comparable past value for the State based on JTPA data or indicates other data that might be useful for the negotiation process. Matrix columns: performance indicator; projected average; explanation of average; how to determine a state average using JTPA data.

Adults

Entered employment — projected average: 71%

Basis for projected average: • Based on wage records for
7 States. Adjusted
upward by 1 percentage
point to account for
differences between the 7
States and all States.
State calculation or comparison: • Exclude individuals employed at application to JTPA.
• Calculated as the percentage with positive
earnings in the wage records in the quarter after
termination.
• See TEGL 7-99, Core and Customer Satisfaction
Performance Measures for the Workforce
Investment System, for instructions on how to
calculate performance on the measure.

Retention — projected average: 78%

Basis for projected average: • Based on wage records for
7 States. Adjusted
downward by 0.5
percentage point to
account for differences
between the 7 States and
all States.
State calculation or comparison: • Based on individuals employed (positive earnings in the wage records) in the quarter after termination.
• Calculated as the percentage with positive
earnings in the wage records in the third quarter
after termination.
• See TEGL 7-99, Core and Customer Satisfaction
Performance Measures for the Workforce
Investment System, for instructions on how to
calculate performance on the measure.

Earnings change — projected average: $3,700

Basis for projected average: • Based on wage records
for 7 States.
State calculation or comparison: • Based on individuals employed (positive earnings in the wage records) in the quarter after termination.
• Calculated as the average of [ total earnings in the
second and third quarters after termination minus
total earnings in the second and third quarters
before application ].
• See TEGL 7-99, Core and Customer Satisfaction
Performance Measures for the Workforce
Investment System, for instructions on how to
calculate performance on the measure.

Credential — projected average: 60%

Basis for projected average: • Assumes that the
definition of credential will
be somewhat flexible and
that about 80% of those
employed in the quarter
after exit will obtain a
credential.
State calculation or comparison: • Cannot be calculated from JTPA data.
• States should provide information on how they
plan to define credentials and on the extent to
which training programs are expected to lead to
credentials.
• See TEGL 7-99, Core and Customer Satisfaction
Performance Measures for the Workforce
Investment System, for instructions on how to
calculate performance on the measure.

Dislocated Workers

Entered employment — projected average: 77%

Basis for projected average: • Based on wage records for
7 States.
• Adjusted upward by 3
percentage points to
account for differences
between the 7 States and
all States.
State calculation or comparison: • Based on all dislocated workers (except NRA).
• Calculated as the percentage with positive
earnings in the wage records in the quarter after
termination.
• See TEGL 7-99, Core and Customer Satisfaction
Performance Measures for the Workforce
Investment System, for instructions on how to
calculate performance on the measure.

Retention — projected average: 85%

Basis for projected average: • Based on wage records for
7 States.
• Adjusted downward by 1
percentage point to
account for differences
between the 7 States and
all States.
State calculation or comparison: • Based on individuals employed (positive earnings in the wage records) in the quarter after termination.
• Calculated as the percentage with positive
earnings in the wage records in the third quarter
after termination.
• See TEGL 7-99, Core and Customer Satisfaction
Performance Measures for the Workforce
Investment System, for instructions on how to
calculate performance on the measure.

Earnings replacement — projected average: 92%

Basis for projected average: • Based on wage records for
5 States. Two of the 7
available States had
extremely high replacement
rates and were excluded
from the average.
State calculation or comparison: • Based on individuals employed (positive earnings in the wage records) in the quarter after termination.
• A. Calculate the sum (over all applicable
dislocated workers) of total earnings in the second
and third quarters after termination
• B. Calculate the sum of total earnings in the
second and third quarters before application.
• The measure is A divided by B.
• See TEGL 7-99, Core and Customer Satisfaction
Performance Measures for the Workforce
Investment System, for instructions on how to
calculate performance on the measure.

Credential — projected average: 60%

Basis for projected average: • Assumes that the
definition of credential will
be somewhat flexible and
that about 80% of those
employed in the quarter
after exit will obtain a
credential.
State calculation or comparison: • Cannot be calculated from JTPA data.
• States should provide information on how they
plan to define credentials and on the extent to
which training programs are expected to lead to
credentials.
• See TEGL 7-99, Core and Customer Satisfaction
Performance Measures for the Workforce
Investment System, for instructions on how to
calculate performance on the measure.

Older Youth (19–21)

Entered employment — projected average: 63%

Basis for projected average: • Based on wage records for
7 States. Adjusted
downward by 3 percentage
points to account for
differences between the 7
States and all States.
State calculation or comparison: • Exclude individuals employed at application to JTPA.
• Exclude individuals who entered non-Title-II
training and do not have earnings in wage records
for the quarter after termination.
• Calculated as the percentage with positive
earnings in the wage records in the quarter after
termination.
• See TEGL 7-99, Core and Customer Satisfaction
Performance Measures for the Workforce
Investment System, for instructions on how to
calculate performance on the measure.

Retention — projected average: 77%

Basis for projected average: • Based on wage records for
7 States
State calculation or comparison: • Based on individuals employed (positive earnings in the wage records) in the quarter after termination.
• Exclude individuals who entered non-Title-II
training and do not have earnings in wage records
for the third quarter after termination.
• Calculated as the percentage with positive
earnings in the wage records in the third quarter
after termination.
• See TEGL 7-99, Core and Customer Satisfaction
Performance Measures for the Workforce
Investment System, for instructions on how to
calculate performance on the measure.

Earnings change — projected average: $3,150

Basis for projected average: • Based on wage records for
7 States
State calculation or comparison: • Based on individuals employed (positive earnings in the wage records) in the quarter after termination.
• Exclude individuals who entered non-Title-II
training and do not have earnings in wage records
for the third quarter after termination.
• Calculated as the average of [ total earnings in the
second and third quarters after termination minus
total earnings in the second and third quarters
before application ].
• See TEGL 7-99, Core and Customer Satisfaction
Performance Measures for the Workforce
Investment System, for instructions on how to
calculate performance on the measure.

Credential — projected average: 50%

Basis for projected average: • Assumes that the
definition of credential will
be somewhat flexible and
that about 70% of those
employed in the quarter
after exit will obtain a
credential.
• Values are lower than for
adults and dislocated
workers because the
employment rate is lower.
State calculation or comparison: • Cannot be calculated from JTPA data.
• States should provide information on how they
plan to define credentials and on the extent to
which training programs are expected to lead to
credentials.
• See TEGL 7-99, Core and Customer Satisfaction
Performance Measures for the Workforce
Investment System, for instructions on how to
calculate performance on the measure.

Younger Youth (18 or less)

Goal attainment — projected average: 72%

Basis for projected average: • Calculated based on PY 97
SPIR data on the
attainment of youth
employment competencies.
It is the total number of
competencies (up to 3 per
youth) divided by the
number of types of
training received (up to 3
per youth, but at least
equal to competencies
attained). The value was
lower in PY 98 (66%).
State calculation or comparison: • Because the values likely to be achieved by the States and locals depend on the difficulty of achieving the goals set under WIA, it is not possible to calculate useful State values using JTPA data.
• See TEGL 7-99, Core and Customer Satisfaction
Performance Measures for the Workforce
Investment System, for instructions on how to
calculate performance on the measure.

Diploma attainment — projected average: 55%

Basis for projected average: • Based on separate
estimates for in-school
youth and for dropouts.
• For in-school youth a rate
of 75% is used. This is the
percentage of youth in
low-income families
(bottom 20% of family
incomes) nationwide who
complete high school,
which was derived from
the Current Population
Survey, October 1996.
• For dropouts JTPA data
on completion of major
level of education was
used, approximately 21%
for dropouts under 19 at
entry but 17 or older at
exit.
• These two numbers were
combined using JTPA data
on the split between in-
school youth and
dropouts.
State calculation or comparison: • Cannot be calculated from JTPA data.
• States may provide data on:

• The percentage of all low income youth who
  complete high school in the State (the
  national average is about 75%).
• The percentage of JTPA youth who
  completed a major level of education among
  those who were school dropouts at
  application (the national average is about
  21%). Exclude youth under 17 at
  termination to approximate the exclusion of
  youth still attending school.
• Expected relative levels of service to in-
  school youth and dropouts.
  • See TEGL 7-99, Core and Customer Satisfaction
  Performance Measures for the Workforce
  Investment System, for instructions on how to
  calculate performance on the measure.

Retention — projected average: 54%

Basis for projected average: • The percentage of youth
18 and under at entry and
17 and older at exit who
entered employment or
non-Title-II training
(advanced training or
postsecondary education)
derived from PY 97 SPIR
data.
State calculation or comparison: • A rough approximation can be calculated from JTPA data:
• Include all youth under age 19 at application and
age 17 or over at termination.
• Measure as the percentage who enter employment
or enter non-Title-II training.
• See TEGL 7-99, Core and Customer Satisfaction
Performance Measures for the Workforce
Investment System, for instructions on how to
calculate performance on the measure.

Customer Satisfaction

Participants — projected average: 68%

Basis for projected average: • Based on an analysis of
data from:
• Enterprise members
(76%)
• A comparison group
of non-Enterprise
members (74%).
• Early data from early
implementation States
(65% - 70%) that
show that State level
results tend to be
lower than the
SDA/SSA-level
enterprise results.
• Lower results than the
Enterprise surveys are
expected when youth are
added to the sample.
State calculation or comparison: • Results of similar customer satisfaction surveys conducted by the State.
• See TEGL 7-99, Core and Customer Satisfaction
Performance Measures for the Workforce
Investment System, for instructions on how to
calculate performance on the measure.

Employers — projected average: 66%

Basis for projected average: • Early data from early
implementation States
suggest that employer
results are likely to be 1 to
4 percentage points lower
than for participants.
State calculation or comparison: • Results of similar customer satisfaction surveys conducted by the State.
• See TEGL 7-99, Core and Customer Satisfaction
Performance Measures for the Workforce
Investment System, for instructions on how to
calculate performance on the measure.
• Beyond the State average, States may provide additional rationale (such as unique economic conditions, program designs or participant characteristics) along with valid data supporting the State’s position of increasing or decreasing their level of performance. The specifications outlined in the matrix are based on the following correspondence between WIA and JTPA data items:

Attachment C — Summary of important terms

Attachment C
Summary of Important Terms
Performance Measures:
The 17 performance indicators required by the Workforce Investment Act of 1998, section 136; Interim Final Rule, 20 CFR part 666, published at 64 Federal Register 18862 (April 15, 1999) to be used for determining return on investment and to be used in the incentives and sanctions process. Program Area: A cluster of measures used in the evaluation of performance for incentive purposes. There are four program areas: Adults, Dislocated Workers, Youth (both older and younger youth), and customer satisfaction (even though it is not technically a “program”). Expected Performance Level: The target performance level a State offers as the first step in the negotiation process. Negotiated Performance Level: The numeric performance target agreed to by the State and the region for each of the 17 core performance indicators. Negotiation Factor: Any information which may affect a State’s expected performance on any of the core performance measures that can be used in the negotiation process. The relevance, validity, and source of this information will be assessed during the negotiation process. Cumulative Program Area Score: The aggregate amount by which States exceed or fall below the negotiated performance levels in a particular program area. Incentive Threshold: To be eligible for incentives, a State must achieve at least 80% of the target performance levels on each performance measure, and have cumulative program scores greater than 100% in each program area. Sanction Threshold: States must achieve at least 80% of the target performance level on each of the 17 core indicators to avoid sanctions. To be subject to monetary sanctions, a State’s performance must fall below 80% of the target performance level on the same measure two consecutive years.