JTPA Title II and III performance standards for program years 1998 and 1999

U.S. Department of Labor
Employment and Training Administration
Washington, D.C. 20210
Classification: Perf. Standards
Correspondence symbol: TP
Issue date: June 30, 1998
Rescissions: None
Expiration date: Continuing
Directive: TRAINING AND EMPLOYMENT GUIDANCE LETTER NO. 12-97
To: ALL STATE JTPA LIAISONS; ALL STATE EMPLOYMENT SECURITY AGENCIES; ALL STATE WORKER ADJUSTMENT LIAISONS; ALL STATE ONE-STOP CAREER CENTER SYSTEM LEADS
From: DAVID HENSON, Director, Office of Regional Management
Subject: Job Training Partnership Act (JTPA) Title II and Title III Performance Standards for PYs 1998 and 1999

Purpose. To transmit additional guidance on JTPA Performance Standards pursuant to Training and Employment Guidance Letter No. 2-97, dated February 19, 1998.

References.

a. Job Training Partnership Act (JTPA) 29 USC 1501, et seq, as amended.

b. JTPA Regulations, 20 CFR Parts 626-629 and 631, published in the Federal Register on September 2, 1994.

c. Training and Employment Guidance Letter (TEGL) No. 2-97, dated February 19, 1998.

d. TEGL No. 2-95, dated August 10, 1995.

e. TEGL No. 1-94, dated August 31, 1994.

f. Training and Employment Information Notice (TEIN) No. 5-93, Change 1, dated June 23, 1994, and Change 2, dated January 24, 1997.

g. TEGL 8-97, dated April 23, 1998.

Background. This Guidance Letter provides additional information on the Secretary's required performance measures and the Secretary's implementing instructions for performance standards for Program Years (PYs) 1998 and 1999 (July 1, 1998-June 30, 1999; July 1, 1999-June 30, 2000). Sec. 106 of JTPA, as amended, directs the Secretary to establish performance standards for adult, youth, and dislocated orker programs. These standards may be updated every 2 years based on the most recent JTPA program experience, as well as program emphases and goals established by the Department of Labor. The Secretary also issues instructions for implementing these standards, and issues parameters for States to follow in adjusting the Secretary's standards for service delivery areas (SDAs) and substate areas (SSAs).

It should be noted that many States have requested and are operating under waivers granted under the FY 1997 and 1998 Department of Labor Appropriations Acts, and that some or all of those waivers may take precedence over some of the provisions in this Guidance Letter. After consideration of the requirements in this Guidance Letter, States may wish to consider requesting additional waivers and/or modifying existing waivers.

To assist the Department in assessing the JTPA operational environment and the constraints likely to influence State and local performance during the reference period, a Performance Standards Work Group was convened in March 1998. The Workgroup had representatives from State and local JTPA programs; and public interest groups, including the National Governors' Association, the U.S. Conference of Mayors, the National Association of Counties, and the National Council on the Aging. This Guidance Letter largely incorporates the Workgroup's recommendations.

Highlights of Changes from PY's 1996 and 1997. Following is a summary of major changes from PY 1996 and 1997 policy:

• JTPA performance standards and national numerical levels for those standards are aligned with system goals for performance and for performance improvement required under the Government Performance and Results Act (GPRA);

• States may continue to use the performance measures from PY's 1996 and 1997, or they may choose to replace some of them with new optional measures;

• All numerical levels in which there was an appreciable difference in average performance between PY l994 and 1995-96 are updated. Thus, numerical levels for five of the six core standards in Title II, as well as the required Title III standard, are raised; however, the level of the Youth Employability Enhancement Rate remains the same;

• Four new adult post-program measures based on Unemployment Insurance (UI) earnings records are introduced. These measures may be used instead of the current four adult measures based on telephone surveys;

• A Youth Positive Termination Rate may be used instead of both the Youth Entered Employment Rate and the Youth Employability Enhancement Rate;

• Three new optional Title III measures are introduced for future use as performance standards. Together with the Entered Employment Rate, these measures are being used to assess attainment of performance goals for Title III under GPRA;

• All Title III measures are consistently defined to exclude terminees who were called back or remained with their layoff employers;

• Overall failure to meet standards under Title II is defined as failing half or more of the Secretary's core standards;

• States are encouraged to develop incentive and sanction policies that differentiate clearly between minimally acceptable performance needed to avoid sanctions and performance worthy of reward; and

• States are encouraged to adopt higher than minimally acceptable performance levels to determine eligibility for performance awards, and to include in incentive policies criteria rewarding sustained high performance, improved performance and performance innovation.

Performance Management Goals for PY's 1998 and 1999. The GPRA requires federal programs to develop quantifiable measures of outcomes, which JTPA has been doing for many years. It also requires a commitment to continuous performance improvement in the outcomes for those measures. Continuous performance improvement is a new challenge for some programs in the JTPA system, but it is a challenge that JTPA is well-positioned to accomplish through its performance standards system.

In JTPA, performance standards have for many years marked the dividing line between minimally acceptable levels of program outcomes and unsatisfactory performance. State policy can also use performance standards to set a different, higher dividing line between rewardable levels of performance and satisfactory levels that do not fully contribute to the system's accomplishment of its performance goals. Policies that set higher levels of performance and performance improvement to qualify for incentive funds than the lower levels required to avoid sanctions have already been adopted in some States. The monetary incentives provided under JTPA have been and will continue to be strong motivators of organizational performance. Incentive funds can be used to motivate the system to meet the new requirements posed by GPRA while affording States continued flexibility in developing policies to meet these new challenges. The Employment and Training Administration (ETA) challenges all States to design their States' JTPA incentive policies to promote higher levels of performance and performance improvement.

Departmental goals for JTPA's performance management system serving disadvantaged adults and youth and dislocated workers were submitted to Congress in September 1997 in the ETA Strategic Plan for FY 1997-2002. The Strategic Plan commits to continuous performance improvement each and every year of the 5 years covered. Those goals have since been revised in the FY 1999 Annual Performance Plan (see TEGL 8-97) to support the Secretary's vision, facilitate increased coordination and foster greater cohesion within ETA and the Department. To achieve its GPRA goals for PY 1998/FY 1999 and its mission, ETA has established for JTPA the following three strategic goals along with associated outcome objectives and performance targets:

STRATEGIC GOAL NO 1

A Prepared Workforce: Enhance opportunities for America's Workforce.

• OUTCOMES

• • Increase employment, earnings and assistance.

  • Assist youth in making the transition to work.

  • Provide information and tools about work.

  • Provide information and analysis on the U.S. economy

PERFORMANCE TARGETS

• • 64% of JTPA adult disadvantaged terminees will be employed one quarter after program exit with average weekly earnings of $292.

  • 77% of JTPA Title II-C youth terminees will be employed or obtain advanced education or job skills.

STRATEGIC GOAL NO. 2

A Secure Workforce: Promote the economic security of workers and families.

• OUTCOME

• • Protect worker benefits.

  • Provide worker retraining.

PERFORMANCE TARGETS

• • Under JTPA Title III for dislocated workers, 74% of program terminees will be employed at an average wage replacement rate of 93% at termination and 76% will be employed one quarter after program exit at an average wage replacement rate of 97%.

STRATEGIC GOAL NO. 3

Quality Workplaces: Foster quality workplaces that are safe, healthy, and fair.

• OUTCOME

• • [None directly related to JTPA performance standards]

These performance targets merit special attention because they directly relate to the subject of this Guidance Letter: the Secretary's JTPA Title II and Title III performance measures, national numerical standards for these measures, incentive award criteria, and sanction requirements.

States have the authority to establish additional standards which reflect State goals, to develop the definitions of passing and failing individual standards, and to develop the specific approach to determining incentive awards. Data to support additional non-cost measures will continue to be reported, and States may use these measures or others in making State incentive award determinations for the up to 25% of incentive funds permitted for additional State standards under Section 106(e). Data on costs, together with program performance data, will provide critical information for State monitoring and fiscal oversight, and assist States in measuring returns on their human resource investments.

Secretary's National Standards for PY's 1998 and 1999. The Secretary's performance measures and national standards for Title II-A, Title II-C, section 204(d) Older Worker programs, and Title III [all of section 302(c)(1) State activities, and sections 302(c)(2) and 302(d) substate area activities] are as follows:

PY 1998 and 1999 Performance Standards Title II-A

Adult Follow-up Employment Rate:
60%
Adult Weekly Earnings at Follow-up:
$289
Welfare Follow-up Employment Rate:
52%
Welfare Weekly Earnings at Follow-up:
$255
or
Adult Pilot Sustained Employment Rate (optional): Baseline TBD
Adult Pilot Sustained Quarterly Earnings: (optional) Baseline TBD
Welfare Pilot Sustained Employment Rate (optional): Baseline TBD
Welfare Pilot Sustained Quarterly Earnings (optional): Baseline TBD

Title II-C

Youth Entered Employment Rate::
45%
Youth Employability Enhancement Rate: 40%
or
Youth Positive Termination Rate* (optional): 72%

Section 204(d) Older Worker Programs

Entered Employment Rate: 56%
Average Hourly Wage at Placement: $6.10

Title III

Entered Employment Rate: 73%
Follow-Up Employment Rate (optional):
72%
Average Wage Replacement Rate at Termination (optional): 91%
Average Wage Replacement Rate at Follow-Up (optional): 93%
Average Wage at Placement (optional):
$9.32

*Indicates Secretary's required "core" measures that are subject to Title II incentives and sanctions.

The four Title II-A adult and welfare follow-up measures will continue to be calculated based on outcomes data for individuals who terminate during the first three quarters of the program year and the last quarter of the previous program year. The four optional pilot Title II-A measures based on wage record data will be calculated on data for individuals who terminate in the program year.

Explanation of Performance Standards Levels. The Title II-A and II-C numerical standards were derived from PY 1995 and 1996 program data aggregated from the Standardized Program Information Report (SPIR). This approach for setting numerical levels is expected to produce local standards that are similar in difficulty to what SDAs/SSAs have experienced in the recent past. Past practice was to set numerical levels on non-earnings measures where at least 75% of SDAs can be expected to meet the standard, and to set levels for earnings measures where at least 60% of SDAs can be expected to meet the standard. The higher level was used for earnings measures to account for expected inflation in wage rates.

Welfare reform and the new Welfare-to-Work programs are expected to have continuing substantial impacts on JTPA Title II-A program outcomes. Because welfare clients make up one of the most significant hard-to-serve groups included in the adult measures, the effects of welfare reform are reflected in both the adult measures and the welfare measures. Changes in the mix of hard-to-serve characteristics of welfare recipients served in JTPA, the emphasis on immediate placement at any wage, and the "earnings disregard" practice in many States are impacting the types of services provided welfare clients and the outcomes achieved. This appears to be slowing the rate of gain in welfare clients' earnings and impacting the employment retention rate. Therefore, national levels for the Title II-A measures impacted by welfare reform were moderated for this performance standards cycle.

The actual levels of performance to be achieved by the JTPA system nationwide in PY's 1998 and 1999 are expected to be higher than the levels of the departure points, all other things being equal. The departure points are viewed as levels of minimally acceptable performance for local programs, and are therefore set at minimums ranging from the 20th to the 35th percentiles. The JTPA system's national average performance can be expected to approximate the 50th percentile of performance, all other things being equal. These distinctions are critical in any discussion of whether the national departure points are sufficient for JTPA to attain its GPRA performance goals.

Title II-A Adult Measures. The Adult Follow-up Weekly Earnings level was set at the 35th percentile, the level at which 65 percent of SDAs would be expected to meet or exceed the standard based on past experience. Past experience indicates that inflation would have the effect of raising above 65 percent the number of SDAs that meet or exceed the earnings standards. The Welfare Follow-Up Weekly Earnings level was continued at the 25th percentile; it had been moderated in the previous 2-year cycle to account for expected impacts of welfare reform. The level for the Welfare Follow-Up Employment Rate was set at the 20th percentile because feedback from local programs indicates that increasingly hard-to-serve welfare recipients required by welfare reform to take immediate employment are changing jobs and employers within the initial 13 week period. The level of the Adult Follow-Up Employment Rate was continued at the 25th percentile level, which results in a modest increase of 1 percentage point over the previous level and is believed to be an achievable yet challenging goal for Title II-A.

Title II-C Youth Measures. Because Title II-C funding was significantly scaled back several years ago, many smaller local areas are experiencing difficulty running programs serving both out-of-school youth and in-school youth, which typically have different expected outcomes. The optional Youth Positive Termination Rate adapts to the changing youth program mix by permitting positive outcomes from both strategies to be combined in one measure. The departure point for the Youth Positive Termination Rate was set at the 25th percentile.

The youth employability enhancement outcome has been clarified to include youth who go on for postsecondary education. The definition of "Entered Non-Title Training" now includes youth who go on to postsecondary education for at least one academic quarter. They can be counted as a positive termination for either the Youth Employability Enhancement Rate or the Youth Positive Termination Rate. Attachment 2 contains more complete guidance on youth employability enhancement definitions.

Title III Measures. Similar to the Title II-A and Title II-C standards, the required Title III standard for PY 1998 and 1999 was derived from PY 1995 and 1996 program data aggregated from the SPIR. This standard is also set at a level that approximately 75 percent of the substate areas can be expected to meet or exceed. There are no core measures for Title III because incentives and sanctions are not required in Title III programs.

There are four additional optional measures for Title III. The Average Wage at Placement has been optional in previous years, and will continue to be optional. The national departure point for this measure was set at a level that approximately 65 percent of the substate areas can be expected to meet or exceed.

The other three optional Title III measures, Follow-Up Employment Rate, Average Wage Replacement at Termination, and Average Wage Replacement at Follow-Up are newly introduced this performance standards cycle, although they have been used to track Title III performance goals under GPRA. National departure points for survey-based measures were set at the 25th percentile of performance, levels which about 75 percent of substate areas can be expected to meet or exceed.

States may choose to use wage record data instead of telephone surveys to track postprogram employment and earnings in Title III. Instructions on reporting Title III outcomes and computing performance will be issued separately at a later date.

Adjustment models and departure points will not be available for setting local wage-based follow up standards in PY l998. In future performance standards cycles, after developmental research is completed, ETA will provide adjustment models and departure points for the full complement of Title III performance standards measures whether they are based on surveys or UI wage records.

For consistency in performance measurement, all Title III measures have been defined to exclude terminees who were called back or remained with their layoff employers (see Attachment 1).

Section 204(d) Older Worker Program Performance Standards. Performance standards levels for the Section 204(d) Older Workers program were set at the 25th percentile of performance for both the Entered Employment Rate and the Average Wage at Placement. Programs operated under section 204(d) are State programs even though they may be managed by various local entities. Therefore, performance standards will be applied to each State's entire Section 204(d) Older Worker program. Unlike the adult and youth programs under Title II-A/C, however, no incentive awards or sanctions are associated with these standards.

Title II-A Optional Pilot Measures. The four Title II-A wage record-based measures may be substituted for the four follow-up measures. The State must specify that the four required core measures for Title II-A be either all of the required telephone follow-up measures or all of the optional pilot wage record-based measures. Of the State's specified core measures, at least one must be an adult employment measure, one must be an adult earnings measure, one must be a welfare employment measure, and one must be a welfare earnings measure.

For States that opt for pilot wage record-based post-program measures, there will be certain requirements that States must meet to implement this new approach to measuring employment retention. These requirements are necessary to allow ETA to manage the transitional dual reporting system that will result when some States adopt wage-based retention measures while other States continue with telephone survey follow up. The requirements are outlined in Attachment 6.

One requirement is that States choosing wage records as their data source during the next 2-year performance standards cycle need to ensure that they will be able to manage their local programs and fully comply with federal reporting requirements. Also, ETA plans to develop a mathematical algorithm to help explain variations in outcome levels between the two measurement systems. Therefore, it will be necessary for States to provide a 5 percent statewide sample of follow-up survey data for PY 1998 (and/or PY 1999, if applicable). Because the pilot retention measures reference the second full quarter after termination, and because there is a further delay of two to three additional quarters to obtain the wage record data, performance information for making incentive awards and imposing sanctions will not be available for a considerable period of time. States will be required to develop alternative incentive and sanctions policies to accommodate this lag time.

The optional wage record-based employment retention measures will be pilot-tested during PYs 1998 and 1999; therefore, the Department has established baseline goals instead of national departure points for measuring PYs 1998 and 1999 performance. In performance reports to the public, the Department plans to identify separately performance based on pilot measures and estimates of comparable national performance on follow-up measures.

ETA will issue implementing guidance for the pilot wage record-based sustained employment and earnings measures, including optional adjustment models, in a future directive. The Department will also develop a technical assistance plan during this pilot test period to provide assistance in the use of wage records for performance measurement, and at the same time, inform users about the implications for local performance results when using these pilot measures.

Optional Youth Positive Termination Rate. States may choose to adopt the single core Youth Positive Termination Rate to take the place of the two core measures Youth Entered Employment Rate and Youth Employability Enhancement Rate.

Implementing Provisions. The following implementing requirements must be followed:

a. Required Standards. For Titles II-A and II-C, States are required to set, for each SDA, a numerical performance standard for each of the core Secretary's measures; for the Older Worker program, States are required to set numerical Entered Employment Rate and Average Wage at Placement standards for programs operated under Section 204(d); for Title III, States are required to set for each substate area a numerical performance standard for the Entered Employment Rate; for Title III Governor's Reserve programs, States are required to set a statewide standard; and for Title III, States are also strongly encouraged to adopt and set numerical levels for all of the optional GPRA measures for substate programs and for Governor's Reserve programs.

b. Setting the Standards. Consistent with provisions in JTPA, States are required to adjust the Secretary's performance standards to reflect local area circumstances (Section 106(d)). Such adjustments apply to Title II-A, Title II-C, Section 204(d) and Title III programs, and must conform to the Secretary's parameters described below:

1. Procedures must be:

• Responsive to the intent of the Act,

• Consistently applied among the SDAs/SSAs,

• Objective and equitable throughout the State,

• In conformance with widely accepted statistical criteria;

2. Source data must be:

• Of public use quality,

• Available upon request;

3. Results must be:

• Documented,

• Reproducible; and

4. Adjustment factors must be limited to:

• Economic factors,

• Labor market conditions,

• Geographic factors,

• Characteristics of the population to be served,

• Demonstrated difficulties in serving thepopulation, and

• Type of services to be provided.

The Department offers States an optional adjustment methodology that conforms both to these parameters and to the requirement in section 106(d). This methodology covers Title II-A, Title II-C, Section 204(d), and Title III programs and will be provided to States in a future directive. Should the State choose to use an alternate methodology, or make adjustments not addressed by the Departmental model, it must conform to the parameter criteria and be documented in the Governor's Coordination and Special Services Plan for the program year to which it applies.

The State Job Training Coordinating Council and, where appropriate, the State Human Resources Investment Council must have an opportunity to consider adjustments to the Secretary's standards and to recommend variations. To determine whether an SDA has met or exceeded a performance standard, States must use actual end-of-year program data to recalculate the performance standards.

5. Performance Standards Definitions. States must calculate the performance of their SDAs, SSAs, and section 204(d) programs according to the definitions included in the Attachments.

6. Titles II-A and II-C Incentive Policies. ETA encourages States to develop incentive policies that encourage higher levels of performance, sustained high performance and performance innovation for local areas to receive incentive awards. (As discussed later, States should also develop parallel sanction policies that distinguish between unacceptable levels and those that are acceptable, but not rewardable.)

States are to develop and implement policies and procedures for awarding incentive grants in accordance with Section 106(b)(7). As the basis for making incentive awards, the State must use all and cannot "zero weight" any of the Secretary's core measures. At least 75 percent of the funds set aside for performance incentives must be related to these measures and, if applicable, the out-of-school youth and employer-assisted benefits criteria, in accordance with section 106(b)(7)(E). Up to 25 percent of the funds set aside may be used to reward performance on additional State standards, excluding cost standards. The following criteria are required in States' incentive award policies:

1. Standard for Service to Hard-to-Serve. A Secretary's standard for service to the hard-to-serve, as required by section 106(b)(7)(B) of JTPA, has been established in the form of a stand-alone eligibility criterion ("gate") for incentive awards.

In order for an SDA to be eligible to receive ANY incentive award, at least 65 percent of BOTH the SDA's (a) Title II-A AND (b) Title II-C (in-school and out-of-school youth combined) participants receiving training and/or other services beyond objective assessment must be hard-to-serve. The definitions of hard-to-serve are to be consistent with the definitions in sections 203(b), 263(b), and 263(d) of the Act.

2. Amount of Incentive Award. For those SDAs that successfully "pass through" the gate, (in addition to any funds set aside for State standards) the amount of the incentive award for SDAs exceeding the Secretary's performance standards will be determined by the State's policy in conformance with DOL requirements.

3. Exceeding All Core Standards. SDAs that pass through the "gate" and exceed all of the Secretary's Titles II-A and II-C core standards by the State's definition of exceeding standards must receive an incentive award.

4. Additional State Measures. States also may select additional non-cost measures, such as increased service to hard-to-serve participants, to include in incentive policies.

5. Cost Standards Not Allowed. Cost standards cannot be used for incentive award purposes. However, States are reminded of the integral role of financial reviews in program management. States are encouraged to explore ways of relating overall costs of job training to more direct measures of long-term employment, earnings and reductions in welfare.

6. Adjustments to Incentive Awards. Incentive policies may include adjustments to incentive award amounts based upon factors such as grant size, additional services to the hard-to-serve, intensity of service, and expenditure level.

7. Excluding Pilot Projects. In PYs 1998 and 1999, States will continue to have the authority to exclude pilot projects serving "hard-to-serve" individuals funded from the 5 percent incentive fund set-aside in computing their standards and actual performance. States and SDAs are encouraged to use such funds to develop or replicate model programs serving out-of-school youth.

NOTE: For those SDAs in which "incentive projects" are indistinguishable from those that provide general training, these programs would not be considered exempt from performance standards.

8. Incentives Using Pilot Measures. States using optional pilot wage record-based performance measures must develop incentive policies that also take into account the lag time in availability of wage record earnings data.

9. Additional Incentive Criteria. States are encouraged to include in their incentive policies criteria relating to: 1) attaining levels of performance and performance improvement that assist the JTPA system to attain its annual and strategic performance goals under the GPRA; 2) programs successfully serving out-of-school youth, and 3) placement in jobs providing employer-assisted benefits. Although successful programs for out-of-school youth remain the cornerstone of out-of-school incentives, SDAs will still be expected to exceed the 50 percent minimum service level to be rewarded nder that criterion. More information is included in the Attachments.

7. Titles II-A and II-C Technical Assistance and Sanction Policies. ETA encourages States to develop sanction policies that treat performance standards as minimal levels of acceptable performance. (States should also develop parallel incentive policies that encourage higher levels of performance and performance improvement for local areas to receive incentive awards. These policies should distinguish levels of performance that are rewardable from those that are acceptable but not rewardable.)

Determination of an SDA's failure to meet standards and the consequent imposition of technical assistance and reorganization requirements under section 106(j) will be based only on the Secretary's Title II-A and Title II-C core measures. "Meeting Performance Standards" overall is defined as meeting more than half of the core standards. Conversely, overall "failure to meet performance standards" is defined as failing half or more of the core standards. Definitions for meeting and failing individual standards will be established by States.

Regardless of whether the State opts for five or six core standards (four adult standards and either one or two youth standards), by these definitions overall failure to meet performance standards means failing three or more core standards. Further, meeting performance standards overall means meeting or exceeding at least four out of six core standards or at least three out of five core performance standards.

Overall failure to meet performance standards as defined above for the first year precludes an SDA from receiving any incentive awards and requires States to provide technical assistance to the underperforming SDA.

Overall failure to meet performance standards as defined above in the second consecutive year precludes an SDA from receiving any incentive award and requires States to impose a reorganization plan.

Furthermore, in the past, the overall effect of States' performance standards adjustment policies, including adjustments beyond the models, may have sustained or encouraged levels of performance outcomes lower than they otherwise might have been in the absence of Governor's adjustments. In some States, the effect of Governor's adjustments across the board may have been to lower all the local programs' performance standards levels below the level of minimally acceptable performance established by the national departure points. States also have the option to adopt additional adjustments that raise the levels higher than they otherwise would be, but that has not often happened in the program's experience.

Reducing standards in this way has the disadvantage of reducing the performance levels at which SDAs qualify for incentive awards in many states, and may possibly encourage and sustain lower levels of performance for all SDAs. ETA strongly encourages States that set policies that lower the national departure points across the board in the State to also adopt more challenging levels to determine eligibility for incentive awards.

Performance Status Summary Reports. Section 106(j)(3) requires each State to report to the Secretary, not later than 90 days after the end of each program year, the actual performance and performance standards for each SDA within the State. Within the same time frame, technical assistance plans developed by the State are required for each SDA failing to meet performance standards overall for the first year. The 90-day time frame also applies to the imposition of a reorganization plan, which is mandatory when an SDA fails to meet performance standards overall for a second consecutive year. Specific procedures for the formal performance standards report and required State action were provided in Training and Employment Guidance Letter 2-95, dated August 10, 1995.

However, in addition to the formal annual process, there should be ongoing oversight of SDA performance and continuous technical assistance and capacity-building aimed at addressing areas where program performance can be improved.

In the future, the submission of quarterly performance reports, in addition to annual reports, may be necessary in order to effectively demonstrate progress toward annual and strategic GPRA goals. While no quarterly submission is currently required in order to document actual performance, States should be aware of this possibility.

Action Required. The following actions are required of States:

a. Distribution. States must distribute this Guidance Letter to all officials within the State who need such information to implement the performance standards policies and requirements for PY 1998 and 1999.

b. State Plan Development. States must specify in the GCSSP their incentive award policy under section 202(c)(1)(B) and 202(c)(3)(A) and policy for imposition of sanctions under section 106(j). As the timing of this issuance may have precluded States from submitting this information with their PY 1998 and 1999 GCSSP, if necessary, States should submit a GCSSP amendment containing complete information on Title II incentive and sanction policies no later than August 31, 1998. (Such amendments are not required for Title III plans.)

Inquiries. Questions concerning this issuance may be directed to Valerie Lloyd at (202) 219-5487, ext. 107.

Attachments.

1. Definitions for Performance Standards

2. Youth Employability Enhancement Definitions

3. Rewarding Model Programs for Out-of-School Youth

4. Rewarding Placements in Jobs Providing Employer-Assisted Benefits

5. Incentive Policies that Promote Continuous Performance Improvement

6. Guidelines for Substitution of Wage Record Post-Program Measures in Lieu of Survey Follow-Up in JTPA Performance Standards for Program Years 1998 and 1999

Attachment 1: Definitions for Performance Standards

Official attachment: https://oui.doleta.gov/dmstree/tegl/tegl97/tegl_12-97a1.htm

TEGL 12-97 Attachment 1



DEFINITIONS FOR PERFORMANCE STANDARDS

Title II-A and II-C



Those terminees who receive only objective assessment and/or supportive services, regardless of whether they enter employment, are to be excluded from the calculation of performance outcomes for Title II-A, Title II-C, and section 204(d) older worker programs. Participants in special 5-percent-funded projects may, at the State's discretion, also be excluded from the calculation of performance outcomes for Title II-A and Title II-C.

The following defines the Title II-A performance standards:

1.  Adult Follow-Up Employment Rate:  Adult respondents who were employed (for at least 20 hours per week) during the 13th full calendar week after termination, divided by adult respondents (i.e., terminees who completed the follow-up interview).

2.  Adult Follow-Up Weekly Earnings:  The sum of weekly earnings for all adult respondents who were employed (for at least 20 hours per week) during the 13th full calendar week after termination, divided by adult respondents employed (for at least 20 hours per week) at the time of follow-up.

3.  Welfare Follow-Up Employment Rate:   Adult welfare respondents who were employed (for at least 20 hours per week) during the 13th full calendar week after termination, divided by adult welfare respondents (i.e., terminees who completed follow-up interviews). Welfare respondents include respondents reported as receiving TANF, GA or RCA at application.

4.  Welfare Follow-Up Weekly Earnings:  The sum of weekly earnings for all adult welfare respondents employed (for at least 20 hours per week) during the 13th full calendar week after termination, divided by adult welfare respondents employed (for at least 20 hours per week) at the time of follow-up.

    NOTE:  The Title II-A adult and welfare follow-up employment measures will continue to be based on individuals who terminate during the first three quarters of the program year and the last quarter of the previous program year. If the response rates for those employed at termination and those not employed at termination in an SDA differ by more than 5 percentage points in either the adult or welfare samples, then the calculations of the follow-up outcomes for that group must be modified to adjust for non-response bias. Individuals will be counted as completing the follow-up survey (respondents) if they answer the question on employment in the 13th week and, if employed, answer the questions on the hourly wage and weekly hours. Responses to the questions on weeks worked and working with the same employer are not needed to be counted as a respondent or to be included in the computation of the performance outcomes.

      (Optional) Adult Pilot Sustained Employment Rate:  Percentage of terminees with wage record earnings of at least $1339 (equivalent to 13 weeks x 20 hours x minimum wage) in the second full calendar quarter after termination.

      (Optional) Adult Pilot Sustained Quarterly Earnings:  Average earnings in the second full calendar quarter after termination for individuals with earnings of at least the amount required by the Adult Pilot Sustained Employment Rate definition.

      (Optional) Welfare Pilot Sustained Employment Rate:  Percentage of welfare recipient terminees with wage record earnings of at least $1339 (equivalent to 13 weeks x 20 hours x minimum wage) in the second full calendar quarter after termination.

      (Optional) Welfare Pilot Sustained Quarterly Earnings:  Average earnings in the second full calendar quarter after termination for welfare recipient individuals with earnings of at least the amount required by the Welfare Pilot Sustained Employment Rate definition.

      Note:  These optional wage record measures are to be based on individuals who terminate during the program year. In calculating these measures, adjustments for out-of-state employment (if out-of-state wage records are not obtained) and noncovered employment are required. A method for making these adjustments will be provided.

    The following defines the Title II-C performance standards:

5.  Youth Entered Employment Rate (YEER):  Youth who entered unsubsidized employment at termination (for at least 20 hours per week), divided by youth who terminated, excluding those potential dropouts who are reported (on the Standardized Program Information Report [SPIR]) as remained-in-school and dropouts who are reported (on the SPIR) as returned-to-school.

    NOTE:  As in past practice, youth terminees who remain-in-school or return-to-school and who also enter employment will not be excluded from the termination pool reflected in the denominator of the Youth Entered Employment Rate. However, only employment of at least 20 hours per week satisfies the requirement for "employment."

6.  Youth Employability Enhancement Rate (YEEN):  Youth who attained one of the employability enhancements at termination, whether or not they also obtained a job, divided by youth who terminated.

    Youth Employability Enhancements include:

    a.  

    b.  Attained (two or more) PIC-recognized Youth Employment Competencies.

    c.  Completed major level of education following participation of at least 90 calendar days or 200 hours in JTPA activity.

    d.  Entered and retained for at least 90 calendar days or 200 hours in non-Title II training or received a certification of occupational skill attainment or entered and retained in postsecondary education for at least one academic quarter.

    e.  Returned to and retained in full-time school (dropouts only) for one semester or at least 120 calendar days, attained a basic or job-specific skill competency, and made satisfactory progress.

        NOTE:  For the purposes of this outcome, and the remained in school outcome described below, "school" includes alternative schools, defined as a specialized, structured curriculum offered inside or outside of the public school system which may provide work/study and/or General Educational Development (GED) test preparation.

    f.  Remained in school for one semester or at least 120 calendar days (for youth at risk of dropping out of school), attained a basic or job-specific skill competency, and made satisfactory progress.

        NOTE:  For youth aged 14 and 15, the acceptable competencies will be basic skills or pre-employment/work maturity.

    g.  Entered postsecondary education full-time prior to termination in a program not funded under Title II of the JTPA, which builds upon and does not duplicate education/training received under Title II. NOTE: The participant must have been retained in that program for atleast one academic quarter. During the period the participant is in non-Title II postsecondary education, s/he may or may not have received JTPA Title II services.

7.  (Optional) Youth Positive Termination Rate:  The number of youth who had a positive termination (either entered employment of at least 20 hours per week or met one of the employability enhancement definitions) as a percentage of the total number of youth who terminated.



Section 204(d) Older Worker Program



The following defines Section 204(d) Older Worker program performance standards:

1.  Entered Employment Rate: Individuals who entered employment of at least 20 hours per week at termination, divided by terminations.

2.  Average Wage at Placement: Hourly wage rate of all terminees who entered employment of at least 20 hours per week at termination, divided by terminees who entered employment of at least 20 hours per week at termination.



Title III

The following defines the Title III performance standard and optional performance measures based on self-reported information. Comparable measures based on administrative data (UI wage records) are still being developed and definitions will be issued separately:

1.  Entered Employment Rate: Individuals who entered employment of at least 20 hours per week at termination, excluding those who were recalled or retained by the original employer after receipt of a layoff notice, divided by terminations, excluding those who were recalled or retained by the original employer after receipt of a layoff notice.

      (Optional) Follow-Up Employment Rate: Title III respondents who were employed (for at least 20 hours per week) during the 13th full calendar week after termination, divided by adult respondents (i.e., terminees who completed follow-up interviews). Individuals who were recalled or retained by the original employer are excluded from the follow-up sample.

      (Optional) Average Wage Replacement Rate at Termination: The average of the wage replacement rates calculated for each individual with both a reported dislocation wage and a reported wage at termination. The wage replacement rate for the individual is the wage at termination divided by the dislocation wage. Note: this measure is not equal to the ratio of the average wage at termination to the average dislocation wage. This measure is based on individuals who enter unsubsidized employment of at least 20 hours per week and excludes those who are recalled to or remain with the layoff employer.

      (Optional) Average Wage Replacement Rate at Follow-Up: The average of the wage replacement rates calculated for each individual with both a reported dislocation wage and a reported wage at follow-up. The wage replacement rate for the individual is the wage at follow-up divided by the dislocation wage. Note: this measure is not equal to the ratio of the average wage at termination to the average follow-up wage. This measure is based on individuals with follow-up employment of at least 20 hours per week. Individuals who were recalled or retained by the original employer are excluded from the follow-up sample.

      (Optional) Average Wage at Placement: The sum of wages at placement for all Title III terminees who entered employment of 20 hours or more, excluding those who remained with or were recalled to the layoff employer, divided by the number of Title III terminees, excluding those who remained with or were recalled to the layoff employer.

      NOTE:  As indicated in the definitions listed above, for performance standards purposes, the term "employment" means employment for 20 or more hours per week. For determining compliance with this provision, a "week" means a period of 7 consecutive days, and the 20 or more hours is to be understood as a condition of the employment. No formal verification is required, but the Department encourages States to set up a system that would, at a minimum, provide for random checking to assess compliance by SDAs.

Attachment 2: Youth Employability Enhancement Definitions

Official attachment: https://oui.doleta.gov/dmstree/tegl/tegl97/tegl_12-97a2.htm

TEGL 12-97 Attachment 2



YOUTH EMPLOYABILITY ENHANCEMENT DEFINITIONS



"Youth Employability Enhancement" means an outcome for youth, other than entered unsubsidized employment, which is recognized as enhancing long-term employability and contributing to the potential for a long-term increase in earnings and employment.

Outcomes which meet this requirement shall be restricted to the following:

- (1)  Attained PIC-Recognized Youth Employment Competencies (two or more reported from SPIR);

  (2)  Returned to Full-Time School;

  (3)  Remained in School;

  (4)  Completed Major Level of Education; or

  (5)  Entered Non-Title II Training.

1.  Attained PIC-Recognized Youth Employment Competencies - A youth who demonstrated proficiency, as defined by the PIC in two or more of the following three skill areas in which the terminee was deficient at enrollment: 1) pre-employment/work maturity; 2) basic education; or 3) job-specific skills. Competency gains must be achieved through program participation and be tracked through sufficiently developed systems that must include: quantifiable learning objectives, related curricula/training modules, pre and post-assessment, employability planning, documentation, and certification. The completely detailed definition for Youth Employment Competency systems is located in the Standardized Program Information Reporting System (SPIR) instructions.

2.  Returned to Full-Time School - A youth who: (1) had returned to full-time secondary school (e.g., junior high school, middle school and high school) including alternative school if, at the time of intake, the participant was not attending school (exclusive of summer school) and had not obtained a high school diploma or equivalent; and (2) prior to termination, had been retained in school for one semester or at least 120 calendar days after becoming a participant in the JTPA program.

    - Alternative School - A specialized, structured curriculum offered inside or outside of the public school system which may provide work/study and/or GED preparation.

      NOTE: To obtain credit for Returned to Full-Time School and Remained in School (described below), SDAs must be prepared to demonstrate that retention results from continuing, active participation in JTPA activities and the youth must: (1) be making satisfactory progress in school; and (2) (for youth aged 16-21) attain a PIC-approved Youth Employment Competency in Basic Skills or Job-Specific Skills; and (3) (for youth aged 14-15) attain a PIC-approved Youth Employment Competency in Pre-employment/Work Maturity or Basic Skills.

      Satisfactory Progress in School - An SDA, in cooperation with the local school system, must develop a written policy which defines an individual standard of progress that each participant is required to meet. Such a standard should, at a minimum, include both a qualitative element of a participant's progress (e.g., performance on a criterion-referenced test or a grade point average) and a quantitative element (e.g., a time limit for completion of the program or course of study). This policy may provide for exceptional situations in which students who do not meet the standard of progress, because of mitigating circumstances, are nonetheless making satisfactory progress during a probationary period.

3.  Remained in School - A youth who, prior to termination, had been retained in a full-time secondary school, including alternative school, for one semester or at least 120 calendar days after becoming a participant in the JTPA program. The youth must be attending school at the time of intake, have not obtained a high school diploma or equivalent, and be considered "at risk of dropping out of school" as defined by the Governor in consultation with the State Education Agency.

4.  Completed Major Level of Education - An adult or youth who, prior to termination, had completed, during enrollment, a level of educational achievement which had not been reached at entry. Levels of educational achievement are secondary and postsecondary. Completion standards shall be governed by State standards and shall include a high school diploma, GED Certificate or equivalent at the secondary level, and shall require a diploma or other written certification of completion at the post-secondary level.

      NOTE: Completion of a major level of education must result primarily from active JTPA program participation of at least 90 calendar days or 200 hours, usually prior to such completion.

5.  Entered Non-Title II Training - An adult or youth who, prior to termination, had entered an occupational skills employment/training program or postsecondary education not funded under Title II of the JTPA, which builds upon and does not duplicate training received under Title II. NOTE: The participant must have been retained in that program for at least 90 calendar days or 200 hours or one academic quarter or must have received a certification of occupational skill attainment. During the period the participant is in non-Title II training, s/he may or may not have received JTPA services.

Attachment 3: Rewarding Model Programs for Out-of-School Youth

Official attachment: https://oui.doleta.gov/dmstree/tegl/tegl97/tegl_12-97a3.htm

TEGL 12-97 Attachment 3



Rewarding Model Programs for Out-of-School Youth



Improving the labor market outcomes achieved by out-of-school youth is a continuing priority. The Department encourages States in their incentive policies to reward out-of-school youth programs that are identified by the Department or recognized by the State as having a demonstrated record of success.

States need to develop ways to identify such programs. Possible approaches include:

- Using outcomes achieved to identify successful programs. Outcomes could include both the two youth performance measures and measures such as learning gains and earnings/retention in full-time employment.

- Alternatively, States could offer SDAs "seed money" from incentive funds to plan/operate programs that provide innovative or high-quality training to out-of-school youth based on criteria established by the State. Examples of such criteria include training that integrates occupational and basic skills training, and training that emphasizes acquiring job skills in demand in the emerging workplace.

NOTE:  Whatever method is used to reward successful out-of-school youth programs, access to such incentives must be limited to those SDAs that serve in excess of 50 percent out-of-school youth in their overall Title II-C program.

Attachment 4: Rewarding Placements in Jobs Providing Employer-Assisted Benefits

Official attachment: https://oui.doleta.gov/dmstree/tegl/tegl97/tegl_12-97a4.htm

TEGL 12-97 Attachment 4



Rewarding Placements in Jobs Providing Employer-Assisted Benefits

The Department encourages States in their incentive policies to reward SDAs for placements in jobs with employer-assisted benefits, including health benefits. Rewarding such placements is intended to increase the focus on overall job quality. To include placements in jobs with employer-assisted benefits in their incentive policies, States will need to:

1.  

2.  Specify how the criterion will be measured. The definition must be consistent with that for SPIR Item 35c (i.e., consisting of, at a minimum, health insurance benefits and coverage under Social Security or an equivalent pension plan). Note that it is not necessary for an individual to actually receive benefits when employment begins as long as they are an acknowledged component of employment conditions. For example, health benefits available after a waiting-period and benefits that are refused because of availability from another source both count as employer-assisted benefits. Examples of measures of jobs with benefits, using data from SPIR Item 35c are:

    - - among terminees who enter employment, the percent who are in jobs providing employer-assisted benefits; and

      - among all terminees, the percent who enter employment and are in jobs providing employer-assisted benefits.

3.  Determine how to reward placement in jobs with employer-assisted benefits. One possible approach would require the State to determine a departure point or benchmark to use in setting rewards levels for benefits. This departure point would serve the same function served by the numerical national standards for the Secretary's core measures. It would be the level that, before any adjustment for local factors, identifies rewardable performance. Under this approach, a State would also decide whether and how the departure point should be adjusted for local clientele and economic conditions when setting reward levels for each SDA.

    Rather than setting reward levels for employer-assisted benefits using a process akin to setting standards for the Secretary's core measures, States may choose a simpler approach. Because there are no sanctions based on this criterion, it is not absolutely necessary to set a minimally acceptable performance level. It is possible to reward SDAs at all levels of performance on employer-assisted benefits. One way to do this would be to set aside a portion of incentive funds to reward such placements. This reward pool could be divided among SDAs based on their proportionate share of all placements in jobs with employer-assisted benefits. This procedure is equivalent to giving a fixed amount for each placement (i.e., the amount is the value of the pool divided by the total number of placements into jobs with benefits in the State).

    Because this approach may be viewed as favoring SDAs in areas where benefits are widely prevalent, it should be viewed as interim until better data on employer-assisted benefits are available.

Attachment 5: Incentive Policies that Promote Continuous Performance Improvement

Official attachment: https://oui.doleta.gov/dmstree/tegl/tegl97/tegl_12-97a5.htm

TEGL 12-97 Attachment 5



Incentive Policies that Promote Continuous Performance Improvement



There are many methods of setting incentive targets that would promote performance improvement each year under ETA's Strategic Plan required by the Government Performance and Results Act (GPRA). Among these methods are: 1) States substitute a new, higher departure point in the performance standards worksheets, and 2) States set a challenging State performance target and reward according to the extent to which SDAs meet target or make progress toward meeting it.

States adopting either of these methods would be adopting a definition of "exceeding a performance standard" that requires performance above the level of performance specified as minimally acceptable and non-sanctionable. Thus, SDAs with performance at or above the performance standard, but below the incentive target (or rewardable level), would be considered as "meeting performance standards". All SDAs with performance in the range between the standard (the sanctionable level) and the rewardable level would be treated as merely meeting the performance standard. Only SDAs with performance above the rewardable level would be considered as "exceeding" performance standards.

Because only SDAs exceeding the rewardable level are defined as "exceeding" standards, the State is not required to provide incentives to SDAs with performance above the standard but below the rewardable level, even if that level of performance is achieved for all of the Secretary's core standards. Incentives are required only for SDAs that exceed all core standards by the Governor's definition of exceeding, which for these methods requires performance above the rewardable level.

For more information on developing incentive policies that reward performance at separate, higher levels than sanctionable levels, see the Guide to JTPA Performance Standards for Program Years 1996 and 1997, Chapter V, pages 15 to 34.

Method A: Adopting a Higher Departure Point.  In the first method, a higher departure point is substituted for the national departure point in the performance standards worksheets just for the purpose of determining which local areas merit performance rewards on each measure (while continuing to use the lower national departure point for sanctions purposes). The higher departure point for incentives could be set at: 1) the GPRA goal level for the appropriate measure, or 2) a percentile of national performance higher than that used for establishing the national departure points (percentiles of national performance are supplied in the Guide to JTPA Performance Standards).

To illustrate the first method using the GPRA goal level for the Adult Follow-Up Employment Rate: in the performance standards worksheet, the national departure point of 60% would be replaced by 64% (the GPRA goal level) for PY 1998, and by 65% for PY 1999. The effect would be to raise the local area's standard calculated by the worksheet to a higher, rewardable level. If the local area achieved the higher level, it would qualify for incentives attached to this measure in the State's policy (all other conditions being met).

Method B: Setting State Performance Improvement Targets.  The second method of using incentives to encourage performance improvement would be for the State to set a target performance improvement level for the State. It would then allocate various levels of performance or performance improvements calculated to achieve the State's target among the local areas. States with average performance at or above the national average could set a performance improvement target equivalent to the national rate of performance improvement implicit in the GPRA goal. States with performance below the national average should aim for a higher rate of performance improvement than the rate implicit in the GPRA goals.

For example, a State with average performance on the Adult Follow-Up Employment Rate of 65% in PY 1996 could say its PY 1998 target is 67%, an increase equivalent to the increase in the national average from 62% in PY 1996 to the GPRA goal of 64% in PY 1998. The State could then say that high-performing SDAs need achieve a lesser rate of performance improvement, and low-performing SDAs must achieve a higher rate of performance improvement, so that the overall effect statewide would be to achieve a 67% rate.

Attachment 6: Guidelines for Substitution of Wage Record Post-Program Measures

Official attachment: https://oui.doleta.gov/dmstree/tegl/tegl97/tegl_12-97a6.htm

TEGL 12-97 Attachment 6



Guidelines for Substitution of Pilot Wage Record-Based Retention Measures in Lieu of Telephone Follow-Up Measures in JTPA Performance Standards for Program Years 1998 and 1999.



The following are minimal guidelines for States to follow when opting to use Unemployment Insurance (UI) wage record data in lieu of telephone surveys for pilot employment retention performance measures for Title II-A or Title III. These guidelines provide some uniformity in system wide measurement when States are using different data sources, and also afford States flexibility in defining measures and setting benchmarks which best meet State needs.

The consolidated planning guidance for PY 1998 and 1999 (TEGL 2-97) announced that optional post-program performance measures will be offered for States wishing to use wage records to track employment retention and earnings. These guidelines provide a general outline of the conditions under which DOL will permit optional performance standards based on wage record post-program measures.

1.  Universal Reporting. State agrees to run a match of the records of all terminees to wage records rather than using a sample.

2.  Reporting Elements, Formats and Electronic Media. State agrees to provide data on all terminee matches according to the specified elements, formats and electronic media (similar to current Standardized Program Information Report instructions). In cases of multiple employers in the referenced period, the State may choose either: 1) reporting of multiple records for each terminee; or 2) creation of one record for each terminee that sums the employment data for all employers.

3.  Reporting Deadline. State agrees to report termination data for all terminees by the Department's reporting deadline, August 15th following the subject year (August 15th, 1999, for Program Year 1998). State further agrees to submit a second report of post-program outcomes data for all terminees by the Department's deadline, August 15th of the year following the subject year (August 15th, 2000, for Program Year 1998).

4.  Out-of-State Placements. State agrees to make good faith efforts to obtain and report data on out-of-state JTPA placements, including one or more of the following:

    a.  implementing data sharing agreements with a majority of contiguous States;

    b.  implementing data sharing agreements with States known to account for a majority of out-of-State JTPA placements; and/or

    c.  participating in an operational national data sharing arrangement, if one becomes a functional reality.

5.  Sample Data and Prior Year Data for National Measures. State agrees to conduct a follow-up survey of a statewide sample of 5% of terminees during the pilot period (PYs 1998 and 1999) of the wage record-based retention measures and to report the data according to Department's specifications. States are also requested to provide both wage record and survey follow-up data for a recent year, such as PY 96, if possible. This will help the Department develop mathematically equivalent performance measures, appropriate adjustment methodologies, and summary national performance information. (It will not be possible to develop State level survey-based outcomes from the 5% sample.)

6.  Adjustments to Performance Standards. State agrees to adjust SDAs' performance standards to account for local variations in economic conditions, client characteristics, and non-covered employment.

7.  Title IIA Optional Performance Standards. While it is possible to capture hourly and weekly earnings with follow-up surveys, it is only possible to capture total quarterly earnings using wage records, which must serve as both a proxy measure of employment as well as earnings. The following measures were developed for Title II to accommodate the fundamental differences between the two data sources. Therefore, the State agrees to use the following performance measures as the Secretary's Title II core performance measures in place of follow-up survey-based measures:

    - Adult Pilot Sustained Employment Rate: Percentage of terminees with wage record earnings of at least $1339 (equivalent to 13 weeks x 20 hours x minimum wage) in the second full calendar quarter after termination.

      Adult Pilot Sustained Quarterly Earnings:  Average earnings in the second full calendar quarter after termination for individuals with earnings of at least the amount required by the Adult Pilot Sustained Employment Rate definition.

      Welfare Pilot Sustained Employment Rate: Percentage of welfare recipient terminees with wage record earnings of at least $1339 (equivalent to 13 weeks x 20 hours x minimum wage) in the second full calendar quarter after termination.

      Welfare Pilot Sustained Quarterly Earnings: Average earnings in the second full calendar quarter after termination for welfare recipient individuals with earnings of at least the amount required by the Welfare Pilot Sustained Employment Rate definition.

    These measures are considered interim and transitional, and are expected to be modified by either requirements in new legislation or further refinement of the JTPA performance standards system. States may also choose to set higher earnings levels for making incentives.

    The Department will issue preliminary baseline performance goals and adjustment models for these pilot wage record-based measures in the near future. The baseline goals and adjustment models will be developed using data from the 1991 wage record-based demonstration projects. Baseline performance levels are being issued as "goals" rather than "standards" since the available data is inadequate for establishing national standards. The 1991 demonstration projects included only 10 states and at that time out-of-state and excluded employment (such as self-employment or non-covered employment) were not included in the project results.

    The adjustment models developed for the PY 1998/1999 pilot measures will take into account the necessary adjustments for out-of-state and excluded employment. National standards will be developed when enough data are available to provide valid and reliable departure points.

8.  Alternative Incentives and Sanctions Policy. State agrees to develop for Departmental approval an alternative incentives and sanctions policy to cope with the lag time in availability of performance data. ETA will develop several optional alternative scenarios for possible State adaptation.

9.  Ability to Manage Programs and Report Performance. States choosing wage records as their data source during the next two-year cycle need to ensure that they will be able to adequately manage their local programs and fully comply with federal reporting requirements.

10. Other Provisions Deemed Appropriate. The Department and/or the State may include other provisions deemed appropriate.

Revised 6/29/98