Revising WIA negotiated performance levels
Is this still the rule?
This document has been rescinded. Ezel checks what federal law requires today, under current federal labor law, and answers your situation with citations.
Employment and Training Administration
Advisory System
U.S. Department of Labor
Washington, D.C. 20210
CLASSIFICATION: WIA/Performance Levels
CORRESPONDENCE SYMBOL: OWS
DATE: February 12, 2002
TRAINING AND EMPLOYMENT GUIDANCE LETTER NO. 11-01
TO: ALL STATE WORKFORCE LIAISONS
ALL STATE WORKFORCE AGENCIES
ALL STATE WORKER ADJUSTMENT LIAISONS
ALL ONE-STOP CAREER CENTER SYSTEM LEADS
/S/
FROM: EMILY STOVER DeROCCO
Assistant Secretary
SUBJECT: Guidance on Revising Workforce Investment Act (WIA) State Negotiated Levels of Performance
-
Purpose. This guidance letter describes criteria and
procedures the Department of Labor (DOL) will use when considering state requests to revise negotiated levels of performance established under title I-B of WIA. -
References. The Workforce Investment Act of 1998, section 136;
20 CFR part 666; 20 CFR 661.230(b)(2); Training and Employment Guidance Letter (TEGL) No. 8-99, March 3, 2000. -
Background. Under WIA title I, subtitle B, each state
workforce agency must negotiate expected levels of performance with the Department for the adult, dislocated worker, and youth programs, as well as customer satisfaction (a total of 17 measures).
Section 136(b)(3)(A)(vi) of the Act permits each Governor to request revisions to the state negotiated levels of performance in the event “unanticipated circumstances arise in a state resulting in a significant change in the factors” described in section 136(b)(3)(A)(iv)(II) of the Act. These factors include the economic conditions for the state, the characteristics of participants, and the services provided to participants.
RESCISSIONS: [blank in source]
EXPIRATION DATE: Continuing
In September 2001, the Department’s Employment and Training Administration (ETA) convened a meeting of state and local representatives to explore criteria for revising state negotiated levels of performance. A workgroup, created as a result of this meeting, then provided additional input over the next two months. This guidance letter largely reflects the contributions of the stakeholders involved in these forums.
-
Policy on Requesting Revisions. The Department views the
process of establishing and revising negotiated levels of performance as a way to promote performance accountability and continuous improvement. The Governor may request a revision to one or more negotiated performance levels at any time prior to the end of the program year for which the revised level(s) would apply. This policy allows adjustments to negotiated performance goals in order to account for changes in economic conditions, changes in the characteristics of the participants served by the program, and changes in service delivery design. The Department expects a request for revisions to performance levels to be submitted by a state workforce agency as soon as possible after the identification of the unanticipated circumstance. The Department will not consider requests to revise PY 2000 state negotiated levels of performance. -
Submitting Requests for Revised Levels of Performance. Each
request submitted by a state workforce agency for a revision to one or more performance levels will be reviewed by the ETA Regional Office based on its own merits.
1) The Request. Each state workforce agency seeking a revision
will develop and submit a written request to the ETA Regional
Office serving the state.
2) Review of the Request. The ETA Regional Office will review the
request and determine whether the state request appropriately
documents and satisfies each of the following three conditions
listed below. Each of these conditions is discussed in more
detail in Attachment I.
a) Condition 1. The request must articulate an unanticipated
circumstance.
b) Condition 2. The unanticipated circumstance, in turn,
must impact one or more factors. There should be evidence
to associate the unanticipated circumstance with an actual
change in one or more of the factors.
c) Condition 3. The stated variations from expected outcomes
identified in the request must be linked to the stated
significant changes in factors. There should be
sufficient and appropriate documentation to explain and
justify the proposed revised levels of performance.
3) The Negotiation. The revised levels of performance will be
negotiated between the ETA Regional Office and state staff to
ensure that reasonable and appropriate levels are set. The ETA
Regional Office will identify where data or information is
insufficient to justify the request. Both the ETA Regional
Office and state staff should work together to ensure the
necessary information is included and considered when
processing the request.
4) The Decision. The ETA Regional Office will convey the results
of the review to the state workforce agency within 30 working
days after receipt of the written request. All approved
revisions will be incorporated into the State Plan through a
plan modification. Please note that modifications to the State
Plan are subject to the same public review and comment
requirements that apply to the development of the original
State Plan. The ETA Regional Office will maintain all
documentation supporting its decision.
Approved revised levels of performance will be effective on the
date agreement is reached between the ETA Regional Office and
the state workforce agency. In the event one or more of these
revised levels are changed afterwards as a result of the public
review and comment requirement, the impacted request and
decision will be reconsidered by the ETA Regional Office.
States should direct any questions about the plan modification
process to their ETA Regional Office.
-
ETA’s Review Process. When determining whether sufficient
information is available to document and satisfy the three conditions, the ETA Regional Office will look for:• A description of the nature of the problem or mitigating
circumstance, including a description of when the
unanticipated circumstance occurred and its duration or
expected duration (Condition 1).• The performance measure(s) and program year(s) affected by
the request (Condition 2).• Evidence of the change in the factors taken into account in
the earlier negotiation(s), the forecasted factor values, if
appropriate, and the estimated impact on the performance
outcome(s) (Condition 2 and Condition 3).• A description of the approach(s) used to determine revised
levels of state negotiated performance, including methods
used to forecast annual factor values, if appropriate
(Condition 3).• A description of the data source(s) used to demonstrate
change in the factors and a description of the data sources
used to forecast values for the factors, if appropriate
(Condition 3).• The computations for the revised performance level(s) included
in the request (Condition 3).
As noted, this guidance for justifying revisions to negotiated levels of performance reflects input from a workgroup of state and local officials. Attachment I summarizes in greater detail each of the three conditions cited earlier and largely reflects comments from the stakeholders’ workgroup. Attachment II describes one approach that state workforce agencies may find useful when calculating appropriate levels of performance. ETA Regional Offices will use these attachments as a guide when reviewing requests. States may also want to consider them when analyzing performance levels and developing appropriate requests for revisions.
-
Action. States should distribute this guidance letter to all
officials within the state who need such information to implement the performance accountability policies under the WIA. -
Inquiries. Questions concerning this issuance should be
directed to your appropriate ETA Regional Office. -
Attachment. Attachment I – Relating the Three Conditions for
Revising Levels of Negotiated Performance; Attachment II – The Weighted Average Approach to Revising Levels of Negotiated Performance.
Attachment I (official PDF: https://www.dol.gov/sites/dolgov/files/ETA/advisories/TEGL/2002/TEGL11-01Attach1.pdf)
Attachment I
Relating the Three Conditions for Revising Levels of
Negotiated Performance
Background
Section 136(b)(3)(A)(vi) of the Workforce Investment Act (WIA)
specifies that a Governor may request revisions to state negotiated
levels of performance in the event “unanticipated circumstances
arise in a state resulting in a significant change in the factors”
that include factors described in section 136(b)(3)(A)(iv)(II) of
the Act. Additionally, 20 CFR 666.130(a) states the change in
factors is to be determined from when the initial plan was submitted
and approved. These factors are widely accepted variables known to
impact resulting outcomes on one or more of the 17 WIA performance
measures. These factors include differences in economic conditions,
the characteristics of participants, and the services provided.
There are three conditions for any state workforce agency request
for revisions to state negotiated levels of performance to be
approved by the ETA Regional Office. These conditions are
graphically displayed in a logic model below.
Relating the Three Conditions
Condition 1 Condition 2 Condition 3
Unanticipated Demonstrated Factor(s) Known to Impact
Circumstance Change in the Performance and Revised
Occurs Factor(s) Levels can be Objectively
Determined
The process for The unanticipated event
requesting revisions to must impact one or The change in the factor(s), in turn,
earlier negotiated levels more of the factors must be related to performance on
of performance is taken into consideration one or more of the measures. The
triggered by an in the earlier degree of the impact on
unanticipated negotiations of performance may be estimated
circumstance. The performance levels. It using historical data.
unanticipated must be reasonable to
circumstance must associate the event with
occur for a request to be a change in the
considered by the factor(s).
Secretary.
Condition 1: Unanticipated Circumstances
Actual performance on the 17 WIA title I measures of performance
varies above or below expected levels of performance for a number of
reasons. Variations from expected outcomes can also result from
significant changes in factors that are beyond the control of the
state workforce agency. In these situations, state workforce
agencies should consider submitting requests to revise negotiated
levels of performance impacted by the significant change in factors.
Variations from expected outcomes can occur from significant changes
in administrative practices within the control of the state
workforce agency. Changes in expected performance under these
circumstances do not merit revisions to negotiated levels of
performance. By themselves, these changes would not be considered
the result of “unanticipated circumstances.”
Examples of unanticipated circumstances are listed below. Please
note the list of unanticipated circumstances is not exhaustive and
does not constrain a state workforce agency from documenting its own
experiences that have prompted the need to seek a revision.
• Limitations in the baseline data used to project performance
levels for program years 2001 and 2002. The baseline data
used in the initial negotiations of these performance levels
were based on actual and estimated outcomes attained by
former JTPA participants. The data sources and collection
methods may be very different than those found under WIA,
warranting possible corrections to baseline data.
• Changes in the characteristics of program participants, such
as service to hard-to-serve customer groups, not taken into
account in the earlier negotiation(s) on state levels of
performance.
• Changes in economic conditions, such as plant closings and
mass layoffs.
• Changes in economic assumptions and outlooks.
• Disasters, such as floods, earthquakes, hurricanes,
tornadoes, and acts of war.
• Significant changes in federal funds allotted to the state to
implement WIA.
• Unanticipated legal or legislative actions that increase the
quality of participant outcomes and, as a result, increase
the level of effort needed by participants to attain these
outcomes. For example, a new state law may impose higher
standards for high school graduation. This action would
increase the standard needed to attain a diploma or
equivalent and could decrease the numbers of graduates.
Condition 2: Demonstrated Change in the Factor(s)
The unanticipated circumstance, by itself, does not provide
sufficient information to justify a revision in state negotiated
levels of performance. There should be sufficient evidence to
demonstrate change in one or more factors initially considered when
reaching agreement on the state negotiated levels of performance.
Those factors might have been an annual unemployment rate of 5.4%, a
baseline adult entered employment rate of 72%, or 53% of the adult
participants demonstrating one or more significant barriers to
employment. It is worthwhile to keep in mind that an unanticipated
circumstance may impact performance measures covering multiple
program years.
Condition 3: Factor(s) Known to Impact Performance and Revised
Levels can be Objectively Determined
Because unanticipated circumstances are, by definition, outside the
normal operating conditions, there can be no predetermined numerical
guidelines within which negotiated levels of performance might be
revised. However, any justification for revising performance levels
should be based on generally acceptable approaches and data sources.
A. Guidelines for Approaches and Data Sources
• The methods used to determine revisions to negotiated levels
of performance must:
- Adhere to widely accepted statistical practices, including
predictive or forecasting techniques where appropriate.
- Demonstrate a reasonable cause and effect relationship
between one or more factors and performance on a measure.
- Be fair, objective, and yield quantifiable results.
- Support, and not undermine, state efforts in achieving
continuous improvement of workforce investment activities.
• The source data must be:
- Developed by 1) a federal, state or local governmental
agency, or 2) some other reputable source such as the
state’s MIS unit, a university, or a private research
foundation.
- Gathered according to acceptable data collection
techniques.
- Compiled according to widely accepted analytical
procedures.
• The factors used to determine revisions to negotiated levels
of performance include:
- Differences in economic conditions.
- The characteristics of participants at the time of
registration.
- Services to be provided to participants.
B. Using Predictive Statistical Models to Estimate Revised Levels
State workforce agencies may develop their own statistical
forecasting models using the multiple regression approach or use one
or more other statistical methods used to predict selected outcomes.
The multiple regression statistical method is a widely used
technique that determines the relationship between a selected
performance outcome and multiple explanatory factors or variables.
While state workforce agencies may develop their own statistical
models, ETA recognizes that many states do not have the statistical
expertise or the necessary resources to develop these models. In
order to assist in determining the degree of relationship between
the factors and each of the performance measures, ETA will work with
the states to develop technical guidance and optional predictive
statistical models using multiple regression analysis for use in the
future.
C. Determining Revised Performance Levels in the Absence of
Predictive Statistical Models
While deriving revised levels of performance from predictive
statistical models is a preferred approach by many state
practitioners, the use of these models may not be feasible because
of a lack of national or state historical data on WIA participants.
This is especially true in the first few years of the operation of a
new program such as those funded under WIA. In the absence of
representative state and/or national historical data, the weighted
average approach should be used to determine revisions to state
negotiated levels of performance. The weighted average approach is
described in Attachment II.
D. The Importance of Negotiation in Revising State Levels of
Performance
Under certain circumstances, the predictive statistical models may
yield levels of performance that are unrealistic. Imprecision in
the models, extreme factor values, and the occurrence of rare,
isolated economic changes (such as those due to natural disasters)
are examples of conditions that may yield unrealistically high or
low levels of expected performance. In these and similar
situations, the proposed revised levels of performance should be
negotiated between the ETA Regional Office and state staff to ensure
reasonable and appropriate levels are set.
The expectation for performance is derived from state experience
with similar unanticipated circumstances or suggested by research
studies. However, the revised level should not be based solely on
past performance or experience under similar circumstances. The
revised level of performance should reflect what is ideally
attainable, given the change in the environment and any needed
changes to the delivery of services.
Since environmental conditions are likely to be unique for each
unanticipated circumstance arising in a state, defining an
appropriate performance level will likely be somewhat subjective.
In reaching agreement on a definition for exemplary performance, the
ETA Regional Office and state staff should consider the following
questions:
• What is the unanticipated circumstance?
• What are the changes in the factors considered in formulating
the original state negotiated levels of performance?
• Who are the participants impacted by the changes in the factors
(i.e., population group)?
• What is the estimated performance impact of the changes in the
factors on the population group?
• What changes can be feasibly made by the state to the design
and delivery of services to address the impact of the
unanticipated circumstance? What programmatic changes are
being proposed by the state?
• Given the changes in the factors, what level of performance is
estimated to occur…
- without changes to the design and delivery of services to the
target group?
- with feasible changes to the design and delivery of services
to address changes in the factors caused by the unanticipated
circumstance?
• What level of performance does the state cite for the balance
of exiters not included in the population groups impacted by
the unanticipated circumstance? Is this level different than
the initial state negotiated level of performance for the
measure? Why? Is the level reasonable and appropriate?
• What level of performance does the state suggest as appropriate
for the measure? Why?
• What level of performance does the ETA Regional Office cite as
ideally attainable? Why?
Attachment II (official PDF: https://www.dol.gov/sites/dolgov/files/ETA/advisories/TEGL/2002/TEGL11-01Attach2.pdf)
Attachment II
The Weighted Average Approach to Revising Levels of
Negotiated Performance
Overview
The weighted average approach views the state negotiated levels of
performance as aggregate levels of local workforce investment board
(WIB) negotiated performance and statewide project performance
goals. Similarly, local WIB negotiated levels of performance should
be viewed as aggregate levels of project and/or target group
performance goals.
Under the weighted average approach, the overall state negotiated
level of performance on a measure is disaggregated into expected
levels of performance for one or more affected target groups and for
the balance of exiters included in the calculation of performance on
the measure. Agreed-upon revised levels of performance are then
applied to each target group and the results are then aggregated to
derive a revised state negotiated level of performance.
In some situations, the unanticipated circumstance may equally
impact the expected outcomes of all exiters included in the
calculation of performance on a measure. In these situations, the
weighted average approach should not be used to derive revised
performance levels. Instead, ETA Regional Office and designated
state staff should reach agreement on a level for all exiters
included in the calculation of performance on a specific measure.
Defining a Special Population Group for Use in the Formula
A “special population group” may be categorized in a number of ways,
including: 1) participants affected by significant changes in
economic conditions; 2) the demographic characteristics of
participants; and 3) the type of services provided to participants.
For the purposes of this paper, a special population group is a
collection of individuals whose outcomes on a measure are expected
to be uniquely impacted by an unanticipated circumstance that
results in a change in one or more of the factors considered in
reaching agreement on the state negotiated levels of performance.
If more than one special population group is identified as being
affected by the unanticipated circumstance, it is important to
ensure these groups are mutually exclusive.
Negotiating Appropriate Performance Expectations
Both the ETA Regional Office and designated state staff should reach
agreement on an expectation for each special population group and
the balance of exiters identified for a specific performance
measure. Negotiating revised levels of performance is discussed on
pages 7, 8 and 9 of this guidance. The expectation may be derived
from state experience with similar unanticipated circumstances or
suggested by special research studies.
The Formula
The weighted average is calculated by multiplying the agreed-upon
performance level for each group by the number of expected exiters
in each group, totaling these results, and then dividing this result
by the total number of expected participants exiting services in the
state.
((SGE1*SGP1)+(SGE2*SGP2)+(SGE3*SGP3)+((TE–SGE1–SGE2–SGE3)*(TP)))
= RP
(TE)
Where…
SGE(1,2 or 3)1 is the number of estimated exiters in the
special population group.
SGP(1,2, or 3)1 is the negotiated performance level for exiters
in the special population group.
TE is the total number of estimated exiters in the state.
TP is the negotiated level of performance level for all exiters
not included in the special population groups. In most
situations, this value is the same as the original state
negotiated level of performance for the measure.
RP is the revised negotiated level of performance for the
measure.
Example Application of the Weighted Average Method
1
Additional special target groups may be included in the formula as needed to
accurately reflect the impact of an unanticipated circumstance.
Note: The following example was created to illustrate how a state
request for a revised performance level might be developed for submission
to the Secretary. For the sake of simplicity, the following example
request is for a proposed revision to one measure for a single program year
impacted by an unanticipated circumstance. It is quite likely, however,
several measures covering multiple program years would be impacted by the
unanticipated circumstance.
The Governor made a request to the Secretary to revise the state
negotiated performance level for the dislocated worker entered
employment rate measure for program year 2001. The Governor
requested that the level be revised from the PY 2001 negotiated
level of 77% to 72.4% to account for the loss of job openings and
the increased number of unemployed in selected communities of the
state that resulted from the closure of one of the state’s largest
employers (Condition 1). The employer ceased operation on August
15, 2001.
The state request indicated that there are 1,300 dislocated workers
included in the calculation of the PY 2001 dislocated worker entered
employment rate, with 350 of the 1,300 dislocated worker program
exiters impacted by the closure. The 350 dislocated worker
participants resided in the impacted communities and exited services
during the first quarter of PY 2001. The unemployment rate for the
impacted communities rose from an average of 5.2% for the nine
months before the closure to 9.7% for the three months after the
closure of the employer (Condition 2). The state contacted a
representative sample of public (non-WIA) and private placement and
temporary staffing agencies serving the impacted communities and
found the average placement rates for these agencies dropped 18%
from 78% during the first quarter of PY 2000 to 60% during the first
quarter of PY 2001 (Condition 3).
The state used the weighted average approach to determine a revised
level for its dislocated worker entered employment rate. The target
group used in the state’s computation is dislocated worker
participants from communities expected to be impacted by the plant
closings. Based on the state’s review of the impact of the closure
on both public and private employment agencies, the state set a goal
of 60% (the average rate reported by employment agencies) for the
estimated 350 exiters in the target group. The goal for the
remaining 950 dislocated worker exiters included in the calculation
was 77% (or the original state negotiated level of performance).
To compute the revised level for the state overall, the state used
the weighted average method to address the estimated impact of the
target group on PY 2001 performance on the dislocated worker entered
employment rate measure. The following scaled-down formula to
compute this revised level was used to determine the proposed
revised performance level requested by the state.
((SGE*SGP)+((TE–SGE)*(TP)))
= RP
(TE)
Where…
SGE = 350 (the number of estimated exiters in the special
population group)
SGP = 60% (the negotiated performance level for exiters in the
special population group)
TE = 1,300 (or the sum of 350 and 950 - the total number of
estimated exiters in the state)
TP = 77% (the original state negotiated level of performance
level for the measure)
Applying the above values in the formula, RP - the revised
negotiated level of performance for the measure – equals 72.4%, the
level proposed in the state’s request.
((350*0.60)+((1,300-350)*(0.77))) / (1,300) = RP
((210)+(731.5)) / (1,300) = 0.724 or 72.4%
Using the results of the weighted average formula, the Governor
requested the level be revised from the current negotiated level of
77% for PY 2001 to 72.4% to account for the loss of job openings and
the increased number of unemployed in selected communities of the
state that resulted from the closure of one of the state’s largest
employers.
What does federal law require today?
This guidance has been rescinded. Ezel checks current federal labor law as it stands now and answers your situation with citations.
Opens in Ezel Pro.
- Checks the law as it stands today, not only this page
- Cites every source it relies on, so you can verify it
- Chat, drafting and research in one workspace