Contracts serving youth with greatest needs

Accessible letter PDF: https://www.dol.gov/sites/dolgov/files/ETA/advisories/TEGL/2010/TEGL13-09acc.pdf

CLASSIFICATION
   EMPLOYMENT AND TRAINING ADMINISTRATION                 WIA/Youth
               ADVISORY SYSTEM                            CORRESPONDENCE SYMBOL
          U.S. DEPARTMENT OF LABOR                        OWI-DYS
             Washington, D.C. 20210                       DATE
                                                          February 16, 2010



ADVISORY:          TRAINING AND EMPLOYMENT GUIDANCE LETTER NO. 13-09

TO:                STATE WORKFORCE AGENCIES
                   STATE WORKFORCE LIAISONS

FROM:              JANE OATES /s/
                   Assistant Secretary

SUBJECT:           Contracting Strategies That Facilitate Serving The Youth Most In Need.

1. Purpose. The purpose of this Training and Employment Guidance Letter (TEGL) is to
provide guidance to states, workforce investment boards, and Workforce Investment Act (WIA)
youth service providers on contracting strategies that facilitate service providers to serve the
youth most in need, while still achieving performance goals.

2. References.
    • U.S. Government Accountability Office, Disconnected Youth: Federal Action Could
       Address Some of the Challenges Faced by Local Programs That Reconnect Youth to
       Education and Employment, February 2008; http://www.gao.gov/new.items/d08313.pdf
    • TEGL No. 03-04, “The Employment and Training Administration's (ETA's) New
       Strategic Vision for the Delivery of Youth Services Under the Workforce Investment
       Act;” http://wdr.doleta.gov/directives/corr_doc.cfm?DOCN=1589
    • TEGL No. 09-08, “Negotiating Performance Goals for the Workforce Investment Act
       Title IB Programs and Wagner-Peyser Act Funded Activities for Program Year 2009;”
       http://wdr.doleta.gov/directives/corr_doc.cfm?DOCN=2702
    • TEGL No. 09-08, Change 1, “Revised National Performance Goal Targets for the
       Government Performance Results Act (GPRA) to support Negotiating State Performance
       Goals for the Workforce Investment Act Title IB Programs, Wagner-Peyser Act Funded
       Activities for Program Year 2009;”
       http://wdr.doleta.gov/directives/corr_doc.cfm?DOCN=2770

3. Background. In a February 2008 report reviewing 39 local youth programs, the Government
Accountability Office (GAO) found that contracts between local workforce investment boards
(WIBs) and youth service providers often require providers to meet performance goals within
time frames that are too short to reasonably achieve successful outcomes with youth most in
need (page 4). The hardest-to-serve youth, like many dropouts and out-of-school youth, require
additional time and assistance to attain positive outcomes as measured by the WIA Youth
common and statutory measures. However, the findings revealed that workforce boards often
procure one year contracts, which may unintentionally discourage programs from working with

 RESCISSIONS                                              EXPIRATION DATE
 None                                                     Continuing

lower skilled youth who may be unlikely to achieve positive outcomes during the term of the
contract. The GAO recommended that ETA work with states and WIBs to provide youth
program operators with information and guidance needed to develop and implement contracts
that facilitate local programs to serve the neediest youth while still achieving performance goals.

ETA has emphasized service to the youth most in need, particularly dropouts and out-of-school
youth. ETA encourages longer-term services for these youth in the WIA Youth program to
prepare them for post-secondary education and/or the workforce. This guidance letter is
intended to provide strategies and tools for states, WIBS, and local service providers to help
them negotiate performance requirements and develop contracts that encourage the system to
enroll youth most in need and deliver the long-term comprehensive services needed for
successful outcomes.
Ensuring accountability, while still encouraging service to the hardest-to-serve participants, has
long been a challenge within the workforce development system. Performance measures play a
crucial role in ensuring that the WIA Youth program fulfills its objective to prepare youth to
compete in the 21st century economy. However, research on the workforce development system
also suggests that performance goals can act as an unintended incentive to “skim the cream”
from the applicant pool, or serve only those eligible individuals most likely to achieve positive
outcomes. A 2002 GAO report found that the need to meet performance levels may drive who
receives WIA-funded services. Consequently, it is crucial that contracts are structured to
mitigate—or at least not exacerbate—disincentives to serve the youth most in need of services.

In November and December 2008, ETA held a series of focus groups with WIBs, local youth
service providers, and state youth staff to discuss the challenges local programs face in
developing contracts that promote service to the neediest youth. The focus groups provided
input around several areas of program contracting and management: a) local workforce area
procurement guidelines and processes; b) innovative strategies and processes that have produced
contracts that encourage service to the youth most in need; and, 3) procedures and practices that
may inhibit program collaboration for serving these youth. The focus group participants echoed
many of the challenges identified in the February 2008 GAO report; however, they placed less
emphasis on the length of contracts—many already employed multi-year or option-year
contracts, with an average contract length of 2.9 years—and more on performance goal
negotiations and enrollment requirements as barriers to serving the neediest youth. Several focus
group participants shared that performance requirements have caused them to enroll participants
based on their likelihood of achieving a successful outcome, rather than their ability to benefit
from WIA services. ETA has based the recommendations in this guidance letter on input from
the focus groups and on responses from direct service providers, WIBs, and state agencies
administering WIA Youth funds.

4. Strategies to Promote Service to the Youth Most in Need. Findings from the focus groups
and discussions with other partners within the workforce development system revealed several
contracting strategies that could promote service to the neediest youth. These strategies fall into
six main categories:
       • Employing Multi-Year or Option-Year Contracting
       • Basing Enrollment Goals on a Total Participant Count in a Performance Period
       • Using “Real-Time” Indicators or Interim Progress Benchmarks to Assess Provider
           Performance in addition to WIA Youth Common or Statutory Measures
       • Factoring Participant Characteristics and External Factors into Performance Goal

                                                 2

           Negotiations
       •   Creating Positive Incentives for Youth Service Providers to Enroll Youth who are
           Most in Need of Services and to Provide Services that will help these Youth Achieve
           Successful Outcomes
       •   Encouraging the Use of Interagency Collaboration to Create a Seamless System in
           which funds from other sources provide Services to Stabilize Basic Shelter, Health,
           and Safety Needs and Meet the Broad Developmental Needs of Youth prior to their
           Enrollment in WIA for Services Targeted to Career Education and Employment

5. Employing Multi-Year or Option-Year Contracting. Most of the 24 local WIBs and
service provider focus group participants indicated that they employ multi-year or option-year
contracts. While nine of the participants indicated that they had one-year contracts, the
participants asserted that unless there were serious performance issues, the renewal of their
contract was virtually assured. However, as the GAO report found, short-term contracts between
local service providers and WIBs can create a barrier to serving the youth most in need. Youth
with many barriers to education and employment require intensive, long-term services, making it
unrealistic for these youth to achieve successful outcomes on the WIA Youth common or
statutory measures within the one-year timeframes written into many contracts between WIBs
and service providers. Consequently, service providers on one-year contracts may be reluctant
to enroll youth who need intensive assistance.

Local WIBs that employ one-year contracts should consider replacing them with multi-year and
option-year contracts to foster extended service to youth most in need. Option-year contracts
have the advantage of maintaining annual accountability goals by tying future years of the
contract to performance in the base year. Additionally, option-year and multi-year contracts help
reduce procurement costs by limiting the frequency with which WIBs need to issue requests for
proposals.

WIBs and state agency staff reported that multi-year contracts provide stability to programs so
that providers do not feel that funds will be taken away from them due to circumstances out of
their control. Multi-year contracts allow providers more time to implement comprehensive
program models.

6. Basing Enrollment Goals on a Total Participant Count in a Performance Period.
Enrollment requirements built into contracts also can have an unintended detrimental impact on
service to youth most in need. Several service providers stated that their contracts require them
to enroll a certain number of new participants each month or each year. Other focus group
participants indicated that their contracts mandated a certain number of exits3 within a given
performance period. Requiring new enrollees or mandating a certain number of exits may have
the unintended effect of causing service providers to prematurely stop service to some youth or
to only enroll youth who can achieve program goals in a short period of time.

ETA encourages local WIBs and providers to track program participation in the same manner
that ETA itself tracks participants. Each year ETA reports national participation in the program
on the basis of the number of youth served, not on the basis of new program enrollees.
Accordingly, a youth served in Program Year (PY) 2007 and PY 2008 would appear in the
participant counts for both years.


                                                3

Instead of requiring a specific number of new enrollees or exits, local WIBs should consider
requiring service providers to maintain a minimum or average enrollment level that ensures that
service providers are operating at a defined capacity, while allowing them to serve youth based
on their needs (including those who need assistance for an extended period of time). For
example, a WIB may set an enrollment goal of 100 participants in the first year of a contract with
a service provider. In the second year of the contract, the enrollment goal would then transition
into a service goal of 100 participants, which would include youth carried over from the first
year and any new enrollees. This would allow providers to serve youth who need additional
assistance for multiple years without fear of failing to meet contractual enrollment targets.

7. Using “Real-Time” Indicators or Interim Progress Benchmarks to Assess Provider
Performance in addition to WIA Youth Common or Statutory Measures. The common and
statutory measures—with the exception of the Literacy/ Numeracy and the Skill Attainment
measures—track outcomes that cannot be measured until well after participants have exited from
the program (e.g. six months or more after exit). In fact, these measures were never intended to
be used to track the performance of service providers in real time and as such do not lend
themselves to use as program management tools. Nevertheless, many service providers are
being measured using common or statutory measures and only a small fraction of service
providers indicate that their contracts contain “real-time” performance measures.

For better provider and WIB tracking of outcomes during and immediately after the end of the
contract period, ETA encourages local WIBs to adopt “real-time” or interim indicators. Short-
term or “real-time” measures are changes in attitudes, behaviors, knowledge, and skills that can
be expected to occur during an individual's participation in a specific program or immediately
upon completion of a program or service. Required short-term outcomes can be included in
service provider Requests for Proposals (RFPs) to assist potential service providers in proposing
services that will lead to the desired short-term outcomes. Short-term outcomes set the stage for
monitoring contractor performance and establishing whether a contractor may deserve a bonus or
incentive payments under performance-based or hybrid contract designs (which provide some
payments based on the delivery of the proposed services and reserve some incentive or bonus
payments for the achievement of desired participant outcomes).

Effective short-term or “real time” measures can serve as proxies for desired intermediate and
long-term outcomes—as they help predict whether the desired outcomes will be achieved. Since
they measure the impact of specific programs or services upon individual participants, short-term
measures may need to be customized for each service design and provider. Three criteria that
can be used to assess the appropriateness of short-term indicators are: (a) whether the outcome
measures can be attributed to the services provided; (b) whether the outcomes can be achieved
during the contract period; and (c) whether the outcome measures provide useful data about
program performance.

The following are examples of short-term measures that might be relevant: a) substantial
learning gain, such as grade level improvement; b) completion of at least one major project
demonstrating progress toward or acquisition of basic skills and work-readiness skills per
program year; c) an increase in the retention rate for youth participating in alternative education
activities; d) for youth without a General Equivalency Diploma (GED) or high school diploma,
completion of an official GED practice test with a minimum score of 45; e) for youth with a
credit deficiency relative to high school graduation requirements, completion of a specified
number of credits toward earning a high school diploma; f) high school grade-level promotion;
                                                 4

g) competencies in specific subject areas; h) 30-day retention (or other interval) in positive exit
status; i) improved attitudes toward reading; and j) increased school attendance.

A. Basic Principles to Consider When Picking Interim Indicators.

 • Developing interim measures that relate to common (or statutory) measures. Common
   or statutory measures represent milestones that have been achieved within two to five years
   of program commencement. One of the most important tasks of any interim measure is to
   provide some real-time indication of progress toward common or statutory measures. Thus,
   if the common measure tracks a given outcome only 18 months after the exit of a
   participant, then the interim measure might track at the point of exit whether the participant
   has achieved a milestone on their way to meeting the measure. For instance, to demonstrate
   progress toward degree or certificate attainment, interim indicators might track satisfactory
   in-program progress, completion of classes or programs leading to a degree or certificate or
   the achievement of program milestones that occur prior to degree or certificate attainment.
   Certificate or degree attainment may also be used as an interim measure. This may be
   particularly useful for youth enrolled in degree or certificate programs that require more
   than one year for completion. To demonstrate progress toward placement in employment or
   education, service providers could track such attainment in real time and thereby avoid the
   tracking delays that are associated with the post-exit tracking that is defined under common
   and statutory measures. For more comprehensive program management, service providers
   may also track work readiness attainment, completions of employment applications or job
   placement interviews. To provide “real-time” management information in support of the
   attainment of literacy/numeracy skills, service providers can track training and test progress
   between the pre-and post-tests or even specific assignment or project completion.

    For example, the State of Delaware’s WIB asks respondents to its Youth RFP to track WIA
    performance outcomes at specific dates in time that are earlier than they would normally be
    measured. To assist in monitoring the retention of youth in employment and other
    outcomes, providers are responsible for reporting on participant retention in employment
    and other outcomes at the following points of time, including:
              • Day 1 of employment
              • Day 30 of employment (continuous from day 1)
              • Day 60 of employment (continuous from day 1- Collecting this measure is
                  optional)
              • Day 90 of employment (continuous from day 1 with a maximum 15-day gap
                  being allowable if the Day 30 goal was met)

     Another example is the San Diego Workforce Partnership which negotiates performance
     standards and outcomes with each of its youth service providers at the beginning of every
     program year. Included in each contractor’s Program Operating Plan are performance
     requirements for “Real-Time Common Measures” measured at program exit, including
     placement in employment or education; attainment of degree or certificate; and
     literacy/numeracy gains, measured separately for in-school and out-of-school youth. Key to
     implementing the Real-Time Common Measures is the Customer Information Services
     Reporting System (CISRS) that produces monthly management reports on each contractor’s
     annual performance to date. Contractors are held accountable for their performance on
     these measures at the end of the 2nd and 4th quarters of each contract year.

                                                 5

• Develop interim indicators that are achievable in the time frame of the contract.
  Interim indicators should be applied in a way so that performance goals can be achieved
  within one contract year. When developing measures that are time frame appropriate, WIBs
  should consider the types of goals or outputs that relate to the common measures that could
  be reasonably achieved by service providers within one contract year. Interim indicators
  may be useful for tracking daily, weekly, monthly, or quarterly progress. Interim measures
  can also be tailored to specific contractors.

• Keep data collection simple. The development of interim measures should not produce
  significant additional data collection burdens for service providers. This can be achieved if
  indicators are based on data elements that are already part of the Workforce Investment Act
  Standardized Record Data (WIASRD) reporting information data set or based on data
  elements that service providers routinely record as part of their normal business processes.
  In addition, for improved data accuracy, data collection and data reporting procedures
  should include frontline staff when possible. If interim measures do require additional data
  collection, be sure to provide user-friendly data collection tools, where possible, and inform
  service providers and contractors of new data collection requirements in the RFP.

• Consider the program management benefits of interim measures. Interim indicators are
  also a useful way for WIBs to manage oversight of service providers. While interim
  indicators associated with achieving the common or statutory measures provide useful
  insights into the effectiveness of a particular service or program, they may not be useful for
  managing your program or tracking the management of your service providers. Developing
  enrollment, attendance, or participation measures or indicators to measure caseload size,
  client demographics or client-staff ratios may provide important program management
  information. For instance, service providers will likely differ in the number of hard-to-serve
  youth enrolled in their programs or the ratio of staff to hard-to-serve clients. WIBs may
  wish to set different benchmarks for contractors serving primarily hard-to-serve youth to
  reflect population-specific challenges. Developing interim measures that take this into
  account may provide information to help with future RFPs, help you develop more
  individualized contracts, or, at the very least, explain lower than expected outcomes.

• Use interim measures to reflect achievements or outcomes not captured by common or
  statutory measures. Interim measures offer WIBs and service providers the opportunity to
  demonstrate their strengths in areas not reflected by the statutory and common measures.
  For instance, a WIB may have a particular interest in tracking skill attainment or job
  placement in a specific trade. WIBs may wish to apply different indicators for contractors
  serving youth with particular challenges. A service provider may have a particular strength
  in reducing gang-affiliation among previously gang-affiliated youth. A local area may have
  a large number of homeless youth who can benefit from education and job placement, but
  first need access to drug rehabilitation programs or appropriate health care. When
  developing interim measures consider the specific needs of your local area, the specific
  strengths of your service providers in meeting those needs, and the measures that would
  capture the alleviation of these needs. While these measures may not directly relate to the
  statutory or common measures, they will provide critical information about service
  provision to specific populations or in targeted trades.

   For example, the Fresno County WIB in California reviews and reports on real time
   measures for youth. Among these measures are the number of youth in the active caseload,
                                           6

    and the number pending high school graduation. The WIB categorizes all youth participants
    as being in one of two tracks- an academic upgrade track and a career track. For youth in
    the academic upgrade track, the short-term outcome of interest is whether the youth is on
    track to achieve a two-grade increase or 10th grade in reading and math. Once youth have
    achieved this outcome, they enter the career track, which begins with the assessment of
    employability skills and work experience. Youth who have not yet graduated from high
    school are tracked according to whether they are proceeding toward high school graduation.

  • Inform contractors about interim measures. As with any new measure or data collection
    requirement, make sure that you clearly communicate your expectations of your contractors
    during the RFP and contract development period. Provide a detailed explanation of data
    collection and reporting procedures for each interim measure collected through the contract.
    Include sample report forms within the RFP and contract. Outline a list of acceptable
    documentation of measure attainment. Supply contractors with easy-to-use data collection
    tools or systems to facilitate the collection of interim measure data.

8. Factoring Participant Characteristics and External Factors into Performance Goal
Negotiations. Many focus group participants agreed that annually renegotiated performance
goals with steady increases in required performance outcomes—a process that one respondent
called “performance creep”—makes it more difficult to serve the hardest-to-serve youth
effectively and succeed in meeting performance outcomes, especially over time as WIA has
experienced several successive increases in performance expectations. Furthermore, most
respondents at the state and local level reported that annual performance goal negotiations do not
take into account the risk factors of the participant population or external factors that may affect
performance outcomes, such as unemployment rates.

In recent TEGLs, the Department of Labor describes several approaches to negotiating
performance goals so as not to discourage services to youth most in need, particularly in local
labor markets with increasing levels of unemployment. These approaches include:
  • negotiating adjusted performance goals based on the level of services to youth most in
      need, based on the higher level of risk involved in achieving successful outcomes for these
      youth; and
  • negotiating adjusted performance goals based on the level of unemployment in the local
      economy for jobs likely to be targeted by youth most in need.

A. Negotiating adjusted performance goals based on the level of services to youth most in
need. When negotiating the Youth goals, states should be aware that ETA's strategic vision for
youth services includes a focus on serving the youth most in need, especially out-of school
youth, youth in foster care, youth in the juvenile justice system, children of incarcerated parents,
and migrant youth, as well as youth with disabilities and Native American youth. In negotiating
with ETA for appropriate Youth performance goals, states that currently serve a high level of the
neediest youth, or that want to transition to serving a higher percentage of youth who are most in
need, are encouraged to provide data that shows how performance outcomes are impacted by
serving this population.

States are encouraged to bring appropriate information to the negotiation process that
demonstrates either their past performance in serving these populations and/or their current
strategy for serving these populations. Relevant information, as described in TEGL 09-08 may
include historical data on performance between PY 2005 and PY 2008, national performance
                                                 7

goals set under the Government Performance Results Act (GPRA), data on the national
distribution of WIA outcomes by region, and national, regional, or local data on how the
characteristics of participants influences outcomes.

In Attachment 7 to TEGL 09-08, ETA provided the results of a number of different bivariate
regression analyses to support state-by-state adjustments based on the characteristics of youth
served. These adjustment factors were based on national-level WIASRD data on outcomes for
exiters in the most recent year available for each performance measure. The adjustment factors
summarize how outcomes are influenced by marginal changes in the proportions of youth
participants who are female; belong to different age groupings; are high school drop-outs; are
basic skills deficient, or have a disability.

In TEGL 09-08, Change 1, ETA announced some important refinements to the statistical models
that are available to support adjustments to state and local performance targets for PY 2009. In
addition to modeling the effects of state and local unemployment rates on performance
outcomes, new multivariate models will provide estimates of how changes over time in
participant characteristics influence national, state-by-state, and WIB-by-WIB performance
outcomes. The estimates provide weights that will be used to reflect the effects of individual
participant characteristics—such as work history, educational attainment, limited English skills,
and disability variables—on expected performance outcomes.

B. Negotiating adjusted performance goals based on the level of unemployment in the local
economy. In the current recession, adjustments to performance goals based on the
characteristics of the local labor market (e.g. unemployment rate, job growth rate) are perhaps
even more important than adjustments based on the characteristics of participants enrolled.
TEGL 09-08, Change 1 announced the lowering of national WIA performance goals under the
Government Performance Results Act (GPRA) in response to increased unemployment levels
throughout the nation. This reduction in overall Federal performance expectations under WIA
acknowledges that current economic conditions will make it more difficult to achieve successful
outcomes for both adult and youth participants in WIA.

Attachment 7 to TEGL 09-08 includes estimates of how unemployment rates affect performance
outcomes based on bivariate regressions of national-level WIASRD data on outcomes for exiters
in the most recent year available for each performance measure. As reported in TEGL 09-08,
Change 1, refined multivariate models have recently been developed to provide estimates of how
changes in unemployment rates over time influence national, state-by-state, and WIB-by-WIB
performance outcomes.

As states negotiate adjusted performance outcomes in response to economic pressures and
planned characteristics of WIA Youth participants, ETA encourages WIBs to make parallel
adjustments to the expected performance outcomes they establish for youth providers, taking
into account both local economic conditions and the characteristics of the youth served by each
provider. Thus, providers serving a higher percentage of homeless or foster youth could have
their performance goals adjusted downwards to account for the higher level of service to youth
most in need, and providers serving fewer youth in need could have their performance goals
adjusted upwards.

9. Creating Positive Incentives for Youth Service Providers to Enroll Youth who are Most
in Need of Services and to Provide Services that will help these Youth achieve Successful
                                                8

Outcomes. Avoiding contracting practices that create disincentives to serve the neediest youth
is important, but equally important for states and local WIBs is to emphasize their policy goals
of serving the youth most in need and to create positive incentives to encourage service
providers to further these goals.

Among the positive strategies that WIBs can use to promote services to youth most in need are
the following:
  • Including clear statements of the WIBs desire to serve youth from the targeted groups in its
      RFPs and other communications with prospective service providers.
  • Using the extent of services proposed to youth most in need as an announced criterion in
    rating provider proposals.
  • Using state-level set-aside funds under WIA or the Recovery Act to issue RFPs for special
    projects to serve youth most in need.
  • In contracts, designing hybrid cost reimbursement and fixed price contracts with payment
    points to provide special bonus payments or higher rates of reimbursement for serving
    youth most in need.


10. Encouraging the Use of Interagency Collaboration to Create a Seamless System in
which Funds from other sources Provide Services to Stabilize Basic Shelter, Health, and
Safety Needs and Meet the Broad Developmental Needs of Youth Prior to their enrollment
in WIA for Services Targeted to Career Education and Employment. Another strategy that
is important in furthering the goals of serving youth most in need and achieving successful youth
outcomes is the development of broad youth partnerships at the Federal, state, and local levels so
that a variety of funding sources can be used to create a seamless array of services to meet the
needs of participants for comprehensive, intensive, and long-term services.


ETA encourages states and local areas to develop formal partnerships that will allow multiple
agencies and organizations to fund different aspects of the youth service system. Each local
system of services can be coordinated through a combination of contracts and memoranda of
understanding between the participating entities. This strategy has the potential to reduce the
performance pressure on WIA service providers by sharing the investment in service reforms
with other agencies, thus making sure the full range of needed services is available to youth
using carefully constructed assessment and referral linkages between partners. As a result of
such a strategy, funds from other agencies like the Department of Justice (under programs for
incarcerated youth or youth on probation) or the Department of Health and Human Services
(under programs for homeless or foster youth) could provide “front-end” services for pre-
employment preparation and/or meeting broad developmental needs of youth, and making sure
that their immediate housing, health, and safety needs are stabilized before they are enrolled in
WIA. Another example of collaborating across agencies is WIA youth programs partnering with
WIA Title II Adult Education programs in order to access the basic skills remediation services
many of the out of school youth in WIA programs need in order to be successful in education
and the workforce.

11. Inquiries. Questions regarding this guidance should be directed to the appropriate ETA
Regional Office.

                                                9