WIA annual report narrative update

Accessible letter PDF: https://www.dol.gov/sites/dolgov/files/ETA/advisories/TEGL/2009/tegl14-00c3acc.pdf

CLASSIFICATION
 EMPLOYMENT AND TRAINING ADMINISTRATION                 WIA/Performance Reporting System
               ADVISORY SYSTEM
                                                        CORRESPONDENCE SYMBOL
         U.S. DEPARTMENT OF LABOR
              Washington, D.C. 20210                    PROTECH
                                                        DATE
                                                        August 24, 2009



TRAINING AND EMPLOYMENT GUIDANCE LETTER NO. 14-00, Change 3


TO:           STATE WORKFORCE AGENCIES
              STATE WORKFORCE ADMINISTRATORS
              STATE WORKFORCE LIAISONS

FROM:         JANE OATES /s/
              Assistant Secretary

SUBJECT:      Workforce Investment Act (WIA) Annual Report Narrative

1. Purpose. To provide additional guidelines to the states about the WIA Annual Report
narrative and to update procedures for submission of the report to the Employment and
Training Administration (ETA).

2. References.
   • The Workforce Investment Act of 1998 (WIA), sections 136 and 185;
   • 20 CFR Part 667.300(d)(e)(1)(2);
   • Training and Employment Guidance Letter (TEGL) 14-00, Change 1, and TEGL
       14-00, Change 2, Guidance on the Workforce Investment Act (WIA) Management
       Information and Reporting System;
   • TEGL 14-03 and TEGL 14-03, Change 1, Performance Reporting Submission
       Procedures for the Workforce Investment Act Standardized Record Data
       (WIASRD), the Annual Report, and the Quarterly Reports under Title IB of the
       Workforce Investment Act (WIA) for Program Year (PY) 2003 and 2004;
   • TEGL 17-05 Common Measures Policy for the Employment and Training
       Administration’s (ETA) Performance Accountability System and Related
       Performance Issues;
   • TEGL 14-08, Guidance for Implementation of the Workforce Investment Act and
       Wagner-Peyser Act Funding in the American Recovery and Reinvestment Act of
       2009 and State Planning Requirements for Program Year 2009;
   • Office of Management and Budget (OMB) Circular No. A-94, Revised Guidelines
       and Discount Rates for Benefit-Cost Analysis of Federal Programs.




RESCISSIONS                                           EXPIRATION DATE
None                                                  Continuing

3. Background. Each state that receives an allotment under WIA Section 127 (youth
activities) or Section 132 (Adult and Dislocated Worker activities) must prepare and
submit an Annual Report of performance progress to the Secretary of Labor in
accordance with WIA Sections 136 and 185.

There are two components to the WIA Annual Report: (1) the required performance
results, as specified in Form ETA 9091—the WIA Title 1B Annual Report (OMB No.
1205-0420) at: http://www.doleta.gov/performance/reporting/, and (2) a narrative report.
This change to TEGL 14-00, Change 2 focuses on the narrative report and what states
should address in the narrative.

4. WIA Annual Report Narrative. The required portions of the annual report narrative
include the information required by WIA Sections 136(d) (1) and (2) and 185(d). This
includes:
    • Performance data on the core and customer satisfaction measures,1 including
       progress of local areas in the state in achieving local performance measures;
    • Information on the status of state evaluation activities;
    • Information on the cost of workforce investment activities relative to the effect of
       the activities on the performance of participants;
    • Assurance that all required elements are reported uniformly so that a state-by-
       state comparison can be made; and
    • Information on participants in the workforce investment system (this information
       is typically addressed in the data portion of the Annual Report).

Although TEGL 14-00, Change 2, encouraged states to describe how waivers have been
used and how waivers have influenced performance outcomes, this new guidance now
makes this description a requirement of the Annual Report narrative. States should
include the waivers for which the state has received approval, how the waivers have
changed the activities of the state and local areas, and how activities carried out under the
waivers have directly or indirectly affected state and local area performance outcomes.

In addition to the required components of the WIA Annual Report narrative, ETA
encourages states to include the following information in its narrative:

    A. A discussion of the state’s unique programs and recent accomplishments. Such a
       report can describe these accomplishments in the most advantageous manner to
       all stakeholders and partners, including Congress, governors, state legislators and
       workforce investment boards. States may want to highlight “success stories” that
       focus attention on successful programs for participants, employers and
       communities. Messages from the governor or information about state workforce
       investment board members, market analysis, strategies for improvement, and
       effects on major industries may also be included.

    B. A discussion of the activities funded by the state’s discretionary (“15 percent”)
       funds. In this section of the narrative report, states may describe activities

1
  States that are approved to report only the common performance measure outcomes for WIA programs
do not have to report outcomes for the customer satisfaction measures, the credential measure for Adults
and Dislocated Workers, or the Older and Younger Youth measures on the WIA Annual Report.
                                                    2

       undertaken in whole or in part with their discretionary funds, and how those
       activities directly or indirectly affect performance.

  C.   A discussion of the initiatives and activities outlined in the WIA and Wagner-
       Peyser Act State Strategic Plan to improve performance.

TEGL 14-08 provided instructions to states for modifying their existing State Plans to
describe the implementation of the American Recovery and Reinvestment Act of 2009
(Recovery Act) and the response to the economic downturn, and also updated the
“National Strategic Direction.”

In their strategic State Plans, many states described initiatives that supported the
Department’s strategic direction, some of which were in the preliminary stages. States
may have progressed considerably in implementing such initiatives after submitting their
State Plan. This section of the narrative may be used to describe state activities in the
time periods between State Plans. States may provide an updated status of these
initiatives and activities, including a discussion of how these initiatives have directly or
indirectly affected performance. States may also use this portion of the narrative to
describe activities or initiatives that began after the submission of the State Plan.

States may want to also include information from their State Plans that highlight
innovative service delivery strategies, including program activities that support dislocated
workers, low-skilled/low-income adults and disadvantaged youth, the outcomes expected,
as well as the actual outcomes for their major customer segments. States may indicate
actual Federal outlays for selected activities, if the information is available.

Status of State Evaluation Activities:
States should include information about evaluation studies, if any were conducted, of
workforce investment activities as described in WIA Section 136(e). The narrative
should include:
    • The questions the evaluation will/did address;
    • A description of the evaluation’s methodology; and
    • Information about the timing of feedback and deliverables.

These evaluation studies, conducted under WIA Title 1B, are to promote, establish, and
implement methods for continuous improvement in the efficiency and effectiveness of
the statewide workforce investment system in improving employability for job seekers
and competitiveness for employers.

Costs of Workforce Investment Activities:
States should explain how the mix of services for adults, dislocated workers, and youth
activities affected the outcomes. For adults and dislocated workers, the activities that
states may wish to address are core, intensive and training services. For youth activities,
states may wish to include information about front-end costs (e.g., intake, assessment and
case management) and aggregated direct service costs for the ten youth program elements
described in WIA Section 129(c)(2).

ETA has a long-term interest in improving program efficiency so that both taxpayers and
customers can be better served. A variety of methods for calculating and presenting cost-
related activities and measures are described in prior WIA State Annual Reports. Among
                                             3

the most common cost calculations reported by the states is the “cost per participant”
indicator. “Cost per participant” is of limited use in measuring the effectiveness of a
program, because it measures appropriations or allotments to participants served.

To facilitate further discussion about costs to outcomes, OMB charged ETA with the
responsibility of developing and implementing an outcome-based measure for programs
using the common performance measures. A study is underway to determine how costs
and other measures of efficiency might be utilized to assess overall program
effectiveness. Results will be available on ETA’s website when the study is completed.
The attachment to this TEGL offers examples of potential efficiency measures.

5. Due Date. The Annual Report is due no later than October 1 following each program
year. For example, the report for Program Year (PY) 2008 will be due October 1, 2009.
The Annual Report, which includes the required performance reports and the narrative,
will reflect accurate performance outcome information that is available by the time the
Annual Report for the program year is due. Failure to submit the performance progress
reports by the deadline may lead to incentive grant ineligibility for a state.

6. Submission. An electronic copy of the WIA State Annual Report narrative is due to
[email protected] by October 1. States should also submit an electronic copy to their
respective ETA Regional Administrator. ETA will publish each state’s report on the
Internet. Acceptable formats include WordPerfect, Microsoft Word, Acrobat, and other
commonly used formats. In an effort to be more “green,” hard copies of the report may
be submitted, but are no longer required.

7. Action Requested. Distribute this TEGL to those personnel responsible for
developing the WIA Annual Report narrative, including personnel responsible for
performance reporting, and to all local areas responsible for administering the WIA
programs.

8. Inquiries. Questions concerning this TEGL should be directed to your appropriate
Regional Office.

9. Attachment. Overview of Potential Alternative Efficiency Measures for
Consideration.




                                             4

Linked official document 1: https://www.dol.gov/sites/dolgov/files/ETA/advisories/TEGL/2009/tegl14-00c3a1.pdf

ATTACHMENT

             Overview of Potential Alternative Efficiency Measures for
                                  Consideration

Examples of costs in relation to effect of participant services and outcomes

(1) Unit Costs = total cost by service / total participation by service.

        Expenditures            Participation                               Unit Costs
   Core Intensive Training Core Intensive Training               Core      Intensive Training
    $       $        $      #       #         #                   $            $        $

Pros:
   • It is applicable to most programs.
   • It makes it easier to understand how costs apply to participant services.
Cons:
   • Program services and costs must be closely tracked by year.

(2) Cost per Participant = This measure is calculated by taking the total program costs
in terms of expenditures and dividing by the number of participants served during the
year by the particular program.

Pros:
   • It is applicable to most programs.
   • Data is readily available.
   • Easy to understand.
   • Can be immediately generated each year.
   • Not costly or burdensome.
Cons:
   • It is of limited use in assessing program effectiveness, because it is not an
      outcome-based measure.

(3) Cost per Exiter = It is calculated by taking total program costs in terms of
expenditures and dividing by the number of exiters terminating the program during the
year by the particular program.

Pros:
   • It is applicable to most programs.
   • Data is readily available.
   • Easy to understand.
   • Can be immediately generated each year.
   • Not costly or burdensome.
Cons:
   • It is of limited use in assessing program effectiveness, because it is not an
      outcome-based measure.


                                              1

(4) Cost per Entered Employment = This measure is calculated by taking total program
    costs in terms of expenditures and dividing by the number of exiters entering
    employment in the first quarter following exit from the particular program.

Pros:
   •    It is applicable to most programs.
   •    Data is readily available.
   •    Easy to understand.
   •    Can be generated about two quarters after the end each Program Year.
   •    Not costly or burdensome.
   •    Measure is an outcome-based efficiency measure. Therefore, it is of substantial
        use in understanding program effectiveness.
Cons:
   • Does not capture those who entered employment in the same quarter of exit.

(5) Cost per Retained Employment = This efficiency measure is calculated by taking
    total program costs in terms of expenditures and dividing by the number of exiters
    who are employed in both the second and third quarters after the exit quarter.

Pros:
   •    Potentially applicable to most programs.
   •    Data is readily available.
   •    Relatively easy to understand.
   •    Relatively low cost and low burden to produce
   •    It is an outcome-based efficiency measure. Therefore, it is of substantial use to
        understanding program effectiveness and costs.
Cons:
   • Lengthier lags in data. Must wait for several quarters after the end of the program
      year.

(6) Cost per $1,000 Increase in Earnings = Total program cost divided by total earnings
    change from 2nd and 3rd pre-program quarters to 2nd and 3rd post-program quarters for
    participants or exiters.

Pros:
   •    Potentially applicable to most programs.
   •    Data is readily available.
   •    Relatively low cost and low burden to produce.
   •    It is an outcome-based efficiency measure. Therefore, it is of substantial use to
        understanding program effectiveness and costs.
Cons:
   • Somewhat difficult to understand.
   • Lengthier lags in data. Must wait several quarters after the end of the Program
      Year.



                                             2

(7) Cost per $1,000 in Post-Program Earnings = Total program cost divided by total
    earnings in 2nd and 3rd post-program quarters for participants of exiters * $1,000.

Pros:
   • Potentially applicable to most programs.
   • Data is readily available.
   • Relatively low cost and low burden to produce.
   • It is outcome-based.
Cons:
   • Lengthier lags in data. Must wait several quarters after the end of Program Year.
   • Somewhat difficult to understand.

(8) Cost per Exiter or Participant Receiving a Particular Service = Total program
    cost of a particular service divided by the number of exiters or participants receiving a
    particular service.

Pros:
   • Easy to understand.
   • No lags in data. Data can be immediately generated at the end of each year.
Cons:
   • Only applicable to programs that distinguish types of service.
   • Data is readily available for some programs, but not all.
   • Is not an outcome-based efficiency measure.
   • Maybe burdensome to generate.

(9) Cost per Placement in Employment or Education = Total program cost divided by
    the number of participants or exiters in employment or enrolled in post secondary
    education and/or advanced training or advanced training occupational skills in the 1st
    quarter after exit.

Pros:
   • The data is relatively easy to understand.
   • Relatively low cost and low burden to produce.
   • The measure is outcome-based so it is of substantial use to understanding program
      effectiveness.
Cons:
   • Limited to primarily the WIA Youth program.

(10) Cost per Individual Recognized Degree or Certificate (Credential include but are
   not limited to, a high school diploma, GED, or other recognized equivalents, post-
   secondary degrees/certificates, recognized skill standards, and licensure or industry-
   recognized certificates.) = Total program cost divided by the number of participants
   or exiters receiving a training service attaining a recognized credential during
   participation or by the end of the 3rd quarter after exit.

Pros:
   • The measure is an outcome-based measure, so it is of substantial use in
      understanding program effectiveness.

                                             3

Cons:
   • Only applicable to programs that provide services and identify individuals as
      receiving training and types of credentialing.
   • Data is readily available for some programs, but not all.
   • The measure is somewhat difficult to understand.
   • Potentially lengthy lags in data.

(11) Return on Investment = Return on Investment (ROI) is a way of summarizing how
large the gain on an investment, such as workforce development, actually is. In its
simplest form, ROI is calculated by dividing the gain by the size of the investment. This
equation can be written as B/C, where B is the sum of all benefits that result from the
investment over the period considered and C represents the costs. For a workforce
program, one would divide the increase in earnings due to the program by the cost of the
program. In more sophisticated analyses, ROI calculations take into account the timing
of the gains due to the program. Economists typically compute a variation called the
internal rate of return (IRR), which is based on the costs and benefits over the life of the
investment. The IRR can be calculated, using a financial calculator or a spreadsheet, by
solving the following equation for i: 0 = -C0 + (B1 –C1)/(1+i) + (B2 –C2)/(1+i)2 + (B3 –
C3)/(1+i)3 + …. + (BN –CN)/(1+i)N where Bt is the benefit received in year I, Ct is the
cost incurred in year I, and N is the last year that benefits or costs occur. (The four dots
mean that the formula includes the same type of term for all years between year 3 and
year N.) The IRR is preferred to the simpler versions of ROI because it takes into
account the timing of the costs and benefits.

Pros:
   • Potentially applicable to most programs.
   • Measure is an impact-based efficiency measure, which controls for factors that
      could potentially influence/bias results. Therefore, it is of the greatest utility in
      understanding program cost-effectiveness.
   • This measure controls for difficulty or cost of serving different populations (e.g.,
      hard-to-serve, service mix, and economic conditions).
Cons:
   • Data is very costly to produce.
   • The measure is difficult to understand.
   • Lengthy lags in data.




                                             4