WIA annual report narrative due October 2011

Accessible letter PDF: https://www.dol.gov/sites/dolgov/files/ETA/advisories/TEGL/2011/TEGL7-11ACC.pdf

CLASSIFICATION
   EMPLOYMENT AND TRAINING ADMINISTRATION                  WIA/Performance Reporting System
               ADVISORY SYSTEM                             CORRESPONDENCE SYMBOL
          U.S. DEPARTMENT OF LABOR                         OPDR
             Washington, D.C. 20210                        DATE
                                                           September 26, 2011

ADVISORY:         TRAINING AND EMPLOYMENT GUIDANCE LETTER NO. 7-11

TO:               STATE WORKFORCE AGENCIES
                  STATE WORKFORCE ADMINISTRATORS
                  STATE WORKFORCE LIAISONS

FROM:             JANE OATES /s/
                  Assistant Secretary

SUBJECT:          Workforce Investment Act (WIA) Annual Report Narrative

1. Purpose. To provide guidance to the states regarding the content of the WIA Annual Report
   narrative, provide strategies for business engagement (as referenced in section 4, part c), and
   address the procedures for submission of the report to the Employment and Training
   Administration (ETA). The Annual Report is due on Monday, October 3, 2011.

2. References.
     Workforce Investment Act, Sections 134, 136 and 185;
     20 Code of Federal Regulations Section 667.300;
     Training and Employment Guidance Letter (TEGL)-14-00, TEGL 14-00, Change 1,
        Guidance on the Workforce Investment Act (WIA) Management Information and
        Reporting System and TEGL 14-00, Change 2 and Change 3, Workforce Investment Act
        (WIA) Annual Report Narrative;
     TEGL 14-03 and TEGL 14-03, Change 1, Performance Reporting Submission
        Procedures for the Workforce Investment Act Standardized Record Data (WIASRD),
        the Annual Report, and the Quarterly Reports under Title IB of the Workforce
        Investment Act (WIA) for Program Year (PY) 2003 and 2004;
     TEGL 17-05 and TEGL 17-05, Change 2, Common Measures Policy for the
        Employment and Training Administration’s (ETA) Performance Accountability System
        and Related Performance Issues;
     TEGL 14-08, Guidance for Implementation of the Workforce Investment Act and
        Wagner-Peyser Act Funding in the American Recovery and Reinvestment Act of 2009
        and State Planning Requirements for Program Year 2009;
     TEGL 26-09, Workforce Investment Act (WIA) Waiver Policy and Waiver Decisions for
        PY 2009 and 2010;
     TEGL 31-09, Program Year 2009/Fiscal Year 2010 Performance Reporting and Data
        Validation Timelines; and
     Office of Management and Budget (OMB) Circular No. A-94, Revised Guidelines and
        Discount Rates for Benefit-Cost Analysis of Federal Programs.

 RESCISSIONS                                              EXPIRATION DATE
 TEGL 9-10                                                Continuing


3. Background. Each state that receives an allotment under WIA Section 127 (Youth
   activities) or Section 132 (Adult and Dislocated Worker activities) must prepare and submit
   an Annual Report of performance progress to the Secretary of Labor in accordance with WIA
   Sections 136 and 185.

   There are two components to the WIA Annual Report: (1) the required performance results, as
   specified in form ETA 9091—the WIA Title 1B Annual Report (OMB No. 1205-0420) found
   at: http://www.doleta.gov/performance/guidance/WIA/WIAAnnualReportSpecifications.pdf,
   and (2) a narrative report. This guidance focuses on the narrative report and what states should
   address in this report.

4. WIA Annual Report Narrative. The required portions of the Annual Report narrative
   include the information required by WIA Sections 136(d) (1) and (2) and 185(d). This
   includes:
      Performance data on the core and customer satisfaction measures, including progress of
        local areas in the state in achieving local performance measures;
      Information on the status of state evaluation activities;
      Information on the cost of workforce investment activities relative to the effect of the
        activities on the performance of participants;
      Assurance that all required elements are reported uniformly so that a state-by-state
        comparison can be made; and
      Information on participants in the workforce investment system (this information is also
        included in the performance results portion of the Annual Report).

   States shall include a listing of the waivers for which the state has received approval,
   information on how the waivers have changed the activities of the state and local areas, and
   how activities carried out under the waivers have directly or indirectly affected state and
   local area performance outcomes. TEGL 26-09 describes the waiver policy for Program
   Years 2009 and 2010.

   Additional information for the content of the required components is included below.

   In addition to the required components of the WIA Annual Report narrative, ETA encourages
   states to include the following information in its narrative:

       A. A discussion of the state’s unique programs and recent accomplishments. Such a
          report can describe these accomplishments in the most advantageous manner to all
          stakeholders and partners, including Congress, governors, state legislators and
          workforce investment boards. States may want to highlight “success stories” that
          focus attention on successful programs for participants, employers, and communities.
          Messages from the governor or information about state workforce investment board
          members, market analysis, strategies for improvement, and effects on major
          industries may also be included.

       B. A discussion of the activities funded by the state’s discretionary (“15 percent”) funds.
          In this section of the narrative report, states may describe activities undertaken in


       whole or in part with their discretionary funds, and how those activities directly or
       indirectly affect performance.

   C. A discussion of programs and strategies for serving employers at the state and local
      level, including the performance metrics used by states or local areas to measure the
      effectiveness of such services and current available performance data.

   D. A discussion of the initiatives and activities outlined in the WIA and Wagner-Peyser
      Act State Strategic Plan to improve performance.

Additional Clarification on Required Elements

TEGL 14-08 provided instructions to states for modifying their existing state plans to
describe the implementation of the American Recovery and Reinvestment Act of 2009 and
the response to the economic downturn, and updated ETA’s “National Strategic Direction.”
In their strategic state plans for PY 2009 and modifications by some states for PY 2010 and
PY 2011, many states described initiatives that supported the Department of Labor’s strategic
direction, some of which were in the preliminary stages. States may have progressed
considerably in implementing such initiatives after submitting their state plan. This section
of the narrative may be used to describe state activities in the time periods between state
plans. States may provide an updated status of these initiatives and activities, including a
discussion of how these initiatives have directly or indirectly affected performance. States
may also use this portion of the narrative to describe activities or initiatives that began after
the submission of the strategic state plan.

States also may want to include information from their strategic plans that highlights
innovative service delivery strategies, including program activities that support dislocated
workers, low-skilled/low-income adults and disadvantaged youth, the outcomes expected, as
well as, the actual outcomes for their major customer populations. States may indicate actual
Federal outlays for selected activities, if such information is available.

Status of State Evaluation Activities

As WIA Section 134(a)(2)(B)(ii) notes, conducting evaluations of workforce investment
activities under WIA Section 136(e) is a required statewide activity. States should include
information about all evaluation studies that were started and/or completed during the
program year for which the WIA Annual Report is being submitted. For each evaluation, the
narrative should include:
   The timeline for starting and completing the evaluation;
   The questions the evaluation will/did address;
   A description of the evaluation’s methodology, including description of any control or
      comparison group and description of the analysis techniques employed;
   The timeline for the final report and other deliverables; and
   Summary of evaluation findings, including a summary of best practices, for those
      evaluations completed during the program year for which the WIA Annual report is
      being submitted.


   If no evaluations were started and/or completed during the program year for which the WIA
   Annual Report is being submitted, then states should submit the following information for
   each planned evaluation:

        Expected timeline for starting and completing the evaluation; and
        The questions the evaluation is expected to address.

   These evaluation studies, conducted under WIA Title 1B, are to promote, establish, and
   implement methods for continuous improvement in the efficiency and effectiveness of the
   statewide workforce investment system in improving employability for job seekers and
   competitiveness for employers. These evaluation studies also may identify best practices or
   replicable models and tools, as well as challenges and potential workforce solutions. This
   information would be helpful to the public workforce system and may inform ETA’s national
   evaluation and research agenda. As such, states are encouraged to share a copy of each final
   evaluation report with ETA via their regional representative. ETA will share relevant best
   practices, lessons learned, and other resources with the public workforce system through its
   online technical assistance (TA) platform (www.Workforce3One.org) and other appropriate
   TA events.

   Costs of Workforce Investment Activities

   States should explain how the mix of services for adults, dislocated workers, and youth
   activities affected the outcomes. For adults and dislocated workers, the activities that states
   may wish to address are core, intensive, and training services. For youth activities, states
   may wish to include information about front-end costs (e.g., intake, assessment and case
   management) and aggregated direct service costs for the 10 youth program elements
   described in WIA Section 129(c)(2).

   ETA has a long-term interest in improving program efficiency so that both taxpayers and
   customers can be better served. A variety of methods for calculating and presenting cost-
   related activities and measures are described in prior WIA State Annual Reports. Among the
   most common cost calculations reported by the states is the “cost per participant” indicator.

   To facilitate further discussion about alternative cost measures (e.g., costs per outcome),
   OMB charged ETA with the responsibility of developing and implementing an outcome-
   based measure for programs using the common performance measures. The attachment to
   this TEGL offers examples of potential, alternative efficiency measures.

5. Due Date. The Annual Report is due no later than October 1 following each program year;
   however, since October 1, 2011 is on a Saturday, the Annual Report will be due on Monday,
   October 3, 2011 this year. The Annual Report, which includes the required performance
   reports and the narrative, will reflect accurate performance outcome information that is
   available by the time the Annual Report for the program year is due. Failure to submit the
   performance progress reports by the deadline may lead to incentive grant ineligibility for a
   state.




6. Submission. An electronic copy of the WIA State Annual Report narrative should be
   e-mailed to [email protected] by October 3, 2011. States should also submit an electronic
   copy to their respective ETA Regional Administrator. Hard copies of the report may be
   submitted but are no longer required, in an effort to be more environmentally friendly. ETA
   will publish each state’s report on the Internet at www.doleta.gov/performance. Acceptable
   formats include WordPerfect, Microsoft Word, PDF, and other commonly used formats.

7. Action Requested. Distribute this TEGL to those personnel responsible for developing the
   WIA Annual Report narrative, including personnel responsible for performance reporting,
   and to all local areas responsible for administering the WIA programs.

8. Inquiries. Questions concerning this TEGL should be directed to your appropriate Regional
   Office.

9. Attachment. Overview of Potential Alternative Efficiency Measures for Consideration.




                                               5

Attachment: Alternative Efficiency Measures

Official attachment PDF: https://www.dol.gov/sites/dolgov/files/ETA/advisories/TEGL/2011/TEGL7-11ATT.pdf

ATTACHMENT

         Overview of Potential Alternative Efficiency Measures for Consideration

Examples of costs in relation to participant services and outcomes

(1)   Unit Costs = total cost by service / total participation by service.

              Expenditures                  Participation                    Unit Costs
       Core Intensive Training Core Intensive Training Core Intensive Training
        $       $        $      #       #        #      $       $        $

       Pros:
           It is applicable to most programs.
           It makes it easier to understand how costs apply to participant services.
       Cons:
           Program services and costs must be closely tracked by year.

(2)   Cost per Participant (CP) = This measure is calculated by taking the total program costs
      in terms of expenditures and dividing by the number of participants served during the year
      by the particular program.

                               CP = All Program Expenditures
                                    All Program Participants

      Pros:
          It is applicable to most programs.
          Data is readily available.
          Easy to understand.
          Can be immediately generated each year.
          Not costly or burdensome.
      Cons:
          It is of limited use in assessing program effectiveness, because it is not an outcome-
            based measure.

(3)   Cost per Exiter (CE) = It is calculated by taking total program costs in terms of
      expenditures and dividing by the number of exiters terminating the program during the year
      by the particular program.

                               CE = Total Program Expenditures
                                    Total Exiters Terminating Program

      Pros:
          It is applicable to most programs.
          Data is readily available.


          Easy to understand.
          Can be immediately generated each year.
          Not costly or burdensome.
      Cons:
          It is of limited use in assessing program effectiveness, because it is not an outcome-
            based measure.

(4)   Cost per Entered Employment (CEE) = This measure is calculated by taking total
      program costs in terms of expenditures and dividing by the number of exiters entering
      employment in the first quarter following exit from the particular program.

                               CEE = Total Program Costs
                                     First Quarter Exiters Entering Employment
      Pros:
             It is applicable to most programs.
             Data is readily available.
             Easy to understand.
             Can be generated about two quarters after the end of each program year.
             Not costly or burdensome.
             Measure is an outcome-based efficiency measure. Therefore, it is of substantial use
              in understanding program effectiveness.
      Cons:
          Does not capture those who entered employment in the same quarter of exit.
          Puts a premium on quick labor exchange at a time we are trying to improve skills.

(5)   Cost per Retained Employment (CRE) = This efficiency measure is calculated by taking
      total program costs in terms of expenditures and dividing by the number of exiters who are
      employed in both the second and third quarters after the exit quarter.

                               CRE = Total Program Costs
                                     Exiters Employed in Q2 & Q3 After Exit

      Pros:
             Potentially applicable to most programs.
             Data is readily available.
             Relatively easy to understand.
             Relatively low cost and low burden to produce.
             It is an outcome-based efficiency measure. Therefore, it is of substantial use to
              understanding program effectiveness and costs.
      Cons:
          Lengthier lags in data (must wait for several quarters after the end of the program
            year).
          Many jobs are lower paying which would result in a negative wage replacement
            rate.




(6)   Cost per $1,000 Increase in Earnings (CIE) = Total program cost divided by total
      earnings change from 2nd and 3rd pre-program quarters to 2nd and 3rd post-program
      quarters for participants or exiters.

                               CIE = Total Program Costs
                                     Total Participant or Exiters Earnings Change from 2nd and
                                     3rd pre-program quarters

      Pros:
             Potentially applicable to most programs.
             Data is readily available.
             Relatively low cost and low burden to produce.
             It is an outcome-based efficiency measure. Therefore, it is of substantial use to
              understanding program effectiveness and costs.
      Cons:
          Somewhat difficult to understand.
          Lengthier lags in data (must wait several quarters after the end of the program year).

(7)   Cost per $1,000 in Post-Program Earnings (CPPE) = Total program cost divided by
      total earnings in 2nd and 3rd post-program quarters for participants of exiters multiplied by
      $1,000.

                               CPPE = Total Program Costs
                                      Total Participant or Exiters Earnings in 2nd and 3rd post-
                                      program quarters multiplied by $1,000

      Pros:
             Potentially applicable to most programs.
             Data is readily available.
             Relatively low cost and low burden to produce.
             It is outcome-based.
             Unlike the prior measure, does not weight prior employment earnings against post
              program earnings.
      Cons:
          Lengthier lags in data (must wait several quarters after the end of program year).
          Somewhat difficult to understand.

(8)   Cost per Exiter or Participant Receiving a Particular Service (CPS) = Total program
      cost of a particular service divided by the number of exiters or participants receiving a
      particular service.

                               CPS = Total Cost of Particular Program
                                     Participants or Exiters Who Received Particular Service

      Pros:
          Easy to understand.


          No lags in data. Data can be immediately generated at the end of each year.
      Cons:
          Only applicable to programs that distinguish types of service.
          Data is readily available for some programs, but not all.
          Is not an outcome-based efficiency measure.
          May be burdensome to generate.

(9)   Cost per Placement in Employment or Education = Total program cost divided by the
      number of participants or exiters in employment or enrolled in post secondary education
      and/or advanced training or advanced training occupational skills in the 1st quarter after
      exit.

                              CPEE = Total Program Costs
                                     Number of Exiters or Participants Employed or in Post
                                     Secondary Education Programs After 1st Quarter Exit

      Pros:
          The data is relatively easy to understand.
          Relatively low cost and low burden to produce.
          The measure is outcome-based so it is of substantial use to understanding program
            effectiveness.
      Cons:
          Limited to primarily the Workforce Investment Act Youth program.

(10) Cost per Individual Attaining a Recognized Degree or Certificate (Credentials include
     but are not limited to, a high school diploma, GED, or other recognized equivalents, post-
     secondary degrees/certificates, recognized skill standards, and licensure or industry-
     recognized certificates.) = Total training program cost divided by the number of
     participants or exiters receiving a training service attaining a recognized credential during
     participation or by the end of the 3rd quarter after exit.

                              CID = Total Training Program Costs
                                    Number of Participants or Exiters who Attained
                                    Certification or Degree by the end of 3rd Quarter after exit

      Pros:
          The measure is an outcome-based measure, so it is of substantial use in
            understanding program effectiveness.
      Cons:
          Only applicable to programs that provide services and identify individuals as
            receiving training and types of credentialing.
          Data is readily available for some programs, but not all.
          The measure is somewhat difficult to understand.
          Potentially lengthy lags in data.




(11) Return on Investment (ROI) . ROI is a way of summarizing how large the gain on an
     investment, such as workforce development, actually is. In its simplest form, ROI is
     calculated by dividing the gain by the size of the investment. This equation can be written
     as B/C, where B is the sum of all benefits that result from the investment over the period
     considered and C represents the costs. For a workforce program, one would divide the
     increase in earnings due to the program by the cost of the program. In more sophisticated
     analyses, ROI calculations take into account the timing of the gains due to the program.
     Economists typically compute a variation called the internal rate of return (IRR), which is
     based on the costs and benefits over the life of the investment. The IRR can be calculated,
     using a financial calculator or a spreadsheet, by solving the following equation for i:

     0 = -C0 + (B1 –C1)/(1+i) + (B2 –C2)/(1+i)2 + (B3 –C3)/(1+i)3 + …. + (BN –CN)/(1+i)N

     Where Bt is the benefit received in year i, Ct is the cost incurred in year i, and N is the last
     year that benefits or costs occur. (The four dots mean that the formula includes the same
     type of term for all years between year 3 and year N.) The IRR is preferred to the simpler
     versions of ROI because it takes into account the timing of the costs and benefits.

     Pros:
         Potentially applicable to most programs.
         Measure is an impact-based efficiency measure, which controls for factors that
           could potentially influence/bias results. Therefore, it is of the greatest utility in
           understanding program cost-effectiveness.
         This measure controls for difficulty or cost of serving different populations (e.g.,
           hard-to-serve, service mix, and economic conditions).
     Cons:
         Data is very costly to produce.
         The measure is difficult to understand.
         Lengthy lags in data.




                                                  5