High-priority designation for poor UI performance

CLASSIFICATION
   EMPLOYMENT AND TRAINING ADMINISTRATION                 Unemployment Insurance
               ADVISORY SYSTEM                            CORRESPONDENCE SYMBOL
          U.S. DEPARTMENT OF LABOR                        OUI/DPM
             Washington, D.C. 20210                       DATE
                                                          July 13, 2016

ADVISORY:         UNEMPLOYMENT INSURANCE PROGRAM LETTER NO. 17-16

TO:               STATE WORKFORCE AGENCIES
                  STATE WORKFORCE ADMINISTRATORS
                  STATE UNEMPLOYMENT INSURANCE DIRECTORS

FROM:             PORTIA WU /s/
                  Assistant Secretary

SUBJECT:          Reengineering Unemployment Insurance (UI) Benefits Program
                  Accountability Process: High Priority Designation of States with Sustained
                  Poor Performance

1. Purpose. To inform states regarding the new selection process for designating low
   performing states as “High Priority” and the process for removal from that designation.

2. References.
    Section 303(a)(1) of the Social Security Act (SSA);
    Title 20 Code of Federal Regulations Parts 602-617, 625, 640, and 650;
    Unemployment Insurance Program Letter (UIPL) No. 22-10, Selecting and Monitoring
      At-Risk States for Continuous Improvement and Compliance with First Payment
      Timeliness and First Level Appeals Promptness;
    UIPL No. 33-11, Identification of “Improper Payment High Priority States” for
      Unemployment Insurance;
    UIPL No. 9-13 Change 1, Integrity Performance Measure for Unemployment Insurance;
    UIPL No. 17-15, Additional Planning Guidance for the Fiscal Year (FY) 2016
      Unemployment Insurance (UI) State Quality Service Plan (SQSP);
    Training and Employment Notice (TEN) No. 8-14, Reengineering Unemployment
      Insurance (UI) Benefits Program Accountability Process;
    TEN No. 3-15, Reengineering Unemployment Insurance (UI) Benefits Program
      Accountability Processes: Update on Implementation Progress and State Impacts;
    Employment and Training (ET) Handbook No. 301, 5th Edition, UI Performs: Benefits
      Timeliness and Quality Nonmonetary Determinations Quality Review;
    ET Handbook No. 382, 3rd Edition, Handbook for Measuring Unemployment Insurance
      Lower Authority Appeals Quality;


 RESCISSIONS                                             EXPIRATION DATE
 None                                                    Continuing

      ET Handbook No. 395, 5th Edition, Benefit Accuracy Measurement State Operations
       Handbook; and
      ET Handbook No. 396, 4TH Edition, Unemployment Insurance Benefit Accuracy
       Measurement Monitoring Handbook.

3. Background. In April 2014, the Employment and Training Administration (ETA) initiated a
   project to reengineer UI benefit program accountability processes. ETA realized that a
   change in the current benefit accountability processes was needed to address the evolving
   changes occurring in the UI program. The UI system has a complex array of accountability
   and performance measurement processes designed to ensure program integrity and quality.
   Over the years, the UI program has changed and evolved and is now facing challenges, such
   as aging information technology systems and increased operating costs to support these
   systems, and staffing reductions due to continued funding constraints.

   TEN Nos. 08-14 and 03-15 outlined the new framework undertaken by ETA. As part of the
   new framework for UI benefit accountability processes, ETA has developed a new process
   for identifying states that are considered to be “High Priority” and in need of more intensive
   technical assistance. For over five years, ETA used two processes and designations that
   relate to states struggling with program performance and/or program integrity issues.

      UIPL No. 22-10 describes the previous process of selecting and monitoring states
       designated as “At-Risk” for continuous improvement with First Payment Timeliness and
       First Level Appeals Promptness performance.
      UIPL No. 33-11 describes the previous process for selecting states with program integrity
       issues to be designated as “High Priority.”

   The goal of both processes described in these two UIPLs was to provide enhanced monitoring
   and intensive technical assistance to the designated states in support of performance
   improvement. As discussed in more detail below, the new reengineered framework envisions
   a single designation of “High Priority” for states with performance challenges.

4. The New High Priority Designation Framework. The new High Priority Model integrates
   a variety of important program performance measures relating to UI benefit payment
   performance, integrity, and operations. The methodology was developed to include integral
   parts of UI Performs core measures, data validation results, and program integrity-related
   data as explained in further detail below. The UI Performs Core Measures used in the model
   are currently documented here:
   http://www.ows.doleta.gov/unemploy/pdf/Core_Measures.pdf. The variables in the model
   primarily contain all core benefit measures, three out of the four Core Performance Program
   Integrity measures, one of the core appeals measures, and one additional program-integrity
   data item not used as a core measure. The model also takes into account state data validation
   results.

   The new High Priority Model consists of four data inputs that will be used to determine if a state
   will be designated as “High Priority.” The data inputs are: 1) Benefits/Appeals and Data


Validations Results; 2) Program Integrity and Data Validation Results; 3) State Self-Assessment
Results; and 4) ETA Regional/National Office Input. A graphic depiction of the High Priority
Model is provided in Attachment A.

The High Priority Model compiles performance data from the states’ UI benefit payment
operations using the following data inputs:
A. Benefits/Appeals and Data Validation Results Module. This component includes the
   core benefits and appeals measures, and data validation results. The following measures
   will be included as part of this module:

    1. Benefit Measures
       a) First Payment Timeliness – percent of all first payments made within 14/21 days
          after the week ending date of the first compensable week in the benefit year. First
          payments must be made within 14 days for states with a waiting week, and 21
          days for states without a waiting week.
       b) Nonmonetary Determinations Time Lapse - Percent of nonmonetary
          determinations (separations and nonseparations) made within 21 days of the date
          of detection of any nonmonetary issue that had the potential to affect the
          claimant’s benefit rights.
       c) Nonmonetary Determination Quality – Nonseparations - Percent of nonseparation
          determinations with quality scores equal to or greater than 95 points, based on the
          evaluation results of quarterly samples selected from the universe of
          nonseparation determinations.
       d) Nonmonetary Determination Quality – Separations - Percent of separation
          determinations with quality scores equal to or greater than 95 points, based on the
          evaluation results of quarterly samples selected from the universe of separation
          determinations.
    2. Appeals Measures – Average Age of Pending Lower Authority Appeals – The sum of
      the ages, in days from filing, of all pending lower authority appeals divided by the
      number of lower authority appeals.

    3. Data Validation Results – Population corresponding with the following areas:
       a) First Payment Timeliness (Benefit Population 4)
       b) Nonmonetary Determinations Timeliness (Benefit Population 5)
       c) Separation Determinations Quality (Benefit Item BTQ1)
       d) Nonseparations Determination Quality (Benefit Item BTQ2)
       e) Lower Authority Appeals Timeliness (Benefit Populations 8 and 10)

B. Program Integrity and Data Validation Results Module. This module will include the
   following program integrity components:

    1. Integrity Measures


       a) Overpayment Detection – Percent of detectable, recoverable overpayments
          estimated by the Benefit Accuracy Measurement (BAM) survey that were actually
          established for recovery by the state.
       b) Overpayment Recovery – Percent of overpayments recovered of established
          overpayments that are not waived.
       c) Improper Payments – Percent of BAM estimated benefits overpaid plus BAM
          estimated benefits underpaid.
    2. Integrity Data - Operational Rate (Non-UI Performs Core Measure) – Percent of fraud
      and nonfraud overpayments that states are reasonably expected to detect and establish
      for recovery (excluding work search, employment service registration, base period
      wage issues and miscellaneous causes, such as benefits paid during a period of
      disqualification, redeterminations, and back pay awards) of total benefit outlays.
    3. Data Validation Results – Population corresponding with the following areas:
       a) Overpayments Established (Benefit Population 12)
       b) Overpayment Recovery (Benefit Population 13)

C. State Self-Assessment Results Module. A new component of the reengineered UI
   benefit accountability processes is a state self-assessment tool that states will complete
   annually to identify and implement improved business processes. The results from the UI
   Benefits Program Operations State Self-Assessment will assist ETA in monitoring the
   states’ quality of operations and identify areas where improvement is needed. The self-
   assessment is completed by dedicated state staff (state employee or contractor). The self-
   assessment consists of fifteen (15) functional areas (see Attachment B). The state self-
   assessment tool was piloted with nine states (California, Colorado, Connecticut,
   Delaware, Georgia, Idaho, Minnesota, New Hampshire, and Utah) and the feedback
   received from the pilot states is being used to further refine the self-assessment
   questionnaires. ETA is in the process of developing an approach to incorporate the
   results from the state self-assessment into the High Priority state composite score. This
   will require a full year of state self-assessment data, so the data from the state’s self-
   assessment will not be part of ETA’s High Priority designation process until Fiscal Year
   (FY) 2018 at the earliest. States will have an opportunity to provide comments on the
   self-assessment tool as part of the Office of Management and Budget’s Paperwork
   Reduction Act clearance process, which was announced in the Federal Register Notice
   (https://www.federalregister.gov/articles/2016/06/30/2016-15467/agency-information-
   collection-activities-comment-request-unemployment-insurance-benefits-operations).

D. ETA Regional/National Office Input. ETA Regional/National Office input is the last
   component used in determining which state(s) will be designated “High Priority.”
   Examples of this input include findings from prior program reviews; Office of Inspector
   General or General Accounting Office audits; awareness of significant changes to state
   policies, IT systems, or program leadership; and other data and information available
   regarding the state’s administration of the program.


5. High Priority Designation Process (Selection as a “High Priority” State). The
   information from each of the modules described above will be part of the process for
   designating “High Priority” states. Attachment A outlines how the scores are computed for
   each module.

   The selected “High Priority” state(s) will be formally notified by ETA with information on
   the specific areas that caused their selection as “High Priority.” One of the key aspects of this
   new process is the on-site review with each “High Priority” state by a select expert review
   team. The expert team is made up of National, Regional and State subject matter experts
   (SMEs) on the issue areas identified as needing improvement. State SMEs will be used only
   for providing technical assistance. The expert team will conduct the on-site review with the
   designated “High Priority” state. The on-site review is designed to identify process
   improvements needed and provide enhanced technical assistance that will support
   implementation of those process improvements and result in improved program performance.
   The expert team may evaluate other areas if the data indicate potential problems in those
   areas that were not previously identified. At the end of the review, states will be issued a
   formal Monitoring Report by the Region that will identify any findings, areas of concern, and
   best practices. Additionally, based on the issues identified in the on-site review and action
   steps for process improvement, states may be required to modify their State Quality Service
   Plan (SQSP) to either revise their existing corrective action plan (CAP) or add new CAP(s),
   as appropriate.

6. High Priority Designation Removal Process. The purpose of the “High Priority”
   Designation and its associated processes is to enable ETA to provide more enhanced
   monitoring and technical assistance to the state to allow the state to improve in those specific
   areas of concern without the detriment to other program areas. ETA will work
   collaboratively with the state to identify strategies and action steps that will enable the state
   to improve its performance in the areas identified during the on-site review.

   In order to have the “High Priority” designation removed, a state must:
       a) Complete corrective actions resulting from the state’s on-site review;
       b) Meet and sustain any Acceptable Level of Performance (ALP) for the core measures
           related to the areas of concerns during the initial designation process:
            The High Priority designation will continue until the state meets the ALP
                standard(s) http://www.ows.doleta.gov/unemploy/pdf/Core_Measures.pdf) for six
                consecutive months;
            The state can show that it is likely to continue to sustain the performance
                improvement; and
       c) Maintain performance in other program areas without any diminution of performance,
           i.e. the improved performance in the identified performance areas must not negatively
           impact the performance in other measured areas.

7. Action Requested. State administrators and UI directors are requested to provide the above
   information to all appropriate state personnel.


8. Inquiries. Inquiries should be directed to your Regional Office.
9. Attachments.
   Attachment A – High Priority Model, Computing the Scores
   Attachment B – The Operational Functions Chart

Attachment (official PDF: https://www.dol.gov/sites/dolgov/files/ETA/advisories/UIPL/2016/UIPL_17-16_Attachment-A_Acc.pdf)

High Priority Model                     Attachment A
                                 Computing The Scores


The new High Priority Model consists of four data inputs that will be considered in determining
if a state will be designated as “High Priority.” The data inputs are: 1) Benefits/Appeals and
Data Validations Results Module; 2) Program Integrity and Data Validations Results Module; 3)
State Self- Assessment Results Module; and 4) Regional/National Office Input. A graphic
depicting the “High Priority Model” process is displayed below. The methodology used to
compute the scores for each module is outlined in the different sections below.


A. The Benefits/Appeals and Data Validation Results Module and the Program Integrity and
   Data Validation Results Module will be used to generate a combined score to be incorporated
   with the State Self-Assessment Results to determine the overall data-based state composite
   score.

   All the measures in the Benefits/Appeals Measures and Data Validation Results Module, and
   the Program Integrity and Data Validation Results Module, with the exception of the
   Overpayment Detection Rate measure, will be evaluated for each one-year period that ends
   March 31st for each of the three years used. The Overpayment Detection Rate evaluation
   period will cover the prior 36-month period ending March 31st of each of the three
   measurement periods used in the measure.
   For each individual measure, the three measurement periods are combined using a weighted
   average, with the most recent year receiving the most weight, and the second and third years
   receiving the least weight. The weight factor for these measures will be 50 percent for the

                                              A-1

   most recent year, 30 percent for the second most recent year, and 20 percent for the third
   most recent year.

   The measures used for the overpayment detection rate, overpayment recovery rate, first
   payment timeliness, nonmonetary determination timeliness, nonseparation determination
   quality score and separation determination quality score are all based on a scale out of 100
   where higher scores are better and therefore are not changed to any other variable. The
   improper payment rate, operational rate and average age of lower authority appeals measures
   all use formats where a lower value is desirable and are therefore transformed via a
   normalization formula to comparable scores. Data validation scores are entered as binary 1/0
   for pass/fail results for each underlying population area with the average of a given year’s
   pass/fail results applying to each 12-month period ending March 31st.

   The Benefits/Appeals and Data Validation Results Module integrates the five benefits and
   appeals performance measures using a straight average across the measures, again with each
   measure receiving an equal weight. After this value is computed, it is then combined with
   the benefits/appeals data validation score based on a weighted formula. The weighted
   formula is applied so that the final performance module score is made up of 75% of the direct
   benefits and appeals performance measures and 25% of the data validation scores. The 25%
   weight for the data validation results is used to adjust potentially unreliable state performance
   measures data for issues identified through the data validation process for those data
   populations.

   The Program Integrity and Data Validation Results Module compiles three integrity measures
   and the operational rate as described in this UIPL (UIPL No. 17-16). The same annual
   weighting factors as described above are applied to the three years of program integrity data
   to produce the program integrity module score. A straight average is used so that each
   component receives equal weight. After this value is computed, it is then combined with the
   integrity portion of the data validation score based on a weighted formula. The weighted
   formula is applied so that the final program integrity module score is made up of 75% of the
   direct program integrity measures and 25% of the data validation scores. The 25% weight for
   the data validation results is used to adjust potentially unreliable state integrity measures data
   for issues identified through the data validation process for those data populations.

B. State Self-Assessment Results Module. As explained in this UIPL (UIPL No. 17-16), the
   exact process by which the state’s self-assessment results will be scored is still in
   development. The state’s self-assessment is a comprehensive review of the state’s UI
   benefits operations covering fifteen functional areas. The state will be responsible for
   ensuring that each functional area is reviewed and the self-assessment tool for each functional
   area is completed annually on a schedule and in a format to be provided by ETA once the tool
   has received clearance pursuant to the Paperwork Reduction Act.

C. High Priority Composite Score. The High Priority composite score is computed using a
   straight average of the Benefits/Appeals and Data Validation Results module score and the
   Program Integrity and Data Validation Results module score, such that each receives equal


                                               A-2

weight. The final overall High Priority composite scores are ranked from lowest to highest
where the lowest scores are those falling in the high priority range. The lowest scores are
selected for final review. The “High Priority” states are selected from this group of lowest
scoring states, using the model outputs, the underlying performance data, other relevant
performance data and additional ETA staff input (ETA Regional/National Office Input). The
weighting factors may be refined in future years based on model performance and results.


                                          A-3

Attachment (official PDF: https://www.dol.gov/sites/dolgov/files/ETA/advisories/UIPL/2016/UIPL_17-16_Attachment-B_Acc.pdf)

Attachment B

State: _____________________                                                 OPERATIONAL FUNCTIONS CHART                                                        Version 8.0
Review Period: ______________
                                                                                        Operational Elements
                                                                                                                                Operational
    Functional Areas          Procedures,                      Workload                                           Customer                  Staffing and
                                                                                  Performance     Information                   Efficiency                   Fiscal
                              Policies and     Training   Analysis/Management                                     Access and                   Merit
                                                                                  Management      Technology                     Resource                  Management
                             Confidentiality                    Controls                                        Communication                 Staffing
                                                                                                                                Allocation
Overarching Operational
Matters (Program Wide)
Intake - Regular
Unemployment Insurance
(UI), Interstate, Monetary
Determinations
Combined Wage Claim
(CWC)
Unemployment
Compensation for Federal
Employees (UCFE)
Unemployment
Compensation for Ex-
Service Members (UCX)
Adjudication/Benefits
Timeliness and Quality
(BTQ)
Continued
Claims/Eligibility Review

Appeals
Benefit Payment Control
(BPC)
Internal Security
Disaster Unemployment
Assistance (DUA)
Reemployment

Data Validation
Trade Readjustment
Allowances (TRA)
Short Time Compensation


 Cell Descriptors                                                Not Applicable


                                                                                          B-1