Overpayment detection and appeals performance measures

EMPLOYMENT AND TRAINING ADMINISTRATION ADVISORY SYSTEM U.S. DEPARTMENT OF LABOR Washington, D.C. 20210 Classification: UI Performs Correspondence symbol: OWS/DPM Date: October 12, 2005 Rescissions: None Expiration date: Continuing

ADVISORY: UNEMPLOYMENT INSURANCE PROGRAM LETTER NO. 14-05, CHANGE 1

TO: STATE WORKFORCE AGENCIES

FROM: CHERYL ATKINSON s/s
Administrator
Office of Workforce Security

SUBJECT: Performance Criterion for the Overpayment Detection Measure; Clarification of Appeals Timeliness Measures; and Implementation of Tax Quality Measure Corrective Action Plans (CAPs)

1. Purpose. To inform State Workforce Agencies (SWAs) of the measurement
   method and the Acceptable Level of Performance (ALP) for the detection of
   overpayments measure; to clarify the methods for measuring the average age
   of pending lower and higher authority appeals; and to clarify the
   implementation schedule for Tax Quality Measure CAPs.

2. References. Unemployment Insurance Program Letter (UIPL) 14-05, Changes
   to UI Performs, dated February 18, 2005, and ET Handbook No. 401, 3rd
   Edition, Change 13, Changes to Form 9055: Appeals Case Aging, dated April
   18, 2005.

3. Background. UIPL 14-05 described changes to the UI Performs performance
   management system, some of which required additional data collection,
   analysis, and policy decisions before full definitions and ALPs could be
   determined. The measures of overpayment detection and the average age of
   pending appeals were among those for which ALPs were deferred.

4. Average Age of Pending Appeals. Subsequent to the issuance of UIPL 14-05,
   the Department of Labor (DOL) received authority to collect data on the age of pending lower and higher authority appeals. Instructions for the data
   collection were issued in Handbook 401, Change 13, dated April 18, 2005.
   That instruction limits the data collection to single-claimant appeals. To
   assure consistency among the advisories, the descriptions of Core Measures
   No. 8 “Average Age of Pending Lower Authority Appeals” and No. 9
   “Average Age of Pending Higher Authority Appeals” in attachment C to
   UIPL 14-05 are being modified to make clear that the measures apply to
   single-claimant appeals, and that the reported data include the median age of
   all pending single-claimant appeals. The modified pages are attached to this
   advisory and should be used to replace the original pages. ALP criteria are
   not yet established.

5. Detection of Overpayments. UIPL 14-05 defines the new Core Measure,
   Detection of Overpayments, as the percent of detectable/recoverable
   overpayments established for recovery. Because the measure was new, DOL
   deferred setting a performance level criterion until data for the measure could
   be assessed. The full definition and the ALP for the measure are presented
   here.

     a. The Measure. The measure is a ratio of the 3-year moving average of
       overpaid dollars a state actually established for recovery as a percent of
       the 3-year moving average of estimated detectable and recoverable
       overpayments.

      •   The numerator is the 3-year average amount of actual overpaid dollars
          that are established for recovery as reported on the ETA 227 report,
          Overpayment Detection and Recovery Activities (also known as
          Benefit Payment Control (BPC) activities). Penalty dollars are
          excluded. The numerator is subject to measurement variability.

      •   The denominator is a sample-based estimate of operational
          overpayments obtained from three years of Benefit Accuracy
          Measurement (BAM) data. The denominator is subject to both
          sampling and measurement variability.

      The data collection period of the numerator (BPC data) begins and ends
      six months after the denominator (BAM data) to allow sufficient time to
      detect and establish overpayments identified through the wage-benefit
      crossmatch.

     b. Acceptable Level of Performance. The ALP for the Overpayment
       Detection Core Measure is 50 percent. This is a minimum performance
level that all states are expected to meet; it is not a goal. States are
   encouraged to set overpayment detection goals that exceed this level.

   The ALP will apply to a 3-year average of performance so that the
   denominator (estimated operational overpayments) will be based on a
   larger BAM sample, thus reducing sampling variability and improving
   the precision of individual state estimates. The 3-year time period also
   minimizes the effect of differences between the time that overpayments
   are actually detected and established and the occurrence of errors
   estimated by BAM, and it reduces the measurement variability in BPC
   overpayment establishment data. National performance has averaged
   near 56 percent for the last 3 years, and two-thirds of the states have
   attained performance levels exceeding 50 percent.

c. Upper Limit. For monitoring purposes, DOL has established an upper
   limit of 95 percent. States cannot generally detect and establish more than
   80%-90% of estimated overpayments. When a higher ratio occurs, the
   cause is likely, but not certainly, the result of inaccurate BPC reporting
   and/or less-than-thorough BAM investigations that result in inaccurate
   (low) estimates. However, a state may establish overpayments in
   categories that are excluded from the operational rate, such as work
   search issues and Employment Service registration issues. Because these
   are excluded from the denominator (BAM estimates) but are included in
   the numerator (BPC establishments), the detection ratio can be higher than
   expected.

d. Corrective Actions. The cycle of the UI Performs management system
   requires planning ahead, and the implementation of the changes in the
   measures must be done in a systematic way so that the state partners
   know in advance what is expected of them. Because implementation
   began in 2005, the three year criteria for the overpayment detection
   measure will not be effective until 2008. The performance period includes
   BPC data from April 2005 through March 2008 and BAM data from
   October 2004 through September 2007. DOL will expect any state with
   performance falling below 50 percent to submit a CAP as part of its State
   Quality Service Plan (SQSP) for Fiscal Year (FY) 2009. In the interim DOL
   will expect any state reporting an overpayment detection rate below 50% to explain the reasons for the low performance in the Narrative section of
   the SQSP.

   Because the measure is based on a 3-year average, poor performance in an
   early year will remain in the calculation and continue to depress the ratio
   even as performance improves. To recognize improved performance, a
state whose performance in the most recent performance year meets or
      exceeds the ALP will not be expected to address performance of the
      overpayment detection measure in the SQSP for the next fiscal year.

      DOL will expect any state reporting an overpayment detection rate above
      95% to explain the reasons for the very high performance in the Narrative
      section of the SQSP. If the high performance is the result of a failure to
      properly administer BAM or BPC activities, then the state also must
      submit in the SQSP a CAP designed to produce valid data for the
      Overpayment Detection Measure. A CAP is required because the
      administration of BAM and BPC has a direct bearing on this Core
      Measure.

6. Tax Quality Measure. As discussed in UIPL 14-05, states are expected to
   complete CAPs for the Tax Quality Core Measure if either of two (2)
   conditions exist:
      • Four (4) or more functions fail the Tax Performance System (TPS)
          review in a calendar year (CY), or
      • One (1) function fails the TPS review in three (3) consecutive years.

     This advisory clarifies the implementation dates for Tax Quality Measure
     CAPs.

   a. Failure of four or more functions: DOL expects any state failing four or
      more tax functions during the CY 2005 TPS review to produce a CAP for
      its FY 2007 SQSP.

   b. Failure of 1 function in three consecutive years: As with the overpayment
      detection measure, DOL will expect any state failing the same tax quality
      function in three consecutive years to submit a CAP as part of its SQSP
      beginning with the SQSP for FY 2009. The three-year period began with
      CY 2005. In the interim, states will address failing the same tax quality
      function in three consecutive years in the Narrative section of the SQSP.

7. Action.

   a. Distribute this document to all appropriate SWA staff.

   b. In UIPL 14-05, replace Core Measures: No. 7 “Detection of
      Overpayments”, No. 8 “Average Age of Pending Lower Authority
      Appeals”, and No. 9 “Average Age of Pending Higher Authority
      Appeals” in Attachment C with the attached Core Measures pages.

8. Inquiries. Direct inquiries to your regional office.

9. Attachments.

   A. Core Measures Attachment C to UIPL 14-05, No. 7 Overpayment
      Detection
   B. Core Measures Attachment C to UIPL 14-05, No. 8 Average Age of
      Pending Lower Authority Appeals
   C. Core Measures Attachment C to UIPL 14-05, No. 9 Average Age of
      Pending Higher Authority Appeals

Attachment C
                               Core Measures

7. Detection of Overpayments

Measure:             Percent of estimated detectable, recoverable
                     overpayments (dollars) established for recovery.

Change:              This measure is new in UI Performs.

Data Source:         Benefit Accuracy Measurement (BAM) and ETA 227,
                     Benefit Payment Control (BPC) reports.

Computation and      The measure is a ratio of overpaid dollars a state actually
Criterion:           established for recovery as a percent of the estimated
                     amount of detectable recoverable overpayments.

                     The numerator is the amount of actual overpaid dollars
                     that are established for recovery, as reported on the ETA
                     227 report.

                     The denominator is a sample-based estimate of
                     Operational Overpayments obtained from the BAM data.
                     The denominator is subject to sampling and
                     measurement variability.

                     The ALP for the Detection of Overpayments Core
                     Measure is 50%, using a 3-year average of the measure.

                     The upper limit of performance is 95%.
Reporting
Categories:          None.

Reporting
Frequency:           Quarterly.
Details for the Computation of the
                Detection of Overpayments Measure

The amount of overpayments actually established through state BPC
operations as a percent of the amount that the BAM program estimates
states can detect and establish for recovery through BPC operations.

                              Overpayments Established (BPC)
UI Payment Accuracy Measure = ——————————————— X 100
                              Estimated Overpayments (BAM)

The data collection period of the numerator (BPC data) begins and ends
six months after the denominator (BAM data) to allow sufficient time to
detect and establish overpayments identified through the wage-benefit
crossmatch and appeals reversals.

BAM Operational Overpayment Rate Definition

The BAM estimate is an “operational overpayment rate.” The operational
rate includes those overpayments that are generally agreed to contain
overpayment errors that the states can be reasonably expected to detect
and establish for recovery. Certain categories of overpayments are
excluded from this rate: work search issues, Employment Service
registration issues, base period wage issues, and certain miscellaneous
causes. The BAM operational overpayment rate includes UI benefits that
BAM determined were overpaid and meet the following criteria:

      Key Week Action (BAM b_errisu table data element ei2)

      The code that specifies the type of overpayment error that the BAM
      investigation identifies for the compensated week of
      unemployment (referred to in BAM as the key week).

      10 = Fraud overpayment / voided offset

      11 = Nonfraud recoverable overpayment / voided offset
Error Cause (b_errisu table data element ei3)

BAM assigns each payment error a cause code. The following causes are
included in the BAM operational overpayment rate:

(a) In the Benefit Year, unreported or errors in reporting/recording
earnings or days/hours of work affecting the Key Week due to:

100 = Unreported (concealed) earnings or days / hours of work

110 = Earnings or days / hours of work incorrectly estimated, reported,
recorded or deducted

120 = Errors in reporting or unreported Severance Pay

130 = Errors in reporting or unreported Vacation Pay

140 = Errors in reporting or unreported Social Security or Pension
Benefits

150 = Other causes related to reporting or recording of earnings or days /
hours of work

(b) Separation Issues due to:

300 = Voluntary Quits
310 = Discharges
320 = Other causes related to separation issues

(c) Eligibility Issues due to:

400 = Ability to work
410 = Availability for work
430 = Refusal of suitable work
440 = Self-employment
450 = Illegal alien status
470 = Other causes related to eligibility issues
480 = Identity theft

(d) Dependents' Allowances incorrect due to:

500 = Dependents' information incorrectly reported / recorded or
allowance incorrectly calculated

510 = Other causes related to dependents' allowances
The following overpayment errors are excluded from the BAM
operational overpayment rate:

Key Week Action (BAM b_errisu table data element ei2)

12 = Nonfraud nonrecoverable overpayment or official action
taken to adjust future benefits by decreasing weekly benefit
amount (WBA), maximum benefit amount (MBA), dependents’
allowance (DA), or remaining balance (RB)

13 = BAM determines payment was too large, although payment
is "technically" proper due to finality rules

14 = BAM determines payment was too large except for formal
warning rule that prohibits official action. Payment "technically"
proper due to law / rules requiring formal warnings for
unacceptable work search efforts

15 = BAM determines payment was too large, although payment
"technically" proper due to rules other than finality or formal
warning rule

16 = Overpayment established or WBA, MBA, DA or RB
decreased which was later "officially" reversed, revised, adjusted,
or modified and BAM disagrees with "official" action (e.g.,
Appeals unit reverses BAM determination and BAM disagrees)

The following causes are excluded from the BAM operational
overpayment rate:

Error Cause (b_errisu table data element ei3)

(a) In the Base Period, errors in reporting / recording earnings or weeks,
days, or hours of work affecting the key week due to:

200 = Earnings or weeks / days / hours of work incorrectly estimated,
reported, or recorded

210 = One or more base period employers not reported by claimant

220 = Other causes related to errors in reporting or recording earnings or
weeks / days / hours of work for base period

(b) Eligibility Issues due to:
420 = Active work search
       460 = Job Service registration

       (c) Other Causes due to:

       600 = Benefits paid during a period of disqualification, even though a
       stop-pay order was in effect

       610 = Redetermination (at deputy level) or reversal (appeal or higher
       authority)

       620 = Back pay award

       630 = All other causes

Estimated Amount Overpaid and Weighting

BAM records the total whole dollar amount of the overpayment error for the
compensated week of unemployment (key week) as determined by the BAM
investigation. Dollars that affect weeks other than the key week are excluded.
This information is coded in Dollar Amount of Key Week Error (b_errisu table
data element ei1).

This amount is multiplied by a weight to estimate the amount that the
overpayment error represents in the population. The weight is equal to the
number of weeks of unemployment compensation paid during the week that the
BAM sample was selected (b_comparison table data element cm2) divided by
the number of completed BAM sample cases for that week. For example, if the
state issued 5,000 UI payments during the week, and BAM sampled and
investigated 10 payments, each completed case would carry a weight of 500
(5,000 / 10).

The weighted amounts overpaid are summed for the 52 weekly samples and
divided by the weighted amount of UI benefits paid to the claimants in the
sample (b_master table data element f13). This ratio is then multiplied by the
amount of UI benefits paid in the population of UI weeks paid from which the
samples were selected (b_comparison table data element cm4). The result is the
estimated UI overpayments that states should be able to detect and establish for
recovery through BPC.

Amount of Overpayments Established

The amount of UI overpayments established through BPC is reported on the ETA
227 report, line 110, columns 4 (UI) + 5 (UCFE/UCX), excluding penalty assessed
for fraud, reported on line 109, columns 4 + 5.
Attachment C
                                 Core Measures

8. Average Age of Pending Lower Authority Appeals

Measure:              The mean and median of all pending single-claimant Lower
                      Authority Appeals.

Change:               This measure is new in UI Performs. It takes the age of all
                      pending single-claimant appeals into account and allows
                      prediction of and planning for future performance.

Data Source:          Universe of pending single-claimant Lower Authority Appeals,
                      ETA 9055.

Computation and       At the close of business on the last day of each month, access
Criterion:            the universe of pending single-claimant Lower Authority
                      Appeals. Determine actual age of each pending single-
                      claimant Lower Authority Appeal by counting days elapsed
                      from the date of filing to the date of count. Sum actual age in
                      days of all pending single-claimant Lower Authority Appeals
                      and divide by the total number of pending Lower Authority
                      Appeals to arrive at the average age.

                      Determine the median age of all pending single-claimant lower
                      authority appeals using the following formula.
                      If all of the pending appeals cases (lower or higher authority)
                      are ranked from the lowest to the highest age, the median is the
                      age of the case at the midpoint of the ranked cases. If there is
                      an odd number of cases (n), the median is the age of the
                      [(n+1)/2] th case. If there is an even number of cases (n), the
                      median is the value midway between the age of the (n/2) th case
                      and the [(n/2)+1] th case.

                      The criterion will be determined after twelve months of data
                      have been recorded.

Reporting:            Report the average age and median age as determined by the
                      above computations.
Reporting
Categories:           None.

Reporting
Frequency:            Monthly
Attachment C
                                 Core Measures

9. Average Age of Pending Higher Authority Appeals.

 Measure:             The mean and median of all pending single-claimant Higher
                      Authority Appeals.

 Change:              This measure is new in UI Performs. It takes the age of all
                      single-claimant pending appeals into account and allows
                      prediction of and planning for future performance.

 Data Source:         Universe of pending single-claimant Higher Authority
                      Appeals, ETA 9055.

 Computation and      At the close of business on the last day of each month, access
 Criterion:           the universe of pending single-claimant Higher Authority
                      Appeals. Determine actual age of each pending single-
                      claimant Higher Authority Appeal by counting days elapsed
                      from the date of filing to the date of count.

                      Sum actual age in days of all pending single-claimant Higher
                      Authority Appeals and divide by the total number of pending
                      single-claimant Higher Authority Appeals to arrive at the
                      average age.

                      Determine the median age of all pending single-claimant
                      higher authority appeals using the following formula.
                      If all of the pending appeals cases (lower or higher authority)
                      are ranked from the lowest to the highest age, the median is the
                      age of the case at the midpoint of the ranked cases. If there is
                      an odd number of cases (n), the median is the age of the
                      [(n+1)/2] th case. If there is an even number of cases (n), the
                      median is the value midway between the age of the (n/2) th case
                      and the [(n/2)+1]th case.

                      The criterion will be determined after twelve months of data
                      have been recorded.

 Reporting:           Report the average age and median age as determined by the
                      above computations.
 Reporting
 Categories:          None.
 Reporting
 Frequency:           Monthly