1997 unemployment insurance benefit accuracy data release

UNEMPLOYMENT INSURANCE PROGRAM LETTER NO. 20-98

U.S. DEPARTMENT OF LABOR
Employment and Training Administration
Washington, D.C. 20210
CLASSIFICATION: UI
CORRESPONDENCE SYMBOL: TEUPR
ISSUE DATE: April 1, 1998
RESCISSIONS: None
EXPIRATION DATE: April 30, 1999
TO: ALL STATE EMPLOYMENT SECURITY AGENCIES
FROM: GRACE A. KILBANE, Director, Unemployment Insurance Service
SUBJECT: Procedures for Release of Unemployment Insurance (UI) Benefit Accuracy (BAM) Data for Calendar Year (CY) 1997

Purpose. To provide SESAs with guidelines for the annual release of UI BAM program data for CY 1997.

References. 20 CFR 602; Federal Register Notice, September 3, 1987, 52 FR 33520; 20 CFR 602; Federal Register Notice, March 26, 1990, 54 FR 11112; UIPL 13-93, January 12, 1993; and Benefits Quality Control State Operations Handbook, ET Handbook No. 395.

Background. The final rule to establish the Unemployment Insurance Quality Control (QC) program is found at 20 CFR Part 602. Effective October 5, 1987, QC programs in the 50 States, the District of Columbia, and Puerto Rico have selected weekly random samples of UI benefit payments for investigation under the mandatory program.

20 CFR 602.21(g) provides that each State shall:

> Release the results of the QC program at the same time each year, providing calendar year results using a standardized format to present the data as prescribed by the Department; States will have the opportunity to release this information prior to any release by the Department.

The notice of procedures for the release of Benefit Quality Control (BQC) data for the UI program was published in the Federal Register on March 26, 1990, at 54 FR 11112.

As part of an overall review of the BQC program, the Performance Enhancement Workgroup agreed to several changes, including changing the name of the program to BAM and eliminating the requirement that States publicly release BAM program data. In addition, it was agreed that while the Department of Labor (the Department) would continue to publish an annual compendium of all States' data, each State could determine whether to independently release its data to the public.

Date of Record. The Department has established close of business (c.o.b.) April 30, 1998, as the date of record for State databases for computation of the error rates for the CY 1997 Annual Report of BAM data. All information in the Annual Report will be based on CY 1997 BAM cases that have been signed off by the BAM program supervisor by c.o.b. April 30, 1998. Any data analysis supporting the Annual Report must be run on the database as it exists at c.o.b. on April 30, 1998.

States must not reopen any CY 1997 BAM cases after this date until they have reviewed the computations of the error rates and have reconciled any differences between the State's and Department's databases. If States make any subsequent changes, their data will differ from the Department's data, which can result in inconsistencies between the State and Department versions of the Annual Report. Any such difference will have to be reconciled before the Annual Report can be issued.

Annual Report Software. As described in UIPL 13-93, the software which produces the Annual Report excludes cases which do not meet the definition of the BAM population, for example, interstate or supplemental payments. These cases are coded "9" in the program code data element (c1 in the b_master table of the UI database). The BAM Annual Report software adjusts the amount of UI benefits paid to the population to exclude the amount paid that is represented by cases with program code value "9".

Calendar Year 1997 footnotes are defined in Attachment 1, pages 2 and 3. If the State has reopened a previously closed case to update information or correct data that the State has determined to be in error, the reopened date is used in computing the time needed to complete the case and determining whether the footnote concerning time lapse requirements applies.

The States may run the Annual Report software as soon as all cases are completed. The National Office (NO) will produce an Annual Report for each State, based on the CY 1997 BAM cases as of c.o.b. April 30, 1998.

Annual Report Format. The Annual Report format is shown in Attachment 1. It is divided into three main sections.

Benefit Payment Accuracy Data

• Total Dollars Paid in Population

Total UI benefits paid to the population of UI claimants who comprise the sampling frames for all weeks in CY 1997 for which the State pulled a BAM sample, adjusted to exclude other UI payments that do not meet the definition of the BAM population.

• Sample Size

Total UI payments selected during CY 1997 (BAM batches 9701 through 9752) and completed (supervisor sign-off) by c.o.b. April 30, 1998, excluding cases that do not meet the BAM population definition. This is the number of BAM sample cases from which the payment accuracy rates and confidence intervals are estimated.

• Proper Payments

The weighted ratio estimate of total dollars properly paid to total dollars paid expressed as a percentage.

• Overpayments

The weighted ratio estimate of total dollars overpaid to total dollars paid expressed as a percentage.

• Underpayments

The weighted ratio estimate of total dollars underpaid to total dollars paid expressed as a percentage.

• 95 Percent Confidence Interval

A confidence interval, expressed as +/- a percentage, is constructed for each of the three estimated rates. The actual rate is expected to lie within 95 percent of the intervals constructed from repeated samples of the same size and selected in the same manner as the BAM sample.

Footnotes

The footnotes that appear on State BAM Annual Reports describe certain conditions that affect the data. These footnotes are defined in Attachment 1, pages 2 and 3.

Narrative

States are invited to supply a narrative analysis to accompany their Annual Report. The Department will include this narrative in the compendium of State Annual Reports that it will publish (see section 10, below).

The narrative may include, but is not limited to, a discussion of State laws or special circumstances that affected the payment accuracy rates, and corrective actions that have been proposed or adopted. The narrative should not exceed two pages in length (66 lines per page with one inch margins).

The narrative must be transmitted by electronic mail (ELM). Transmit the narrative to "tonight!yvette" on the Sun system ELM or to "[email protected]" on the internet. List the subject as "Annual Report Narrative for XX" where XX is the State Postal FIPS abbreviation. For help with this process, call the Division of Information Technology Hotline on 1-800-473-0188. Any narrative submitted other than electronically will not be included in the compendium of State annual reports.

State narratives must be received in the National Office no later than June 5, 1998. States that decide not to submit a narrative should notify the NO no later than June 5, 1998, that no narrative will be submitted. States will provide a copy of their narrative or notice that no narrative will be submitted to the appropriate Regional Office (RO).

State narratives received after the June 5, 1998, deadline will not be included in the compendium of State annual reports.

Supplemental Data. In order for the SESAs to interpret the data in the Annual Report and provide narrative explanations of the data, the Department recognizes the need for additional information on payment error cause and responsibility. The software that produces the BAM Annual Report also generates a separate report consisting of up to 16 responsibility categories and up to 6 cause categories. Responsibility and cause data are reported as percentages of UI dollars overpaid. No sampling errors are computed for these percentages. States should be aware that some of these percentages are based on small numbers of BAM sample cases and may have relatively large sampling errors. States can combine categories which contain few sample cases, where appropriate. The format for this report is shown in Attachment 2.

If cause and responsibility percentages do not add up to 100 percent, States should check for cases in which the key week amount overpaid coded in data element h5 in the b_master table of the UI database does not equal the sum of dollars overpaid coded in data element ei1 for key week actions 10, 11, 12, 13, and 15 in the b_errisu table. The amount overpaid cannot exceed the amount paid coded in data element f13 in the b_master table.

The Department may publish overpayment error cause and responsibility data and other analytical data in the appendix of the annual compendium of State data.

Comment Period. BAM Annual Reports produced by the Department will be mailed to each State by May 8, 1998. Each State should carefully review its Annual Report and submit any comments to the NO by May 29, 1998, to:

> Andrew Spisak
> U.S. Department of Labor
> ETA/Unemployment Insurance Service
> 200 Constitution Avenue, NW
> Room S-4522
> Washington, DC 20210
>
Alternatively, States may submit comments by e-mail to [email protected] via the internet or tonight!andy via the State's Sun computer e-mail utility or by fax to Andrew Spisak at (202) 219-8506.

Each State will send a copy of any comments sent to the NO to the appropriate RO by May 29, 1998. National and RO staffs will discuss and reconcile issues raised in the States' comments by June 5, 1998.

Public Release by SESA. As discussed above, States are no longer required to release their annual BAM program data. However, the Department will continue to release a compendium of all States' data. States should provide the data to interested parties upon request. States must continue to follow the reporting requirements described section 11, regardless of their decision to release their data to the public.

Federal Release of Data. The Department will announce the availability of BAM program data for CY 1997 through a notice published in the Federal Register on or before July 31, 1998. BAM data will be published as part of a UI PERFORMS report that will also include data for the Benefit Timeliness and Quality (BTQ) and Tax Performance System (TPS) programs. UI PERFORMS is the Department's closed-loop system for promoting continuous improvement in the performance of the UI system.

The BAM section of the report will include background information on the BAM program and a brief discussion of how the payment accuracy rate estimates are computed. Readers will be discouraged from comparing results among States. Each State's section, including its Annual Report and narrative, will be displayed separately. State sections will be ordered alphabetically to discourage ranking and comparisons. The name of the State official designated by the State as its BAM Annual Report contact will also be published.

If the Department determines that a State has failed to follow the prescribed methodology -- regarding sample selection, data collection methodology, or case completion timeliness -- to such an extent that the data are not reliable, the Department will publish an explanation of the major deficiency in lieu of an Annual Report for that State. Also, in accordance with instructions in the Program and Budget Plan, ET Handbook No. 336 (11th edition, change 2), States will be required to address these areas by submitting corrective action plans for the upcoming fiscal year.

For CY 1997, the Department will continue the practice begun with CY 1989 data of publishing certain analytical information on aspects of the BAM findings.

Key Dates. The following is a summary of key dates for the public release of CY 1997 BAM data. These dates are the latest that the task or action must be completed.

> | | |
> |----|----|
> | 4/24 | States submit any request for waiver of time lapse requirements, along with supporting analysis, to the appropriate Department of Labor ETA Regional Administrator. |
> | 4/30 | The Department electronically transmits the modified Annual Report software and footnote look-up table to each State's Sun computer. |
> | 4/30 | States should check their UI database by the April 30, 1998, date of record 1) to ensure that all cases for which the BAM investigations have been completed signed off by the BAM supervisor, and 2) to reopen any case in order to update information or correct any data errors. |
> | 4/30 | Cases closed by supervisors after April 30, 1998, will not be included in the Annual Report. States should not reopen any CY 1997 cases to change the database after April 30, 1998, until their Annual Report is finalized. |
> | 5/8 | The Department sends Annual Reports to the State Administrators over signature of Grace A. Kilbane, Director, UIS. |
> | 5/8 | States provide name, address, and phone number of contact person (or persons) to ETA Regional Administrators. |
> | 5/29 | States review Annual Reports, including footnotes, and provide comments to the NO, with copies to ETA Regional Administrators. |
> | 6/5 | National and RO Offices discuss SESA comments with the States and reconcile any differences between the State's and Department's databases. |
> | 6/5 | States electronically transmit to the NO, with a hard copy to the ETA Regional Administrators, narratives or notification that a narrative will not be submitted. |
> | 7/31 | The Department publishes notice in the Federal Register announcing the availability of BAM data. |

Action Required. State Administrators are requested to:

1. provide copies of these guidelines to the appropriate staff;

2. ensure that the following items are supplied to ETA Regional Administrators:

   • (1) any request for a waiver of time lapse requirements by c.o.b. April 24, 1998; and

   (2) the name, address, and telephone number of the contact person(s) to whom interested parties may write or call with questions or inquiries about the State's BAM data by c.o.b. May 8, 1998.

3. ensure that the following items are supplied to the Department's NO with copies sent to ETA Regional Administrators:

   • (1) any comments on the CY 1997 Annual Report data that require reconciliation of differences between the State's and Department's databases by c.o.b. May 29, 1998; and

   (2) a narrative or notification that a narrative will not be submitted by June 5, 1998.

Inquiries. Questions should be directed to the appropriate RO Office.

Attachments. (1) Annual Report Format and Footnotes and (2) Format for BAM Annual Report Supplemental Data.

Attachment 1 — official text: https://oui.doleta.gov/dmstree/uipl/uipl98/uipl_2098a1.htm

Attachment 1

BAM ANNUAL REPORT
(New Version)
CY 1997

Run as of MM-DD-YY

State: ZZ

Unemployment Insurance Benefit Accuracy Measurement

Annual Report for Calendar Year 1997

Footnotes

An explanation of the footnotes follows on the next two pages.

Narrative Comments

See section 6, above, for an explanation of the narrative preparation, format, and transmittal procedures.

BAM ANNUAL REPORT FOOTNOTES

Footnote 1: Claimants failing to conduct required work search were given formal warnings and no overpayment was established. The proper payment rate would be lower and the overpayment rate would be higher if these cases were counted as erroneous payments.

Condition: Any CY 1997 BAM case with key week action code '14' in field ei2 of the b_errisu table of the UI database.

Footnote 2: Percentages apply to less than a complete year of UI payments due to the State not pulling a sample for xx weeks.

Condition: There is no record in the b_comparison table of the UI database for one or more BAM batches 9701 through 9752.

Footnote 3: The State selected samples that were below the minimum prescribed levels for xx weeks.

Condition: State selected one or more BAM weekly samples below the minimum level prescribed in Benefits Quality Control State Operations Handbook, ET Handbook No. 395, chapter III, p. 27. Cases sampled which fail to meet the BAM population definition are counted toward meeting the minimum weekly sample.

Footnote 4: Percentages based on data collection procedures that were not completely in accordance with the program methodology prescribed in ET Handbook No. 395.

Condition: Based on program monitoring conducted by the Department RO Office and NO.

Footnote 5: The State completed xx percent of the cases within 90 days. The program standard is 95 percent completed within 90 days.

Condition: State failed to meet case completion objectives established in Benefits Quality Control State Operations Handbook, ET Handbook No. 395, chapter VI, p. 6. Cases not meeting the BAM population definition are not counted in calculating State time lapse rates.

Footnote 6: x.x percent of the sample cases were not completed when this report was prepared. This exceeded the program requirement that no more than 2 percent of the cases for the year remain incomplete.

Condition: The percentage is based on the number of BAM cases that were not completed (no supervisor sign-off) by c.o.b. April 30, 1998, divided by the number of valid cases selected for BAM weekly samples during CY 1997 (BAM batches 9701 through 9752). Cases that do not meet the BAM population definition are not counted in calculating State case completion rates. Reporting the number of sample cases that were not completed is a generally accepted procedure in reporting the results of a statistical survey.

Footnote 7: The annual sample for [State] is [#] cases below the allocated annual sample for the State. The precision of the data might be reduced due to the failure to sample at the prescribed level.

Condition: The State's annual sample selection is 60 or more cases below its allocated annual sample (with alternative data collection methods, one staff year equals 120 cases). Cases that do not meet the BAM population definition are counted toward meeting the annual sample allocation.

Footnote 8: The population from which the BAM sample was selected did not include all of the UI benefits paid. This limits the degree to which inferences about the population can be made from BQC data.

Condition: The dollars paid in the BAM population for CY 1997 vary from the dollars paid reported in the State's ETA 5159 Claims and Payment Activities reports by more than the statistical control limit.

Attachment 2 — official text: https://oui.doleta.gov/dmstree/uipl/uipl98/uipl_2098a2.htm

Attachment 2

BAM ANNUAL REPORT
Supplemental Data
CY 1997

Responsibility for Overpayments (Percent of Dollars Overpaid)

NOTE: The percentages for the responsibilities or causes do not sum to 100 percent. You should check field (h5) to verify that all cases have been coded correctly within your specified batch range. Also the responsibilities and cause categories are ranked by their respective percents.