Release of calendar year 2000 benefit accuracy data

U.S. Department of Labor
Employment and Training Administration
Washington, D.C. 20210
Classification: UI
Correspondence symbol: OWS
Issue date: March 9, 2001
Rescissions: None
Expiration date: March 31, 2002

DIRECTIVE: UNEMPLOYMENT INSURANCE PROGRAM LETTER NO. 25-01

TO: ALL STATE EMPLOYMENT SECURITY AGENCIES

FROM: GRACE A. KILBANE
Administrator
Office of Workforce Security

SUBJECT: Procedures for Release of Unemployment Insurance (UI) Benefit Accuracy Measurement (BAM) Data for Calendar Year (CY) 2000

1. Purpose. To provide State Employment Security Agencies (SESAs) with guidelines for the release of UI BAM program data for CY 2000.

2. References. 20 CFR 602; Federal Register Notice, September 3, 1987, 52 FR 33520; Unemployment Insurance Program Letter (UIPL) No. 13-93, January 12, 1993; UIPL No. 19-96, April 24, 1996; UIPL No. 18-00, March 8, 2000; and Benefit Accuracy Measurement State Operations Handbook, ET Handbook No. 395.

3. Background. The final rule to establish the UI Benefit Accuracy Measurement program, formerly Benefits Quality Control (QC), is found at 20 CFR Part 602 (52 FR 33520). 20 CFR 602.21(g) provides that each State shall:

Release the results of the QC program at the same time each year, providing calendar year results using a standardized format to present the data as prescribed by the Department; States will have the opportunity to release this information prior to any release by the Department.

UIPL No. 19-96 announced the elimination of the requirement that States publicly release BAM program data. While the Department of Labor will continue to publish an annual compendium of all States' data, each State may determine whether to independently release its data to the public.

4. Federal Release of UI PERFORMS Data. BAM data are published as part of the UI PERFORMS Annual Report, which also includes data for the Benefits Timeliness and Quality (BTQ) and Tax Performance System (TPS) programs. UI PERFORMS is the Department of Labor's management system for promoting continuous improvement in the performance of the UI system.

The CY 2000 UI PERFORMS Annual Report will be posted on the Employment and Training Administration (ETA) Office of Workforce Security (OWS) public Internet site: http://workforcesecurity.doleta.gov. The Department of Labor will distribute a limited number of the printed version of the UI PERFORMS Annual Report to SESA administrators. Additional copies of the printed version of the report can be obtained through the National Technical Information Service (NTIS) (Internet: www.ntis.gov). The Department of Labor will announce the availability of CY 2000 BAM program data and information on ordering the CY 2000 UI PERFORMS Annual Report from the NTIS in a notice published in the Federal Register on or before August 31, 2001.

The CY 2000 UI PERFORMS Annual Report will include background information on the BAM program, a brief discussion of how the payment accuracy rate estimates are computed, and national summary data. The UI PERFORMS Annual Report is arranged alphabetically by State in order to discourage ranking and comparing results among States. The Department of Labor may publish analytical information on CY 2000 BAM program findings in addition to the States' BAM Annual Reports.

If the Department of Labor determines that a State has failed to follow the BAM methodology prescribed in ET Handbook No. 395 -- regarding sample selection, data collection, or case completion timeliness -- to such an extent that the data are not reliable, the Department of Labor will publish an explanation of the major deficiency in lieu of a BAM Annual Report for that State. States will be required to address these areas by developing corrective action plans as part of the State Quality Service Plan (SQSP) process.

5. Narratives. UIPL No. 18-00 stated that, "Effective with the publication of the CY 2000 UI PERFORMS Annual Report, the Department of Labor will no longer post or publish State narratives." Since the elimination of the requirement that States publically release BAM data, few States in recent years have opted to submit narratives. States can address issues related to the BAM, BTQ, and TPS data published in the UI PERFORMS Annual Report through the SQSP process.

6. BAM Date of Record. The Department of Labor has established close of business (c.o.b.) April 30, 2001, as the date of record for State databases for computation of the CY 2000 BAM Annual Report payment accuracy rates. All information in the Annual Report will be based on CY 2000 BAM cases that have been signed off by the BAM program supervisor by c.o.b. April 30, 2001. Any data analysis supporting the Annual Report must be run on the database as it exists as of c.o.b. on April 30, 2001.

States must not reopen any CY 2000 BAM cases after this date until they have reviewed the computations of the payment accuracy rates and have reconciled any differences between the States' and the Department of Labor's databases. If States make any subsequent changes, their data will differ from the Department of Labor's data, which can result in inconsistencies between the State and the Department of Labor versions of the Annual Report. Any such differences must be reconciled before the Annual Report can be issued.

7. BAM Annual Report Software. As described in UIPL No. 13-93, the software which produces the BAM Annual Report excludes cases that do not meet the definition of the BAM population, for example, interstate or supplemental payments. These cases are coded "9" in the program code data element (c1 in the b_master table of the UI database). The BAM Annual Report software adjusts the amount of UI benefits paid to the population to exclude the amount paid that is represented by cases with program code value "9".

Prior to the date of record, the Department of Labor will transmit a footnote lookup table for the CY 2000 BAM Annual Report. States can run the Annual Report software as soon as all BAM cases are completed. The National Office will produce an Annual Report for each State, based on the CY 2000 BAM cases as of c.o.b. April 30, 2001.

8. BAM Annual Report Format, Definitions, and Supplemental Data. The BAM Annual Report format is displayed and the components of the Annual Report are defined in Attachment 1. In order for the SESAs to interpret the data in the BAM Annual Report, the software that produces the BAM Annual Report also generates a separate report consisting of up to 16 responsibility categories and up to 6 cause categories. Responsibility and cause data are reported as percentages of UI dollars overpaid. No sampling errors are computed for these percentages. States should be aware that some of these percentages are based on small numbers of BAM sample cases and may have relatively large sampling errors. States can combine categories which contain few sample cases, where appropriate. The format for this report is shown in Attachment 2.

If cause and responsibility percentages do not add up to 100 percent, States should check for cases in which the key week amount overpaid coded in data element h5 in the b_master table of the UI database does not equal the sum of dollars overpaid coded in data element ei1 for key week actions 10, 11, 12, 13, and 15 in the b_errisu table. The amount overpaid cannot exceed the amount paid coded in data element f13 in the b_master table.

9. Comment Period. BAM Annual Reports produced by the Department of Labor will be mailed to each State by May 11, 2001. Each State should carefully review the Annual Report produced by the Department of Labor to verify that it agrees with the Annual Report produced from the State's BAM database. States should report any discrepancies or submit questions concerning the BAM data or Annual Report footnotes to the National Office, with a copy to the appropriate Regional Office, by June 1, 2001, to:

Andrew Spisak
U. S. Department of Labor
ETA / Office of Workforce Security
200 Constitution Avenue, NW
Room S-4522
Washington, DC 20210

Alternatively, States may report discrepancies or submit questions concerning the BAM data by e-mail to [email protected] via the Internet or tonight!andy via the State's Sun computer e-mail utility or by fax to Andrew Spisak at 202-693-3229. National and Regional Office staffs will discuss and reconcile issues raised by the States by June 8, 2001.

10. Public Release by SESA. Although States are no longer required to release their annual BAM program data, States should provide the data to interested parties upon request. States will provide the name, address, phone number, and e-mail address of the person(s) designated by the State as its BAM Annual Report contact to the Department of Labor (attention of Andrew Spisak, at the above address), by June 8, 2001. The name of the individual(s) designated by the State as its BAM Annual Report contact will be published in the Federal Register notice announcing the availability of BAM data in the UI PERFORMS Annual Report. States must continue to follow the reporting requirements described in section 12, regardless of their decision to release their data to the public.

11. Key Dates. The following is a summary of key dates for the public release of CY 2000 BAM data. These dates are the latest that the task or action must be completed.

4/20/01

• States submit requests for waivers of BAM time lapse requirements, along with supporting analysis, to the appropriate Department of Labor ETA Regional Administrator.

4/27/01

• The Department of Labor Regional Offices respond to States on the disposition of State requests for waivers of BAM time lapse requirements.

4/30/01

• The Department of Labor electronically transmits the footnote lookup table for the BAM Annual Report software to each State's Sun computer.

4/30/01

• Each State checks its UI database to: 1) ensure that the BAM supervisor has signed off all cases which the BAM investigators have completed, and 2) reopen any case in order to update information or correct any data errors.

Cases closed by supervisors after April 30, 2001, will not be included in the Annual Report. States should not reopen any CY 2000 cases to change the database after April 30, 2001, until their Annual Report is finalized.

5/11/01

• Grace A. Kilbane, Administrator, OWS, sends the BAM Annual Reports to the State Administrators.

6/1/01

• States review BAM Annual Reports, including footnotes, and report any discrepancies or submit questions concerning the BAM data or Annual Report footnotes to the National Office (to the attention of Andrew Spisak, as discussed in section 9), with copies to ETA Regional Administrators.

6/8/01

• National and Regional Offices discuss issues raised by the States and reconcile any differences between the States' and Department of Labor's databases.

6/8/01

• States provide the National Office (to the attention of Andrew Spisak, as discussed in section 10) the name, address, phone number, and e-mail address of BAM Annual Report contact person (or persons), with copies to the ETA Regional Administrators.

8/31/01

• Department of Labor publishes notice in the Federal Register announcing the availability of BAM data as part of the UI PERFORMS Annual Report.

12. Action Required. State Administrators are requested to:

a. provide copies of these guidelines to the appropriate staff;

b. ensure that any request for a waiver of time lapse requirements are sent to the ETA Regional Administrators by c.o.b. April 20, 2001;

c. ensure that any issues related to CY 2000 BAM data or Annual Report footnotes that require reconciliation of differences between the State's and Department of Labor's databases are sent to the Department of Labor's National Office and copies are sent to ETA Regional Administrators by June 1, 2001; and

d. ensure that the name, address, telephone number, and e-mail address of the contact person(s) to whom interested parties may write or call with questions or inquiries about the State's BAM program data are sent to the Department of Labor's National Office and copies are sent to ETA Regional Administrators by June 8, 2001.

13. Inquiries. Questions should be directed to the appropriate Regional Office.

14. Attachments. (1) BAM Annual Report format, footnotes, and definitions; and (2) BAM Annual Report Supplemental Data format.

Attachment 1: BAM Annual Report definitions and footnotes

Official attachment: https://www.dol.gov/sites/dolgov/files/ETA/advisories/UIPL/2001/UIPL25-01_Attach1.pdf

BAM Annual Report Definitions

Total Dollars Paid in Population

Total UI benefits paid to the population of UI claimants who comprise the sampling frames for all weeks in CY 2000 for which the State pulled a BAM sample, adjusted to exclude other UI payments that do not meet the definition of the BAM population.

Sample Size

Total UI payments selected during CY 2000 (BAM batches 200001 through 200053) and completed (supervisor sign-off) by c.o.b. April 30, 2001, excluding cases that do not meet the BAM population definition. This is the number of BAM sample cases from which the payment accuracy rates and confidence intervals are estimated.

Proper Payments

The weighted ratio estimate of total dollars properly paid to total dollars paid expressed as a percentage.

Overpayments

The weighted ratio estimate of total dollars overpaid to total dollars paid expressed as a percentage.

Underpayments

The weighted ratio estimate of total dollars underpaid to total dollars paid expressed as a percentage.

95 Percent Confidence Interval

A confidence interval, expressed as +/- a percentage, is constructed for each of the three estimated rates. The actual rate is expected to lie within 95 percent of the intervals constructed from repeated samples of the same size and selected in the same manner as the BAM sample.

BAM Annual Report Footnotes

Footnote 1

Claimants failing to conduct required work search were given formal warnings and no overpayment was established. The proper payment rate would be lower and the overpayment rate would be higher if these cases were counted as erroneous payments.

Condition: Any CY 2000 BAM case with key week action code '14' in field ei2 of the b_errisu table of the UI database.

Footnote 2

Percentages apply to less than a complete year of UI payments due to the State not pulling a sample for xx weeks.

Condition: There is no record in the b_comparison table of the UI database for one or more BAM batches 200001 through 200053.

Footnote 3

The State selected samples that were below the minimum prescribed levels for xx weeks.

Condition: State selected one or more BAM weekly samples below the minimum level prescribed in Benefit Accuracy Measurement State Operations Handbook, ET Handbook No. 395, chapter III, p. 27. Cases sampled which fail to meet the BAM population definition are counted toward meeting the minimum weekly sample.

Footnote 4

Percentages based on data collection procedures that were not completely in accordance with the program methodology prescribed in ET Handbook No. 395.

Condition: Based on program monitoring conducted by the Department of Labor Regional and National Offices.

Footnote 5

The State completed xx percent of the cases within 90 days. The program standard is 95 percent completed within 90 days.

Condition: State failed to meet case completion objectives established in Benefit Accuracy Measurement State Operations Handbook, ET Handbook No. 395, chapter VI, p. 6. Cases not meeting the BAM population definition are not counted in calculating State time lapse rates.

Footnote 6

x.x percent of the sample cases were not completed when this report was prepared. This exceeded the program requirement that no more than 2 percent of the cases for the year remain incomplete.

Condition: The percentage is based on the number of BAM cases that were not completed (no supervisor sign-off) by c.o.b. April 30, 2001, divided by the number of valid cases selected for BAM weekly samples during CY 2000 (BAM batches 200001 through 200053). Cases that do not meet the BAM population definition are not counted in calculating State case completion rates.

Footnote 7

The annual sample for [State] is [#] cases below the allocated annual sample for the State. The precision of the data might be reduced due to the failure to sample at the prescribed level.

Condition: The State's annual sample selection is 60 or more cases below its allocated annual sample (with alternative data collection methods, one staff year equals 120 cases). Cases that do not meet the BAM population definition are counted toward meeting the annual sample allocation.

Footnote 8

The population from which the BAM sample was selected did not include all of the UI benefits paid. This limits the degree to which inferences about the population can be made from BAM data.

Condition: The dollars paid in the BAM population for CY 2000 vary from the dollars paid reported in the State's ETA 5159 Claims and Payment Activities reports by more than the statistical control limit.

Attachment 2: BAM Annual Report supplemental data format

Official attachment: https://www.dol.gov/sites/dolgov/files/ETA/advisories/UIPL/2001/UIPL25-01_Attach2.pdf

BAM ANNUAL REPORT
Supplemental Data
CY 2000

Responsibility for Overpayments (Percent of Dollars Overpaid)

Cause of Overpayments (Percent of Dollars Overpaid)

NOTE: The percentages for the responsibilities or causes do not sum to 100 percent. You should check field (h5) to verify that all cases have been coded correctly within your specified batch range. Also the responsibility and cause categories are ranked by their respective percentages.