Unemployment insurance data validation program guidance

U.S. DEPARTMENT OF LABOR
Employment and Training Administration
Washington, D.C. 20210

Classification: UI
Correspondence Symbol: OWS/DPM
Issue Date: November 21, 2003
Rescissions: None
Expiration Date: September 30, 2004

ADVISORY: UNEMPLOYMENT INSURANCE PROGRAM LETTER NO. 03-04
TO: STATE WORKFORCE AGENCIES
FROM: CHERYL ATKINSON, Administrator, Office of Workforce Security
SUBJECT: Unemployment Insurance Data Validation (UI DV) Program Status and Guidance

1. Purpose. To report on the status of UI DV implementation, provide pertinent information on program implementation policies for fiscal years (FY) 2004 and 2005, identify key contacts for assistance and technical information, and provide questions and answers related to DV issues.

2. References. Unemployment Insurance Program Letter (UIPL) No. 03-01, "Unemployment Insurance Data Validation (UI DV) Program Update and Implementation Plans" (October 23, 2000); UIPL 14-02, "Unemployment Insurance Data Validation (UI DV) Program Implementation Plan" (February 22, 2002); UIPL 10-03, "Unemployment Insurance Data Validation (UI DV) Status and Guidance for Fiscal Year (FY) 2003" (December 16, 2002); UIPL 28-03, "Call Memo for the Fiscal Year (FY) 2004 Unemployment Insurance (UI) State Quality Service Plan (SQSP)," (July 2, 2003).

3. Background. The Office of Management and Budget approved the UI DV program in December 2001, as a cost-effective means of ensuring the accuracy of data used for performance measurement as well as budget development and allocation. States were asked to implement the DV program by July 31, 2003, and were required to report validation results to the Department of Labor (DOL) by September 30, 2003. States not implementing UI DV were to conduct a Workload Validation by September 30, 2003, unless a waiver was granted.

4. Status. During 2003, most states carried out UI DV implementation activities to meet the September 30, 2003, deadline for reporting UI DV results. In the course of implementing UI DV, many states identified issues and raised questions about the validation process, definitions, and operation of the Mathematica Policy Research, Inc. (MPR) software. These issues have been resolved in a variety of ways:

   • A few have led to modifications of underlying reporting definitions and instructions in Handbook 401, most of which have been issued as handbook changes;
   • Some resulted in changes to the UI DV handbook instructions or to the UI DV software;
   • Some have identified state reporting practices that were inconsistent with Handbook 401 reporting instructions and UI DV validation guidelines that states needed to change; and
   • Others identified areas of legitimate state reporting practice that produced inconsistencies between counts reconstructed by the software that were judged too minor to warrant software changes.

   Attached to this UIPL is a summary in question-and-answer format of the major issues and how they have been resolved. This summary will be posted to the Office of Workforce Security (OWS) Web site, and additional issues will be added as needed.

   Also during 2003, OWS staff developed a database to store validation results using Microsoft Access®. States were originally instructed to export their results using an Excel® spreadsheet option built into the MPR validation software, and e-mail the spreadsheets to the National Office, which would compile them. As the database was refined, it became apparent that it would be much more efficient to receive validation results as text files instead of as Excel® spreadsheets. Accordingly, MPR incorporated a text file export option into Benefits Version 1.6 and Tax Version 1.5 (both released in August 2003). States should use the text file option to transmit results of all validations completed after they have installed the August 2003 software versions and e-mail them to the address given below. However, validations completed using Benefits Software Version 1.5 or earlier, or Tax Version 1.4 or earlier, may be submitted in Excel® format.

   By the end of October, two states had submitted all required validation reports to the Department. Twenty-one other states submitted partial sets of reports, while the Department had received no reports from 22 additional states that were known to be conducting validation activities. Eight states had not begun any validations.

   Further, four states have yet to receive either benefits training or tax training or both. No training will be available after December 31, 2003, when the training contract with Sparhawk Group, Inc., ends. Some states have expressed an interest in refresher training; such requests will be honored if resources permit.

5. Actions and Policies in FY 2004. DOL's emphasis on completing the implementation of UI DV will continue in FY 2004. States unable to submit validation results for all 15 benefits and all five tax populations by the September 30, 2003 deadline were to "include a detailed CAP [Corrective Action Plan] in their FY 2004 SQSP specifying (1) when they plan to complete UI DV implementation and (2) the steps they will take to ensure full implementation" (UIPL 28-03). For the FY 2004 SQSP, no CAPs were required if a validated population or sub-population failed the validation test of being within ±2 percent for quantity validation or ±5 percent for quality. States are expected to correct reporting deficiencies that caused the validation failure even though no CAP was required for the FY 2004 SQSP.

   We will also develop and promulgate a policy on how validation results and performance measurement results will be linked. This policy will indicate how workload data used for budget formulation and allocation, as well as performance results used for both UI PERFORMS and the Government Performance and Results Act (GPRA), are to be interpreted in the light of information on the underlying quality of the data used for the performance measures.

   During FY 2004, the standardized data validation software developed by MPR will be rewritten to run on the Department's Sun/Unix computers that every state uses for reporting operations. The goal is for all validation operations from FY 2005 onward to be conducted in that environment. We urge state validation teams to identify changes in the existing MPR software and in related validation procedures that will improve validation operations so that these can be incorporated into the Department's version of the validation software.

6. Actions and Policies in FY 2005. For the FY 2005 SQSP, states will be required to prepare CAPs for any uncompleted populations, and CAPs will be required to correct any report elements that failed to pass validation during FY 2004. Also beginning in FY 2005, per a requirement of DOL's Office of Inspector General, the validations of the sub-populations pertaining to the GPRA elements will use tighter tolerances (±1 percent instead of ±2 percent) and will be done annually. Validation results will be regularly reported as part of the UI PERFORMS Annual Report.

7. DV Contacts. DOL National Office contacts for the UI DV program are Mr. Burman Skrable at (202) 693-3197 and Mr. Douglas Scott at (202) 693-3191). Results of state validations should be sent as an e-mail attachment in text format or Excel® format to [email protected]. Technical software support (benefits and tax) is available from Mathematica Policy Research (phone Ms. Grace Roemer at (609) 936-2782, or e-mail [email protected]). The Web site maintained by OWS for the DV program can be accessed at oui.doleta.gov/dv. This site is also the source for the current release of both the UI benefit and tax validation software and accompanying documentation.

8. Actions Required. State Workforce Agency Administrators are requested to:

   • Distribute this advisory to appropriate staff;

   • Ensure that states complete UI data validation;

   • Send results of completed validations to the National Office ([email protected]) , with a copy to the appropriate Regional Office (RO) DV contact;

   • Identify initial training or refresher training needs; coordinate with RO DV staff to make arrangements for training so that it is received before December 31, 2003;

   • Identify technical issues, administrative issues or other barriers that may impede implementation of the complete DV program and review this situation with appropriate RO staff to devise workable solutions available to the state;

   • To help refine the Sun/Unix version of validation software and streamline future validation processes, please send all operational issues identified with the existing MPR software as well as suggestions for improving the validation software and validation procedures to Doug Scott or Burman Skrable at the addresses given below by December 15; and

   • In cases where state implementation of the DV program is not feasible, conduct the Workload Validation program in accordance with DOL policy and submit workload validation results to the RO.

9. Inquiries. Please refer any questions to the appropriate RO, or Mr. Burman Skrable at (202) 693-3197 or Mr. Douglas Scott at (202) 693-3191.

10. Attachment. Questions and Answers Pertaining to Unemployment Insurance Data Validation (UIDV) Issues.

Attachment (https://oui.doleta.gov/dmstree/uipl/uipl2k4/uipl_0304a.htm)

Questions and Answers Pertaining to Unemployment Insurance Data Validation (UIDV) Issues

Benefits Questions

a. Question: Cases occur where the designation of "final payment" is determined based on claim status, and then additional wages are added to the claim resulting in an extension of the entitlement period (monetary redetermination). Is the count for original "final payment" valid?

Answer: Yes, the claim met all conditions for final payment at the time the designation was assigned.

b. Question: How long should states keep records pertaining to completed validation studies?

Answer: In accordance with established audit policy, the records should be kept for three years.

c. Question: A situation occurs in certain states where an individual is claiming payments in a continued claim series when the status of the monetary determination is pending. Should these weeks be counted as weeks claimed? A related issue is determining whether weeks claimed should be counted when a pending monetary determination precludes a consideration about whether excessive earnings are being reported which would invalidate the "weeks claimed" count.

Answer: To be counted, weeks claimed must be associated with a monetarily valid claim (Handbook 401 re: 5159 Report, "Exclude weeks of which . . . the claimant is monetarily ineligible"). Weeks claimed with reference to a monetarily valid claim awaiting a monetary redetermination should be counted. Step 11 of Module 3 of the UI DV Benefits Handbook contains details for determining whether a claim is valid by checking the potential eligibility of the claim.

Cases where a claimant in continued claim status is reporting wages which may invalidate the count depending on a pending monetary are extremely rare. The ±2 percent error tolerance for validation was designed to prevent such cases from causing states to fail validation and we anticipate that it should cover these.

d. Question: States with provisions for partial disqualification have encountered situations where the claimant received a final payment under a claim but the claimant had access only to a portion of the state's maximum benefit entitlement based on the monetary determination. The claimant receives the maximum benefit payment to which he is entitled, but not the maximum which would have been available absent the disqualification. Should this final payment be classified as "final payment for maximum duration" on the ETA 218 Report?

Answer: For purposes of the 218 Report, maximum duration is defined by the monetary determination. All claimants whose monetary determination entitles them to the state's maximum duration would be reported on the 218 Report as qualifying for maximum potential duration [cell 28 (c28) and perhaps c6].

Claimants who only have access to a portion of their entitlement as defined by the monetary determination due to a penalty, and receive all the payments they have access to, should be reported on the 5159 (c26, c27, or c28) as having received a "final payment." They should not be included in the 218 Report as "Number at Maximum Duration" (c35) because they did not exhaust eligibility at the state's maximum based on the monetary determination.

e. Question: Reporting instructions refer to a final payment of zero as indicative of claim exhaustion. This presents difficulties when state policy allows for situations where a balance exists but is not available to the claimant because of a disqualification. How are final payments identified?

Answer: Claim exhaustion, or the receipt of a final payment, is consistently defined in both the validation handbook and the Handbook 401 definitions for the ETA 5159 and 218 Reports. State reporting procedures may use "final payment of zero" or "final balance of zero" as a proxy for exhaustion but these must be harmonized with the common definition of final payment. See, for example, Handbook 401, page 1-2-6 (ETA 5159).

f. Question: Some states have provisions for claimants to draw benefits through the UI program while pursuing self-employment. How should these claims be counted?

Answer: Self-employment records should be included in two extracts: 1) the data set including the records for the program type (UI, UCFE, etc.), and 2) the extract for self-employment counts. This is similar to the manner in which Combined Wage Claim payments are handled.

g. Question: Some states use the "processing" date for new claims instead of the "claim-filed" date. Is this an issue?

Answer: "Processing date" and "claim-filed" date are accepted as being synonymous in this context. Claim processing is initiated when the transaction is first submitted to the state's mainframe computer. Thus the time of the two events is actually the same.

h. Question: Current versions of DV software consider any week(s) of claim duration in excess of the 25^th^ week as being "maximum duration" on the assumption that 26 weeks of benefit entitlement always equals maximum duration. How should claims for more than 26 weeks be handled with reference to the identification of maximum duration?

Answer: Some states establish claims for greater than 26 weeks of potential benefits. The current DV software is not sensitive to this situation and identifies claims for 26 weeks as being for maximum duration. If the DV results are distorted by this situation, include an explanation in the Comments section of the summary report.

i. Question: The time requirement for the issuance of non-monetary determinations (nonmons) is a key performance indicator. States have different methods of recording the dates used for this calculation. How can procedures be standardized to assure that states are in compliance?

Answer: The UI DV benefits Population 5 record layout requires that dates be provided for issue "Detection date" and "Notice date." These terms are defined in Handbook 401 and the validation handbook; and the validation handbook was changed during spring 2003 to make it consistent with Handbook 401 on the matter of detection date for states whose law or policy requires a week to be claimed for a nonmon to be valid. All states must report the data for this measure in accordance with these instructions.

j. Question: Benefit overpayments reported on the 227 Report should be included in the "dropped" category on the report when they are no longer in active status. How is it determined when overpayments should be categorized as "dropped"?

Answer: For reporting purposes, overpayments should be classified as "D" (dropped) when the overpayment has been reported for nine or more quarters and was in active collection in the prior quarter but has been removed from active collection status during the report quarter. These criteria are only for reporting purposes, and do not affect how long state law or policy requires them to keep overpayments "on the books."

k. Question: The DV software automatically eliminates duplicate records when deriving counts for summary validation. The criteria used by the software to detect duplicate records sometimes result in the identification of records as being duplicate when, in fact, they are not. How can states cope with the situation when records are erroneously identified as duplicate and eliminated from the data set resulting in failure for a given population?

Answer: The criteria built into the software consistently apply federal requirements. These identify "true" duplicate records in most states but are too simple to address all situations in all states. Instances exist where records appear to be duplicates when, under unique state procedures and processing conventions, they are legitimately reportable cases. For example, the software identifies as duplicates multiple initial claims filed by the same claimant (same Social Security Number) in the same quarter. Some states, however, do permit more than one claim to be filed when no valid claim establishing a benefit year has already been filed. These are legitimate claims, reportable on the ETA 218 and 5159 reports, as long as they result in an appealable monetary determination. Based on our discussions with state staff, it appears that such cases are relatively infrequent, and the ±2% validity standard will easily accommodate them. Thus, it did not seem advisable to introduce the additional complexity into the extract files and the software needed to determine they are legitimate claims. States should note the number of such cases in the Comments field and explain why they are legitimately reportable cases. If it should happen that they exceed the ±2% threshold, the comments will serve both as an explanation and as a guide for the design of future versions in the software.

\

Tax Questions

a. Question: The "number of liable quarters" is defined as the "the number of quarters between the date the employer last became active . . . ." This implies that the count of quarters should start over again when the account is reactivated. How is employer status established?

Answer: The Tax Population One extract file in the current version of the tax software includes a data element for "reactivation process date." The number of quarters that a given employer is in active status can be established using this element in combination with quarterly wage file extract data. The criteria for the identification of active employers uses data from Tax Population One as explained on page A-4 in the UI Tax Data Validation Handbook.

b. Question: The count of new employers required for item 14 on the ETA-581 Report is susceptible to state-specific definitions and administrative procedures. Some states assign account numbers to potential employers before they actually establish a payroll. The DV software rejects employers for whom no records indicating wages paid by that employer were submitted. How can the validation methodology be reconciled with state policy in this regard?

Answer: ETA-581 reporting instructions specify that potential employers who have not actually met a specific threshold or condition of liability contained in a state's unemployment compensation law do not meet the definition of "employers" and should not be included in the count of new employers until wages paid data indicate that they are "liable employers."

c. Question: Items 53 through 58 on the ETA-581 Report pertain to audit data for under- and over-reported wages and contributions. Should these amounts be "netted" with only the net results reported?

Answer: No. The under- and over-reported amounts should be reported separately for each quarter. Reporting the amounts separately provides information about how accurately employers are reporting wage information.

d. Question: Some states do not require reimbursable employers to report wages to the UI agency. Claims filed against these employers require that wage information be obtained by request. Because there is no record of wages paid during preceding quarters, the DV software does not identify these employers as "active". How can this discrepancy be dealt with?

Answer: One criterion for "active employer" status for data validation is evidence of wages paid. States (such as Delaware) that do not require reimbursing employers to report wage records should describe the situation in the comments section of the summary report. RO coordinators will review comments when deciding when to impose CAPs in response to specific DV issues.

e. Question: Category 201 on the ETA-581 Report requires states to identify the number of employer accounts which have been "resolved" through obtaining a report, determining non-liability for taxes, or issuing an assessment. According to ETA 581 definitions, reports can only be resolved by assessments if the assessments are "final." State procedures vary considerably on this matter; some consider a report resolved when they issue an assessment that is legally binding, even though it is not a "final assessment" because the appeal period is not exhausted. Others only classify the account as "resolved" when the final assessment is issued. These different interpretations of when an account is resolved by assessment can affect how long it takes to resolve reports and thus distort the information provided in the "resolved reports" category of the ETA-581 Report. How can this apparent distortion be avoided?

Handbook 401 defines a Resolved Report as, "A contribution report which has been received or resolved by a final assessment of tax that is legally due and collectible or by a determination of non-liability." Unfortunately, "legally due and collectible" and "final assessment" do not mean the same thing in all states. Many states consider an assessment "legally due and collectible" when it is issued; it becomes "final" only when the employer's opportunity to appeal the formal legal notice of the amount of the unpaid contributions has expired. In deciding when a report is resolved for ETA 581 purposes, states should be guided by the two elements that normally make up a Final Assessment. These are: first, that the employer's appeal period must have passed; and second, that the tax must be "legally due and collectible." States should therefore use the date on which the assessment has satisfied these two elements to indicate when a report is resolved.

\

General Questions

a. Question: How are states to submit validation reports to the NO and ROs?

Answer: Tax Version 1.5 and Benefits Version 1.6 of the software contain an export function that allows the results to be exported as a text file. E-mail these results to the NO to [email protected] with a copy to the appropriate RO DV coordinator. Whenever possible, obtain results from these versions and send the text files; they can be input directly into the Access® database the NO will use to store and retrieve UI DV results. For Tax Version 1.4 and earlier, and Benefits Versions 1.5 and earlier, use the Excel® Spreadsheet export option and e-mail to the NO (using the above address) with copy to the RO coordinator. Sending results in hard copy (e.g., using the .pdf option) is acceptable but is discouraged because the results must be hand-entered into the database, increasing the time for data entry and the risk of error.

b. Question: What kind of CAPs will be required for UI DV activities during FY 2004 and 2005?

Answer: For the FY 2004 SQSP, CAPs were required only for incomplete implementation in FY 2003, i.e., a state did not implement UI DV at all, or did not complete one or more populations. (Although no CAPs will be required for populations that failed quantity or quality validation, states will be expected to remedy reporting problems in preparing for the re-validation within a year.) For the FY 2005 SQSP, CAPs will be required for any populations that remain incomplete. In addition, CAPs will be required for populations or sub-populations that failed the quantity or quality validation criteria in FY 2004.

c. Question: When will a state be required to conduct a Workload Validation (WV)?

Answer: The Department's overriding concern is to implement UI DV as quickly as feasible, and thus the basic principle we have applied in deciding whether states should conduct WV, or receive a waiver, is whether doing WV will divert resources from UI DV and thus delay its implementation. Unless UI DV will not be implemented for a significant time--approximately a year or more--after the initial reporting deadline of September 30, 2003, we would be inclined to waive WV so that efforts can be concentrated on UI DV. This decision will be based on the recommendation of the RO coordinator. The request for a waiver should be in a formal letter to the Regional Administrator citing the reason(s) for the waiver.

d. Question: When reporting errors are found in the course of validation, how far back should reports be revised?

Answer: It would be useful to revise reports as far back as can be done at reasonable cost so that analyses and projections which rely on this data will be accurate. Because actuarial projections--e.g., of the number of nonmonetary determinations used as workload items--are usually based on data over time (time series), accurate time series data help produce accurate projections. However, costs of making retroactive changes can often be high. Consult with RO or NO staff regarding the importance of making retroactive changes to specific data series.

e. Question: How will DV results affect the interpretation of performance results and the size of administrative allocations?

Answer: Policy regarding how DV results will be used in the calculation of funding allocations and the calculation and interpretation of UI PERFORMS and GPRA performance measurements, and when the Department will begin using DV results for these purposes, is under development. This issue will be addressed in a subsequent advisory.

f. Question: How can validation be accomplished in cases where a state generates Federal reports from a cumulative detailed record file which is incremented on a daily, weekly, or monthly basis?

Answer: As explained in the Data Validation Handbook, the DV methodology assumes that report extract files can be constructed to replicate reported numbers. The approach described above does not lend itself to this methodology because the record file may not contain the actual set of records used to generate the report (e.g., the file may simply contain an incremented count of certain transactions). If a state produces some or all of their Federal report elements in this way, validation will consist of an independent count as described in the Handbook (e.g., UI Tax Data Validation Handbook, Appendix B). RO coordinators should be notified of the situation and provisions made to transmit the validation documentation to the NO.

g. Question: States sometimes discover through the DV process that data gathered for a Federal report is incorrect or incomplete. Even if the state has a commitment to correcting the problem it has no way of preparing correct and/or complete report information until the problem is diagnosed and corrected. Should the state provide report data even if it knows that it is not correct?

Answer: States should never knowingly provide incorrect or incomplete report data. If the status of the data is suspect, this information should be provided in the comments section of the report and subsequently corrected. Cells for which data are questionable should not be left blank. ET Handbook 401, 3^rd^ Edition, Introduction and General Reporting Instructions states, "For six regular program workload reports (ETA 5159, ETA 5130, ETA 218, ETA 207, ETA 581, and ETA 586) any values not filled in will prevent the report from being transmitted to the National Office. Incomplete reports are not acceptable. For non-workload reports or for workload reports other than regular versions, cells not filled in are assumed to be zero and are automatically zero filled when left blank."