Unemployment data validation activities for 2007 and 2008

UIPL22-05_Ch1_508.pdf (https://www.dol.gov/sites/dolgov/files/ETA/advisories/UIPL/2006/UIPL22-05_Ch1_508.pdf)

EMPLOYMENT AND TRAINING ADMINISTRATION ADVISORY SYSTEM
U.S. Department of Labor
Washington, D.C. 20210
CLASSIFICATION: Data Validation
CORRESPONDENCE SYMBOL: OWS/DPM
DATE: July 21, 2006

ADVISORY: UNEMPLOYMENT INSURANCE PROGRAM LETTER NO. 22-05, Change 1
TO: STATE WORKFORCE AGENCIES
FROM: EMILY STOVER DeROCCO, Assistant Secretary
SUBJECT: Unemployment Insurance Data Validation (UI DV) Program Activities During Validation Years (VY) 2007 and 2008 and Policy Clarification

1. Purpose. To provide updated guidance on UI DV activities for VY 2007 and VY
   2008, and to clarify certain DV policies.

2. References. Unemployment Insurance Program Letter (UIPL) No. 22-05,
   “Unemployment Insurance Data Validation (UI DV) Program Software and Policy
   Guidance.”

3. Background. In June 2005, the Department installed data validation software that
   runs on the state Sun computers and, beginning July 1, 2005, states were required to
   use the software to validate reports data and submit results to the National Office
   (NO). UIPL No. 22-05 provided guidance regarding validation schedules and
   explained the basic rules for determining whether a validation passed or failed and
   when various validations needed to be done and submitted.

   As state validators began using the software in regular validation activities to
   transmit results to the NO, various deficiencies became apparent. These involve the
   software itself, the data transmission, and the structure of the database that stores
   DV summary results from which the NO determines compliance.

RECISSIONS: None
EXPIRATION DATE: July 21, 2007

4. Revised Validation Schedule and Activities. Until the software and database
   deficiencies have been corrected, we are suspending software-related validation
   submissions for VY 2007, i.e., validations of UI required reports for the period
   April 1, 2006, through March 31, 2007, that would be due by May 10, 2007. Other
   DV activities will continue as discussed below. A new version of the software is
   under development and is scheduled for release in late calendar year (CY) 2006.
   Version 1.1 will correct the software bugs identified, improve duplicate detection
   criteria, facilitate validators’ use of the software, and enhance the Department’s
   ability to administer the DV program. Shortly after Version 1.1 is released, the
   Department will issue revised versions of the generic Benefits and Tax DV
   handbooks. The revised handbooks will reflect the policy guidance in UIPL No. 22-05, clarify the specifications for building extract files and incorporate references to
   the new software.

   VY 2007 Actions
   • States are required to:
      • Refrain from submitting any validations of populations using the DV
         software.

       • As in VY 2006, submit results from the validation of Tax Performance System
         (TPS) and Benefits Timeliness and Quality (BTQ) samples in Microsoft Word®
         documents as e-mail attachments sent to [email protected] using the
         template posted on the Data Validation Web page at
         http://ows.doleta.gov/dv/.

              •   Because the BTQ sample validation includes determining whether the
                  correct sample size is being drawn for each of the CY’s quarterly
                  reviews, the BTQ validation should always occur before the state
                  conducts the quality review for the first quarter of the calendar year.

       • In the State Quality Service Plan (SQSP), address failures to implement DV,
         i.e., failures to build extract files for all populations, or to submit BTQ or TPS
         validations. States do not need to address in the SQSP a population that
         failed VY 2006 validation, due to the possibility that the software caused the
         failure. If a state believes a software problem caused a population to fail, it
         should provide documentation of this to the NO through its regional
         coordinator.

       • Conduct or have completed Independent Validation and Verification (IV&V)
         activities if Fiscal Year 2005 Supplemental Budget Request funds were
         received for them. (Clarified extract file record layout specifications and the
         Report Validation Specifications necessary to conduct IV&V activities are
         available on the Data Validation Web page at http://ows.doleta.gov/dv/.)

       • Complete unfinished population extract files and assure that all extract files
         conform to the record layouts for all DV populations.

       • Update Module 3 and submit this information to the NO via e-mail to
         [email protected].

       • Obtain software training, if needed, from the Minnesota Training Center,
         after release of the new software. The Center’s scheduled offerings can be
         seen at http://www.tc.state.mn.us/calendar.html.

       • Upon the release of software Version 1.1, validators should begin to
         familiarize themselves with it in preparation to resuming the validation of
         reports for the VY beginning April 1, 2007, and inform the Department of any
         functionality problems.

   VY 2008 Actions
   States will be required to resume validation activity using software Version 1.1. This
   includes validations that were due for VY 2006 and were not done because of
   software problems or other reasons. The suspension of validation submissions for
   VY 2007 does not affect the timing of validations that are due for VY 2008; e.g.,
   validation populations due to be submitted in 2008 because they passed in 2005 are
   still due for 2008.

5. Data Validation Policy Clarifications. Through UIPL No. 22-05, the Department
   articulated policy and procedures regarding administration of the DV program.
   This change includes the following refinements:

   •   The status and use of Benefits Data Element Validation (DEV) samples other than
       random samples (generally called “non-random” samples); and
   •   Procedures for submitting results for validations of BTQ and TPS quality
       samples and Wage Items.

   Status and Use of Non-random Benefits Samples.
   The DEV process for Benefits DV uses a series of sample investigations to assess
   whether the data used for reporting from the state’s underlying database are
   consistent with Federal reporting requirements, and whether the validation extract
   files are properly built. There are 16 two-stage random samples, which are part of
   the formal process for determining whether 11 of the 15 populations pass or fail
   validation. In addition, there are numerous smaller “non-random” DEV samples:
   (a) minimum samples (two cases per subpopulation); (b) missing subpopulation samples
   (one case per each subpopulation not represented in a random sample); and (c)
   outlier samples (the five largest and five smallest observations, usually on the basis of
   time lapse, in a set of subpopulations.) Although these non-random samples are not
   used to determine whether a population passes or fails, they are important

   diagnostic tools to help ensure that extract files are built properly and that
   underlying data meet reporting definitions. Because they are not used in the formal
   pass/fail determination, the results of benefits non-random sample investigations
   do not have to be submitted to the NO during VY 2008, but as an integral part of the
   DV design they need to be done and the results saved outside the software on a local
   area network, CD, or another storage medium. Non-random sample results may be
   subject to regional office review, Office of Inspector General audit or other reviews.

   Submitting Results for BTQ and TPS Quality Samples and Wage Items
   BTQ, TPS, and Wage Item validation results are submitted independently from any
   population and are valid for three years if they pass.

   •   Wage Item validation must be repeated the following year if at least one mode
       does not pass with an error rate of 2 percent or less. Wage Item validation results
       are submitted using the DV software.

   •   For the review of BTQ and TPS samples,
          • If all validation components pass, then the results are valid for three years.

          • If a sample does not pass the test for randomness, the sample must be
            redrawn before the BTQ or TPS sample can be used. If the universe from
            which the sample is drawn is not within 2 percent of the reference
            population, the validation must be repeated the following year.

          • The results of these sample reviews are to be submitted in Microsoft
            Word®, using the template posted on the DV Web site, and sent by e-mail
            to [email protected].

6. Actions. State Workforce Administrators are requested to distribute this advisory to
   appropriate staff.

7. Inquiries. Direct questions to the appropriate Regional Office.