Benefits fraud detection system studies

To Whom It May Concern:

Pursuant to the Colorado Open Records Act, I hereby request records related to ID verification and unemployment fraud detection tools used by the department.

In an Unemployment Insurance Program Letter published in November 2023 (UIPL No. 1-24), the U.S. Department of Labor provided the following instructions to state agencies responsible for unemployment benefits:

"When a state notices that individuals are unable to verify their ID through web-based or selfservice means, or the state flags individuals as suspicious with automated cross-matching or data analytics activities, states must generate and retain documentation of these instances. States are also required to investigate the flags and make appropriate determinations so that eligible claimants are able to access UI benefits without undue barriers or delay. The state UI agency should then analyze the data to determine if the state’s ID proofing system, fraud detection and prevention solutions, cross-matching and data analytic parameters, or service provider(s) correctly identified fraud or incorrectly flagged claimants. In line with their nondiscrimination obligations under 29 CFR §38.51, states should conduct statistical or other quantifiable data analyses to determine whether their fraud detection systems are disproportionately flagging certain groups."

I request:

1) All contracts the department has entered into since Jan. 1, 2020 for the provision of unemployment fraud detection tools. This includes identity verification tools and data analytics services such as risk-scoring and fraud-scoring algorithms.

2) All analyses of the accuracy of the department's fraud detection tools, as requested by the Department of Labor in UIPL No. 1-24. This should include analyses conducted by the department or provided to the department by the vendor or a third pary since Jan. 1, 2020.

3) All analyses of whether the department's fraud detection tools disproportionately flag certain groups, as requested by the Department of Labor in UIPL No. 1-24. This should include analyses conducted by the department or provided to the department by the vendor or a third pary since Jan. 1, 2020.

I ask that all fees be waived as I am a journalist and intend to used the requested records to publish articles in the public interest and not for any commercial purpose.

If you choose to impose fees, I request a detailed breakdown of the fees, including the hourly wage of each employee involved and an explanation of the employee hours required to fulfill the request.

If you choose to reject this request or redact portions of responsive documents, I ask that you provide a detailed breakdown of the statutory exemptions and associated case law underlying your decision to withhold each/any portions from public review.

The requested documents will be made available to the general public, and this request is not being made for commercial purposes.

In the event that there are fees, I would be grateful if you would inform me of the total charges in advance of fulfilling my request. I would prefer the request filled electronically, by e-mail attachment if available or CD-ROM if not.

Thank you in advance for your anticipated cooperation in this matter. I look forward to receiving your response to this request within 3 business days, as the statute requires.

Sincerely,

Todd Feathers

Hi,

Thanks very much for your quick response to this request.

You state that the the division has not implement any of the provisions of UIPL 1-24 and therefore no records responsive to items 2 and 3 of my request exist.

So to confirm: Colorado DLE is using a machine learning model to label unemployment claims as fraudulent, but DLE has no documentation of how accurate that machine learning model is?

Thank you,
Todd

Hello,

Thank you again for the quick response.

However, I'm a bit confused. Could you please explain how the records I've requested would reveal the security arrangements of critical infrastructure or aid in planning an attack on such infrastructure?

I've asked only for documentation of the accuracy and bias of the models, not any underlying code, database architecture, or other information that would jeopardize the security of the fraud detection system. If records responsive to my request also contain information that is exempt under C.R.S. 24-72-204 (2) (a) (VIII) (A), then the department is required by CORA to redact the exempt portions of the records but still provide the non-exempt portions for public inspection.

any jurisdictions require that agencies proactively publish accuracy and bias metrics for the automated decision systems they use. For example, the U.S. Office of Management and Budget's recent guidance directs federal agencies to publish the results of validation and disparate impact analyses on their websites.

Please acknowledge receipt of this message and confirm that you are processing the responsive documents you identified for release.