Humanity & AI Safety Accord
Menu

Incidents

Level 1. Deviation

Man's wrongful arrest puts NYPD's use of facial recognition surveillance tech under scrutiny

Developer
Not established
Model
Not established
When it happened
August 1, 2025
When it was disclosed
Not established
How well it is stood up
Not verified yet
What caused it
Not established
Where it stands now
Not established

What happened

Reported by abc7ny.com. Listed here with a link to their report. Nobody on this side has checked it against that report, so it carries no level and this is not our account of what happened.

Our assessment

Actual
1 of 5Deviation
Potential
3 of 5Significant control incident

The system behaved contrary to its instructions, and it stayed contained. The potential level is what the behaviour demonstrated here could credibly have led to, judged from the same evidence.

This level rests on news reports. The developer has not confirmed it here.

Why this level

  • Sensitive data was reachable. Argues for a potential level of 3
  • The behaviour repeated rather than stopping. Argues for a potential level of 3

What is established, and what is not

Confirmed

Nothing is confirmed yet.

Reported, not confirmed

  • The behaviour repeated rather than stopping. A claim, so it counts toward potential only

    In a letter sent to authorities including Inspector General Jeanene Barrett, Legal Aid detailed an alarming pattern of false arrests based on facial recognition data, and claimed that the NYPD is relying on facial matches sourced from outside its approved photo database, and that the NYPD is relying on other city agencies, like the FDNY, to do what the NYPD is barred from doing.
    Man's wrongful arrest puts NYPD's use of facial recognition surveillance tech under scrutiny
  • Sensitive data was reachable

    In the instant case, law enforcement officials extracted still images from video surveillance; employed unauthorized facial recognition technology software to identify a suspect; inappropriately accessed a DMV database to retrieve the suspect's photograph; technologically altered the physical features depicted therein; and finally placed the changed photograph in an array for identification.
    People v Zuhdi A. (2025 NY Slip Op 51047(U))

Checked, and did not happen

Nothing here was checked and ruled out.

Not confirmed

Nothing is outstanding.

Where this came from

The reports above are other people’s. The levels are ours, worked out under our published rules. No organisation that reported this assigned our number, and none of it is legal advice.

Something here is wrong? Tell us. Send us a source we can open, and a person will read it.

Man's wrongful arrest puts NYPD's use of facial recognition surveillance tech under scrutiny, Humanity & AI Safety Accord