Letter to Buenos Aires Mayor Horacio Rodríguez Larreta re: facial recognition system and children’s rights

Washington, DC, October 9, 2020

Mayor Horacio Rodríguez Larreta
Buenos Aires — ARGENTINA

Re: Facial recognition system and children’s rights

Dear Mayor Larreta,

I am writing on behalf of Human Rights Watch to share our concern regarding the loading of images and personal information of children into a facial recognition system deployed in some subway stations of the city of Buenos Aires. Our research shows that the source database is used in ways that violate children’s rights to privacy in criminal proceedings and that it contains significant errors, and that the facial recognition technology has a higher risk of false matches for children.

As you know, Argentina’s Justice and Human Rights Ministry maintains a national database of individuals with outstanding arrest warrants for serious crimes, known as the Consulta Nacional de Rebeldías y Capturas (CONARC; National Reference of Fugitives and Arrests).[1] This database, which the ministry makes publicly available online, also includes the private personal information of children.

Analysis by Human Rights Watch reveals that at least 166 children have been listed in CONARC between May 2017 and May 2020.[2] Even children suspected of minor crimes are included. The most common crime that these children are accused of is theft–63 children, or 37.5 percent. Persistent errors and obvious discrepancies in CONARC, which is updated every morning at 7 a.m., indicate that the system lacks basic safeguards to minimize data entry errors, which can have serious consequences for a child’s reputation and safety.

Under international human rights law and standards, states should guarantee that the privacy of any child alleged to have committed a crime is fully respected at all stages of the proceedings.[3] No information should be published that may lead to the identification of the child.[4] On May 17, 2019, the United Nations Special Rapporteur on the right to privacy warned Argentina that its use of CONARC was violating children’s rights.[5] By publicizing the arrest records of children, the government is putting these children at risk of harm.

These harms have been amplified in the City of Buenos Aires. Since April 24, 2019, the city government of Buenos Aires has used the CONARC data—including the data on children—in its facial recognition system, the Sistema de Reconocimiento Facial de Prófugos (SRFP; Facial Recognition System for Fugitives).[6] That technology scrutinizes live video feeds of people catching a subway train, walking through or in the vicinity of a subway station, and identifies possible matches with the identities in CONARC. Because CONARC does not include photos of those charged with a crime, the SRFP pulls reference photos from the country’s population registry.[7]

In an official response to an information request by the Observatorio de Derecho Informático Argentino (Argentine Computer Law Observatory), a civil rights organization, the city of Buenos Aires’s Ministry of Justice and Security denied that the facial recognition system identifies children, as “CONARC does not contain the data of minors.”[8] As we established above, this is incorrect.

We are further concerned by the absence of public or legislative debate around the necessity and proportionality of the facial recognition system, especially considering the technology’s adverse impacts on children.[9] Facial recognition has considerably higher error rates for children, in part because most algorithms have been trained, tested and tuned only on adult faces.[10]

This is the case with the facial recognition technology used by the city of Buenos Aires, which was contracted to Buenos Aires-based company Danaide S.A. and developed by Russian company NtechLab.[11]

In tests conducted in a controlled lab setting, using photos posed under ideal lighting conditions, the US Department of Commerce’s National Institute of Standards and Technology (NIST), which evaluates the accuracy of facial recognition algorithms worldwide, found that the three algorithms that NtechLab submitted for testing produced a higher rate of mistaken matches among children than adults. [12]

Based on NIST’s evaluation results, Human Rights Watch calculates that, in a controlled setting, these algorithms falsely identify a child aged 10 to 16 six times more often than an adult aged 24 to 40. The younger the children are, the more pronounced the errors.

Since children experience rapid and drastic changes in their facial features as they age, facial recognition algorithms also often fail to identify a child who is a year or two older than in a reference photo. Because the facial recognition system matches live video with identity card photos collected by the country’s population registry, which are not guaranteed to be recent, it may be making comparisons with outdated images of children, further increasing the error rate.[13]

According to an official response to an information request by the Asociación por los Derechos Civiles (Civil Rights Association), another Argentinian organization, prior to its deployment the SRFP was only tested on the adult faces of employees from the city’s police department and the Justice and Security Ministry.[14] The city government did not request that Danaide S.A., its vendor, perform tests or checks to minimize bias against children.[15] Danaide S.A. and NtechLab did not respond to multiple requests for comment.

Moreover, error rates substantially increase when facial recognition is deployed in public spaces where the images captured on video surveillance cameras are natural, blurred, and unposed. Deploying this technology in the city of Buenos Aires’s subway system, with a daily ridership of over 1.4 million people and countless more passing through or near its stations, will result in people being wrongly flagged as suspects for arrest.[16] Buenos Aires is currently deploying the system on a small scale, for budgetary reasons. The European Union has estimated that when using live facial recognition in places visited by millions of people, like subway systems, even a relatively small error rate like 0.01 percent may result in hundreds of people being incorrectly identified once the system is scaled up to its full capacity. In public statements, the Buenos Aires government has spoken of error rates of 3 percent or higher.[17]

When using a new technology such as live facial recognition, it is necessary to assess the risks of wrong identification and the affected fundamental rights. For example, the Berlin police carried out a large facial recognition trial at a train station in 2017 and 2018, which reviewed live video from CCTV cameras and identified people against a watchlist. On average, the tests had a false positive identification rate of 0.34 percent; this meant that, out of every 1,000 people crossing the surveillance system, between three and four people would be wrongly identified as matches by the system. The German authorities deemed this unacceptable; given “the number of people crossing train stations every day, this would lead to a large number of people incorrectly stopped (or at least flagged to the police).”[18]

Official documents obtained by the Observatorio de Derecho Informático Argentino (Argentine Computer Law Observatory), reveal that the city of Buenos Aires police are stopping and detaining people solely on the basis of the automated alerts generated by the SRFP.[19] Adults have been mistakenly detained and arrested.[20]

Human Rights Watch has written a letter to President Alberto Fernández to request that the Ministry of Justice and Human Rights immediately remove all children (all individuals under 18) from the publicly available version of CONARC.

In turn, we respectfully urge the City Government of Buenos Aires to immediately suspend the SRFP, conduct a privacy and human rights impact assessment, and publish verifiable statistics on the system’s performance to date. The City Government should also invite meaningful engagement with civil society—in particular those with expertise on the right to privacy, the societal and ethical implications of this technology, and children’s rights—in assessing the necessity, proportionality, and legality of the use of live facial recognition surveillance, with special consideration given to the implications for the rights of children.

I take this opportunity to express to you my highest consideration and esteem.

José Miguel Vivanco
Human Rights Watch