The EU should establish an independent review mechanism to monitor the implementation of its large-scale IT systems for border management, the EU’s Fundamental Rights Agency (FRA) said in its 2023 annual report.
The report, published Thursday (8 June), surveyed the EU’s progress in the development of three new large-scale IT systems to facilitate border management and mitigate internal security risks, in part through AI-assisted risk profiling.
While the legal instruments establishing these three systems have “significant fundamental rights safeguards, including in relation to data protection and discrimination”, the FRA cautioned that “the implementation phase is crucial to ensure that these safeguards are not purely on paper”.
As the new IT systems are still under development, with much of the detail on how they will work together still hazy, their “potentially vast impact on fundamental rights therefore remains partly unknown”.
Digital rights advocacy groups and civil society researchers have echoed the FRA’s concerns, particularly regarding the operations of the European Travel Information and Authorisation System (ETIAS), expected to launch in 2024.
While such technologies can expedite decision-making processes, they also increase vulnerabilities for asylum seekers, argued Derya Ozkul, a researcher at the University of Oxford, adding that they “involve inherent risks of bias, discrimination and potential ‘machine mistakes’”.
For visa-exempt third-country nationals who make an online application to enter the Schengen zone, ETIAS will screen the applications against databases to identify if the applicant poses a security, irregular migration or epidemic risk, using algorithms to make predictions.
According to the EU’s border agency, Frontex, 97% of ETIAS applications are expected to be processed within minutes, while the remaining 3% will be flagged for review.
A Fundamental Rights Guidance Board, of which the FRA is a member, will have an advisory role on ETIAS’s operations.
“The advice of the board will play a key role in mitigating the risk that algorithms developed for ETIAS may have discriminatory effects,” FRA wrote in its report.
Complementing the work of ETIAS, the European Criminal Records Information System for Third-Country Nationals (ECRIS-TCN) will establish a centralised database for identifying convictions of third-country nationals.
Additionally, the automated Entry/Exit System (EES) will register biometric data for crossings into and out of the Schengen area by third-country nationals, alongside refusals of entry, with the aim of simplifying calculations of short stays across the border-free zone and identifying overstays.
Fundamental rights impacts
However, the use of AI in risk management and profiling systems “raises a broad range of structural and fundamental rights considerations”, digital rights advocacy group Access Now highlighted.
Such predictive technologies, based on predetermined risk indicators such as nationality, geographic location, education levels and job groups, “by nature violate the right to non-discrimination in the migration context insofar as they codify assumptions about the link between personal data and characteristics with particular risks”, according to Access Now.
The deployment of such technologies risks violating foundational tenets of international refugee and human rights law, such as the principle of nonrefoulement – the illegal return of an asylum seeker to their country of origin.
In 2020, the UK Home Office announced it would scrap an algorithm used to categorise visa applicants in a “traffic light” system, partially based on nationality, following a campaign by digital rights group Foxglove.
How to regulate AI-powered tools in migration and border control is a hot issue in the context of the AI Act, the EU’s flagship legislation designed to regulate artificial intelligence based on its potential to cause harm.
The European Parliament is set to vote on its position on the file next week, and left-to-centre MEPs are pushing amendments that would ban the use of algorithms to risk-assess migrants or predict migration flows.
AI Act’s plenary vote cast with uncertainty as political deal crumbles
The agreement among the leading groups in the European Parliament on the AI regulation is dead, opening the door for amendments from both sides of the aisle.
By contrast, the EU Council of Ministers has lightened up the requirements for AI systems used in the areas of law enforcement and migration control.
“The European Parliament must enter AI Act trilogues with the strongest possible position to protect the rights of all people inside and entering the EU — regardless of their migration status,” said Caterina Rodelli, EU policy analyst at Access Now.
[Edited by Luca Bertuzzi]
Read more with EURACTIV
UK to host world’s first AI summit: PM SunakBritain will host the world’s first summit on artificial intelligence later this year, in a bid to broker a common approach by countries to limit the technology’s potential doomsday risks while harnessing its potential, Prime Minister Rishi Sunak said Wednesday (7 June).
Source: euractiv.com
