AI safety boundaries put privacy and human control to the test
Published on 26 September 2026 at 00:09 GMT
By Editorial
Artificial intelligence can influence access to employment, education and public services. AI safety boundaries concern the evidence used to assess these systems, restrictions on their uses and the authority to intervene when their outputs affect people’s lives.
Existing approaches address different aspects of these decisions. European regulation separates prohibited practices from uses subject to safeguards. Data protection guidance examines the treatment of personal information. International ethical principles and a US voluntary framework address responsibility across development and deployment. Their differences matter: an ethical commitment, an internal assessment and an enforceable legal obligation do not carry the same consequences.
Data protection before deployment
The European Data Protection Board (EDPB) addressed model training and deployment in its December 2024 opinion. Models trained on personal data cannot automatically be considered anonymous. The board calls for a case-by-case assessment of whether the likelihood of extracting or obtaining those data, including through queries, is insignificant, considering the means reasonably likely to be used.
The board also explained that legitimate interest can provide a legal basis only after an assessment of the interest, necessity and the balance against individuals’ rights. Public availability is one factor in assessing reasonable expectations, alongside the collection context and the relationship with the organisation processing the data. A publicly accessible post therefore does not, by itself, settle whether reuse is justified.
The EDPB said unlawful processing during development could affect the lawfulness of later deployment, depending on the circumstances. Duly anonymising a model can separate later processing from that earlier infringement, but new personal data processed during deployment remain subject to the General Data Protection Regulation (GDPR). Anonymisation does not retrospectively legalise the original processing.
Preventing harm without treating every use alike
The European Commission describes the EU AI Act as a system of rules proportionate to risk. Its prohibitions include untargeted scraping of internet or CCTV material to build facial recognition databases. Its high-risk categories include certain employment, education and public-service applications. The distinction separates uses excluded by law from activities that remain possible under specified conditions.
Timing is part of that distinction. The European Commission’s updated implementation schedule places rules for specified high-risk uses from 2 December 2027 and those for systems integrated into regulated products from 2 August 2028. Presenting all safeguards as already applicable would obscure that transition. The European Commission also identifies regulatory sandboxes and simplified requirements as instruments intended to support innovation and compliance.
For an applicant affected by an automated assessment, oversight concerns the information behind the result, the detection of errors and the possibility of changing an outcome. These issues extend beyond the software’s performance to the procedures used when its outputs inform decisions.
The US National Institute of Standards and Technology (NIST) offers a voluntary risk-management framework organised around governance, mapping, measurement and management. It calls for testing before deployment and during operation, with attention to context and uncertainty. It also includes feedback from affected communities and processes for reporting problems and appealing outcomes.
That framework treats risk assessment as continuing work. A favourable test result is evidence about the conditions tested, rather than a universal assurance about every later use. NIST also identifies independent review as a way to address internal bias and conflicts of interest. These are proposed management practices, not a global licensing regime or a guarantee that harm cannot occur.
Human oversight with the ability to intervene
Article 14 of the EU AI Act specifies a concrete model of human oversight for high-risk systems, subject to the applicable implementation timetable. It addresses understanding capabilities and limitations, recognising anomalies, interpreting outputs and avoiding automatic over-reliance on machine recommendations. It also provides for the ability to disregard, override or reverse an output and to interrupt operation through a safe stopping procedure.
Oversight measures are linked to the system’s risks, autonomy and context. The provision therefore goes beyond the presence of a person at a screen. For example, in a hypothetical recruitment process, an employee who sees only a ranking faces a different task from an assessor who can examine relevant evidence and reject the ranking. The distinction concerns the information and intervention available, rather than the job title of the person supervising.
Implementation also involves decisions about review time, escalation and responses when a human disagrees with a system. A nominal approval step does not establish whether a reviewer has the information and authority described in Article 14.
Who sets the limits
UNESCO’s Recommendation on the Ethics of Artificial Intelligence, adopted in 2021, locates ultimate responsibility with humans and calls for participation by diverse stakeholders. It advocates proportionality, privacy protection, auditability and impact assessment. Its position makes governance broader than decisions by developers alone, while recognising national sovereignty and international law.
Those principles leave room for disagreement about institutional design. How much authority belongs to legislators and regulators, how much technical discretion remains with developers, and how can affected communities influence decisions? Public participation and specialist assessment perform different roles: one concerns whose interests are represented; the other examines evidence about a system’s operation. Neither question disappears when a product performs well on a benchmark.
UNESCO also recognises tensions between transparency, privacy and security. More disclosure is not automatically harmless if it exposes personal information. Conversely, restricted access raises questions about whether an assessment can be independently examined. The balance between safety and innovation consequently involves several boundaries, each attached to a particular use, source of data and form of accountability, rather than a single declaration that a technology is safe.
Written by a human author, edited with AI assistance.
Further information:
European Data Protection Board, Opinion 28/2024 explains anonymity, legitimate interest and the consequences of unlawful processing during model development.
European Commission, its AI Act overview supports the risk categories, implementation timetable and innovation measures.
National Institute of Standards and Technology, AI RMF 1.0 sets out the voluntary framework’s testing, monitoring and accountability practices.
European Commission, the AI Act Service Desk reproduces Article 14 on human oversight and intervention.
UNESCO, its ethics recommendation explains proportionality, stakeholder participation, human responsibility and tensions between transparency and other protections.
