AI in public procurement: what a human approver needs to see before signing

A simple test shows whether an agency’s human checkpoints mean anything: give a colleague ten AI-prepared decisions and ask her to explain each, using only the paperwork. Where she can’t, you’ve found the checkpoint to fix before a supplier challenges it.

Rafael Patron is the President of the Scientific-Technical Committee at Italy's Professional Association for AI Practitioners (AIPIA).

Picture a procurement officer on a Friday afternoon with forty items in her approval queue: supplier shortlists, draft purchase orders and two flagged invoices, all prepared by the agent that arrived with the last ERP update.  


The question for artificial intelligence (AI) in public procurement is no longer whether a human signs off. She will.  


What matters is whether that signature means anything when an auditor or an excluded supplier asks about it months later. 


In his guest column on GovInsider, Animesh Dhole describes a maturity path from "assist" to "act within limits". He warns that an agency whose tools work at the "act with approval" stage while its policies still assume "assist" is exposed.  


That approval stage is where many public buyers now work, and it deserves a closer look at the desk where the clicking happens. 

When "approve" turns into a rubber stamp 


Approval checkpoints sit at the centre of most guidance on agentic AI.  


Singapore's Model AI Governance Framework for Agentic AI, launched in January 2026 by the Infocomm Media Development Authority (IMDA), asks organisations to make humans meaningfully accountable by defining significant checkpoints at which human approval is required.  


The same announcement names the problem of automation bias, which is the tendency to over-trust an automated system that has performed reliably in the past. 


The EU's AI Act uses almost the same words.  


Article 14(4)(b) of Regulation (EU) 2024/1689 expects the people who oversee a high-risk system to remain aware of the tendency to rely, or over-rely, on its output.  


It singles out systems that recommend decisions to be taken by people.  


Many procurement tools may sit outside the regulation's high-risk category, yet the description fits the buyer's desk exactly.  


An agent that has been right for three months trains its approvers to stop reading. 


The fix is a better checkpoint, which usually means a smaller queue with more information in each item. 

What the approver must see on the screen 


A checkpoint is only as good as the evidence placed in front of the person who signs.  


Dhole's own recommendation points this way: for each approval gate, define who approves, what they must be able to see and what they are allowed to stop.  


In practice, an AI-prepared shortlist or purchase order should arrive with a short evidence pack: 


  • what the agent did, in plain words, and which rule or threshold triggered it; 

  • the data it relied on, with the date of the supplier records and any gaps it found; 

  • who was ranked out or never considered, and the main reason in each case; 

  • what has changed since the last comparable decision, including vendor updates to the scoring logic; 

  • the options open to the approver: accept, amend, reject or send back for a manual review. 


The fourth item matters more than it looks. If the approver cannot see that the evaluation template changed, she is approving a process she has never seen.  


The last item matters for a different reason. An approver with no visible way to say no is only a witness, and the EU text lists the ability to disregard, override or reverse the output among the conditions of effective oversight. 

A record that survives an audit and a supplier challenge 


The second test of a checkpoint comes later, when someone asks what happened.  


Here, a European text offers a useful counterpoint because it is written partly from the supplier's side: Article 30 of Italy's public contracts code, Legislative Decree 36/2023, sets three principles for automated decisions across the contract lifecycle.  


Firstly, every economic operator has the right to know that automated decision processes concern it and to receive meaningful information about the logic used.  


Secondly, the decision must never be exclusively algorithmic: there has to be a human contribution capable of checking, validating or overturning it.  


Thirdly, the contracting authority must take technical and organisational measures to prevent discriminatory effects on operators. 


Read from a buyer's desk, those principles translate into a record with a specific shape.  


It should show which evidence the approver opened, whether she changed anything, and the reason for her decision in her own words, even one line.  


A log that records only "approved, user 0147, 16:42" proves that a button was pressed. It says nothing about whether a human contribution capable of overturning the output existed. 


It’s worth tracking from the first week how often approvers amend or reject what the agent prepares.  


A rate close to zero over hundreds of decisions is a warning. 

Watching for drift, and for the supplier who never makes the shortlist 


Approval catches errors one decision at a time. Some errors only appear in aggregate.  


Dhole recounts a sourcing evaluation template that quietly reverted to an older scoring structure, caught only because people were watching. He also notes that small, new and local suppliers are often the first to lose out when scoring learns from past awards. 


Neither problem is visible from inside a single approval. Both become visible in a sample. 


A scheduled review run jointly by procurement, audit and IT, as Dhole suggests, can ask what no individual approver can answer: which suppliers stopped appearing on shortlists, whether first-time bidders fare worse than incumbents with similar offers, and whether results shifted after a vendor release.  


For agencies that publish a summary, the same figures become evidence that oversight is real. 

This work rests on data quality, which is where Asia-Pacific governments are already putting money.  
 
In January 2026, IDC's Ravikant Sharma cited the firm's prediction that 40 per cent of national governments in Asia/Pacific excluding Japan will invest 10 per cent of their IT budget in data architecture and governance this year.  


The aim is to close the gaps that stop them benefiting from agentic AI. 

Skills for AI in public procurement come before the rulebook 


None of this works if the approver does not understand what she is looking at.  


The AI Act is direct on this point. Article 26(2) asks deployers of high-risk systems to assign human oversight to people with the necessary competence, training and authority.  


Article 4, as amended by Regulation (EU) 2026/1744, asks providers and deployers of any AI system to take measures to support the AI literacy of their staff.  


Knowledge, experience and the context of use all count.  


IMDA's framework makes the same link: its fourth dimension of end-user responsibility rests on transparency and on education and training. 


For a buyer, literacy is concrete.  


It means knowing why a ranking can mirror past awards, recognising when an explanation is too generic to support a decision, and feeling entitled to send a recommendation back. 


This is also where public buyers, auditors and the people who build these systems need a shared vocabulary.  


The Associazione Italiana Professionisti dell'Intelligenza Artificiale (AIPIA), an independent non-profit based in Rome, brings together developers, data scientists, researchers, consultants and companies working with AI: the people on the other side of the approval screen.  


For readers following the European side of the question, AIPIA's guide to the AI Act, in Italian and updated after the 2026 amending regulation, explains the role of the deployer and the literacy duty. 


It sets out first preparation steps, from an inventory of systems in use to staff training.  


An inventory of that kind is the same exercise Dhole proposes when he asks agencies to map what is already live. 


The forums where GovInsider's readers compare notes on AI in government are a natural place to test that common language between procurement, audit and IT teams across jurisdictions. 

A test for next month's sample 


There is a simple way to find out whether an agency's checkpoints are meaningful.  


Take ten AI-prepared decisions approved last month and hand the records to a colleague who was not involved. Ask her to explain each decision, and the suppliers it left out, using only what was written down. 


Where she can, the human contribution is on the record. Where she cannot, the agency knows which part of the checkpoint to rebuild before a supplier asks the same question in a formal challenge. 


-----------------------------------------


The author is the President of the Scientific-Technical Committee at Italy's Professional Association for AI Practitioners (AIPIA), which is a professional network for AI specialists who are data scientists, developers, engineers, and businesses.