Businesses are being sold a clean story: "We will audit your automations and make you compliant."
Sometimes the seller is a lawyer who never sees how the system actually works. Sometimes it is an automation consultant making legal claims they are not qualified to make. The final deliverable is often a policy document, a training session, and a green box in a spreadsheet.
None of this proves that the deployed system still respects the original human intent that justified building it.
A useful illustration of how easily that gap opens comes from research published in July 2026. Reuters reviewed more than 80 Chinese academic papers and patents. Researchers had used outputs from large American models as training material for smaller, specialised systems designed to run locally, with limited computing power, for tasks such as analysing code, processing images, or operating when network connections fail.
Some capabilities do transfer. Anthropic notes that when large models are distilled into smaller ones, some of the original safety safeguards may be weakened or lost.
But the research doesn't show that key safety behaviours like refusing unsafe requests, expressing uncertainty, or escalating sensitive cases were consistently tested and confirmed to be missing.
And that is the real issue: we simply don't know what carries over and what doesn't. A smaller model can still look just as capable, while quietly losing the rules and limits that shaped how the original system was meant to behave.
This is not only a military story. The same architectural problem can appear when a business fine tunes a model, creates an internal agent, or trains a cheaper specialist system. Useful behaviour may move. The original purpose, refusal rules, and decision limits may not move with it.
A lawyer reviewing the supplier contract may not see this. An automation consultant checking that the workflow "works" may not see it either. Both can produce a clean compliance package while the operating system has drifted away from the intent that was supposed to govern it.
The AI Act doesn't require a lawyer to sign off every automation. It doesn't require 2 full time people in every company. Its obligations depend on the organisation's role and the particular use: provider or deployer status, risk classification, transparency, documentation, human oversight, monitoring, and other requirements.
Since August 2nd 2026, some transparency obligations have become applicable. The Commission has also opened 3 scope limited channels: an AI Act Complaint Tool for qualifying alleged infringements, an AI Act Whistleblower Tool for eligible insiders which can be used anonymously, and a dedicated complaints channel for downstream providers reporting possible failures by general purpose model suppliers.
These tools only become useful when evidence already exists. A policy document cannot reconstruct logs, model versions, approval trails, or missing supplier documentation that the system never recorded.
These 3 channels are already open. One of the first things that can now be reported is missing transparency.
From 2 August 2026, Article 50 transparency obligations apply. Users must be clearly told when they are interacting with an AI system, for example a chatbot or an AI voice instead of a human.
Certain AI generated or manipulated content must be disclosed visibly. This includes deepfakes and, under specific conditions, AI generated text published to inform the public about matters of public interest when there has not been meaningful human review or editorial responsibility. Providers of generative systems also have obligations around machine readable marking and detectability. The precise disclosure depends on the type of system, content and use, not everything created with AI requires the same visible label. For audio, such as AI generated podcasts, this usually means a spoken notice right at the beginning. For written texts, the practical expectation is a visible AI label placed above the text or near the headline.
Providers of generative systems must also embed machine readable markings so that other systems can detect the content was produced by AI.
Systems already on the market before 2 August have until 2 December 2026 only for the machine readable part. The duty to inform users they are talking to an AI, and the duty to label AI generated content, already apply.
Fines for breaking these rules can reach €15 million or 3% of worldwide annual turnover. Small and medium sized companies are not exempt from the obligation, although the size of the fine can be adjusted for proportionality.
A large number of companies using generative AI or customer facing chatbots are still catching up. A policy document that says "we comply with the AI Act" doesn't automatically put the required spoken notices or visible labels on the actual outputs people see or hear. That is another reason why a lawyer's signature or an automation checklist is not enough. The system that is running has to produce the transparent behaviour the law now requires. Businesses therefore need 3 things, not 2 job titles:
Someone who can show what the system actually does: which models, tools, and data are connected, which actions it can propose or execute, what happens when it is wrong, which records exist and who is responsible when things drift away.
Someone who can interpret what the law requires for that specific use.
One accountable owner inside the business who decides the residual risk, can stop the system, and defines what evidence must exist before automation expands.
External advisers can supply technical findings or legal advice. But they can't absorb the organisation's responsibility by just signing a report.
Here, Palantir's design pattern is useful because it is operational. Its Deployment Strategists and Forward Deployed Engineers (Echo and Delta team structure) approach the same institutional problem through complementary but overlapping roles. The Deployment Strategist (Echo) may focus more heavily on understanding the organisation, challenging the initial request and defining the operational outcome, while the FDE (Delta) may focus more heavily on turning that understanding into working software. But neither role begins only after the other has finished, and neither assumes that every problem requires AI.
As I started to study more closely some of their frameworks and their deployment platform, with Palantir AIP (Artificial Intelligence Platform), I saw that human users can review AI proposed actions, inspect execution rules and history, and approve critical decisions through structured checkpoints. Its governance guidance treats the complete system, data pipelines, permissions, interfaces, audit trails, evaluation and testing workflows, and human decisions, rather than only the model.
Its sovereign AI framing adds another test: can the organisation run across cloud, on premises, sovereign cloud, or the edge, change models, preserve its data, and continue operating without surrendering control?
Using an American vendor doesn't automatically make a European company sovereign. Sovereignty is demonstrated through control, portability, auditability, and realistic exit power.
A human approval button is still meaningless when the reviewer lacks the time, context, competence, or authority to reject the action.
Before approving the next automation, ask:
Can we identify the model, its version, and its origin? Which actions require human approval, and can that person genuinely stop them? If someone collected 100,000 outputs, what part of our decision logic could they reproduce? Can we keep operating and retain our data if the supplier or network disappears? If a complaint arrives tomorrow, what evidence could we produce?
Responsible AI doesn't start when a policy document gets a check mark. It starts when what is written in the policy actually matches what the system is doing in practice, and when a human can still understand, control, and stop the system if it starts behaving in a way that wasn't intended.
That is the gap I work on.
Sources
[1] Reuters, "Chinese military researchers tap US AI models to train defence systems," 31 July 2026. reuters.com
[2] European Commission, AI Act Service Desk FAQ. ai-act-service-desk.ec.europa.eu
[3] European Commission, AI Act Complaints Tool. digital-strategy.ec.europa.eu
[4] European Commission, AI Act Whistleblower Tool. digital-strategy.ec.europa.eu
[5] European Commission, Complaints channel for downstream providers using GPAI models. digital-strategy.ec.europa.eu
[6] Palantir, AIP Ethics & Governance documentation. palantir.com
[7] Palantir, Protect Your Sovereignty (Sovereign AI framing). palantir.com