Adverse Uses of AI

Every general-purpose technology gets used badly. What is worth understanding is the specific way AI changes the picture: it does not usually invent new harms, it removes the cost and skill barriers that used to limit them.

The categories

Synthetic media. Convincing fake images, audio and video of real people. Voice cloning in particular now needs only a short sample. This turns a resource-intensive attack into a cheap one.

Misinformation at scale. Generating unlimited plausible text makes it economic to flood a channel with fabricated reviews, comments or news. The constraint on such campaigns used to be human writing capacity.

Surveillance. Facial recognition and behaviour analysis applied to whole populations. The technology is neutral; the governance around it is not, and it varies enormously by jurisdiction.

Targeted manipulation. Using behavioural data to identify who is persuadable on what, and reaching them with tailored messages.

Autonomous weapons. Outside most organisations' concerns, but a live subject of international debate.

The economics is the story

Consider fraud where a finance employee is deceived into transferring money.

Traditionally this needed an email that looked convincing. Staff were trained to be suspicious of unusual email requests, and to verify by phone.

Voice cloning changes the arithmetic. The verification step — call the person and check — is now attackable with a short sample of their voice, which for any executive who has spoken publicly is freely available. There have been reported cases internationally of voice-cloned executive fraud, and the pattern is well enough established that it belongs in ordinary finance controls.

Nothing about the fraud is conceptually new. What changed is that a control which used to work no longer does.

The organisational response is not detection. Do not train staff to spot fake audio; that is a losing race. Change the control so it does not depend on recognising a voice:

• Payment changes above a threshold require verification through a separately initiated channel — you call a number from your own records, never one supplied in the request.
• Two-person authorisation for transfers above a limit, with no exception for urgency.
• An explicit policy that urgency and secrecy are themselves red flags, because every one of these attacks uses both.

That third point is the most valuable. Attacks of this kind universally apply time pressure and instruct the target not to discuss it. Naming that as the signal is more durable than any technical detection.

Erosion of trust

There is a second-order effect that is easy to miss and arguably more consequential than any individual fake.

Once convincing fakes are known to be possible, genuine evidence becomes deniable. A real recording can be dismissed as synthetic. This is sometimes called the liar's dividend: the mere existence of the technology gives cover to anyone caught on record.

The practical consequence for organisations is that provenance becomes more valuable than content. Where it matters that a communication is genuine, the answer is not better fake-detection but verifiable channels: signed documents, authenticated systems, recorded approvals in a system of record rather than a message.