"Is that photo real?" - how source authentication can protect our society in the AI era

Verification as a priority

It is not hard to imagine that things created by AI could also be used by people with less noble goals. And we see it already: the increasing presence of AI leads to an increase of misleading content and fraud. Is that photo ‘real’? Is the person holding this identification document a ‘real’ person? Protecting ourselves from manipulation and fraud should be a top priority. We need to be able to determine whether information used for identification and authentication (let’s call it ‘verification’) is real.

Every day, we initiate ourselves numerous times a verification process to gain access. Think of how you use your private phone, how you log in at work and how you do your financial transactions. We are aware that we need to rely on a solid identification and authentication process. Otherwise our (online) identity can be taken over, leading to misuse and other criminal activities.

From bank robbery to online crime

Criminals going after money are as old as time. However, the bank robbery of the 19th and 20th century is no more and has been replaced by digital and online crime: generative AI and deepfakes are already doing damage. See for example the message from CNN on February 4, 2024: Finance worker pays out $25 million after video call with deepfake ‘chief financial officer’. We know this will find a way to other sectors of society. The current criminal use of generative AI and deepfakes is just the beginning.

The risk of AI will only further increase, fraud and other misuse by criminals in banking and financial services. While at the same time, digital techniques and products (like Electronic Identification and Trust Services and the 'European Digital Identity Wallet' (EUDIW) as part of eIDAS 2.0 will be implemented. And attempts to misuse or take over identities in other sectors, will lead to criminal activities geared at financial services. Not to mention the possible impact on free and democratic societies. See for example the message from CNN on 23 May 2023: ‘Verified’ Twitter accounts share fake image of ‘explosion’ near Pentagon, causing confusion.

To successfully tackle this problem, we need to protect the authenticity of content, documentation and information. And we need to work together on this. When government, financial partners, big tech and other relevant parties all put their weight behind it and take ownership, a fast and yet careful solution is possible. Europe could and should play a pivotal role in this.

At the same time, there are already initiatives that show a possible route to solve the problem we are facing. The Coalition for Content Provenance and Authenticity (C2PA), for example, develops standards to verify media content origin and edits of digital content to combat online misinformation. And the C2PA consists of exactly the broad range of organisations needed, including media partners. And including big tech & platform owners. This bottom-up approach is very welcome, since it is important that a standard will be widely accepted and subsequently applied as quickly as possible.

The solution? Source authentication

By now, you probably think what we exactly need to do. The answer to that question: source authentication, or provenance. This technique enables us to verify the creator and origin of content. For example, a camera can use unique attributes provided securely by the operating system to confirm its authenticity. And can therefore provide the output with authenticity features. By appending a sort of digital signature, the camera enables recipients to verify that photos or videos are genuine and unaltered. This technique also helps prevent man-in-the-middle or live deepfake manipulation as hardware and used software authenticate themselves ensuring the video stream comes directly from the source without tampering. And from a privacy point of view, it is good to stress that it is not necessary for source authentication to be traceable to a personal or device level. 

This is not a wholly new idea. The music and film industry has gained a lot of experience with digital rights management for protection of intellectual property. This took a while to take off, but has proven itself by now. Our stakes and thus our goal aims higher, however: we want to protect the reliability and trustworthiness of content like documents, audio, pictures, video itself. And imagine: what if documents or information that is tampered with become immediately void and useless for financial or official purposes?

Align with laws

Of course, everything that needs to be done, needs to be done in line with relevant laws and regulation. To this end, the EU AI Act mandates that AI systems that are used to create deep fakes must clearly disclose that content is artificially generated or manipulated. Similarly, the American Content Origin Protection and Integrity from Edited and Deepfake Media Act mandates that any AI-generated or manipulated content must clearly disclose its artificial origin to prevent misinformation. The COPIED Act is also relevant in another way: it puts journalists, artists and musicians in control of their content and prohibits tampering with or disabling AI provenance information. And a joint approach geared at source authentication can also provide a means for the protection of original content for creators.

We know that criminals by definition do not obey the law. Therefore, we argue that not only when AI systems generate content, but in general when content is created, the provenance of content should be captured. That requires corporation between all parties involved. It is good to see that the groundwork has been laid. Let us join forces and collectively start nudging toward source authentication!

NVB Position Paper

This blog is based on the Expression of Interest from the Dutch Association of Banks as part of the Public consultation on transparency requirements for certain AI systems by the European Commission. Additionally, a position paper on source authentication has been published. 

Lees ook: