Sony Pixel Power calrec Sony

French MoD challenge : Thales performs a successful sovereign AI hack and presents enhanced security solutions for military and civil AI





The Friendly Hackers team from Thales, a world leader in data protection and cybersecurity, has won the CAID[1] challenge organised by the French Ministry of Defence2 during the fifth edition of European Cyber Week in France (November 21 - 23, 2023).

The challenge, the first of its kind to be organised by the French Ministry of Defence, was designed to evaluate the extent to which teams of hackers could exploit certain intrinsic vulnerabilities of AI models.

Thaless work on AI security and trust is aligned with the requirements of both the defence community and civilian organisations such as critical infrastructure providers, which all face the same challenges of protecting their training datasets and intellectual property, and guaranteeing that AI-generated results can be trusted for critical decision-making.

Rodolphe LAMPE, Senior Data Scientist in the Thales team, with Alice H liou, Vincent Thouvenot, Cong-Bang Huynh and Baptiste MorisseThe French Ministry of Defences AI security challenge

Participants in the CAID challenge had to perform two tasks:

In a given set of images, determine which images were used to train the AI algorithm and which were used for the test.

An AI-based image recognition application learns from large numbers of training images. By studying the inner workings of the AI model, Thaless Friendly Hackers team successfully determined some of the images that had been used to create the application, gaining valuable information about the training methods used and the quality of the model.

2. Find the images of aircrafts used by a AI algorithm that had been protected using unlearning techniques.

An unlearning technique consists of deleting the data used to train a model, such as images, in order to preserve their confidentiality. This technique can be used, for example, to protect the sovereignty of an algorithm in the event of its export, theft or loss. For example, a drone equipped with AI must be able to recognise any enemy aircraft as a potential threat. On the other hand, models of aircraft from its own army would have to be learned to be identified as friendly, then erased by a technique known as unlearning. In this way, even if the drone was stolen or lost, the sensitive aircraft data contained in the AI model could not be extracted for malicious purposes. However, the Friendly Hackers team from Thales managed to re-identify the data that was supposed to have been erased from the model, thereby overriding the unlearning process.

Exercises like this help to assess the vulnerability of training data and trained models, which are valuable tools and can deliver outstanding performance but also represent new attack vectors for the armed forces. An attack on training data or trained models could have significant consequences in a military context, where this type of information could give an adversary the upper hand. Risks include model theft, theft of the data used to recognise military hardware or other features in a theatre of operations, and backdoors to impair the operation of the system using the AI. While AI in general, and generative AI in particular, offers significant operational benefits and provides military personnel with intensively trained decision support tools to reduce their cognitive burden, the national defence community needs to address new threats to this technology as a matter of priority.

The Thales BattleBox approach to tackle AI vulnerabilities

The protection of training data and trained models is critical in the defence sector. AI cybersecurity is becoming more and more crucial, and needs to be autonomous to thwart the many new opportunities that the world of AI is opening up to malicious actors. Responding to the risks and threats involved in the use of artificial intelligence, Thales has developed a set of countermeasures called the BattleBox to provide enhanced protection against potential breaches.

BattleBox Training provides protection from training-data poisoning, preventing hackers from introducing a backdoor.

BattleBox IP digitally watermarks the AI model to guarantee authenticity and reliability.

BattleBox Evade aims to protect models from prompt injection attacks, which can manipulate prompts to bypass the safety measures of chatbots using Large Language Models (LLMs), and to counter adversarial attacks on images, such as adding a patch to deceive the detection process in a classification model.

BattleBox Privacy provides a framework for training machine learning algorithms, using advanced cryptography and secure secret-sharing protocols to guarantee high levels of confidentiality.

To prevent AI hacking in the case of CAID challenge tasks, countermeasures such as encryption of the AI model could be one of the solutions to be implemented.

AI provides considerable operational benefits, but it requires high levels of security and cybersecurity protection to prevent data breaches and misuse. Thales implements a large range of AI-based solutions for all types of civil and military use cases. Intended to be explainable, embeddable and integrated within robust critical systems, they are also designed to be sovereign, frugal and reliable thanks to advanced methods and tools used for qualification and validation. Thales has the dual AI and line-of-business expertise needed to incorporate these solutions into its systems to significantly improve their operational capabilities, said David Sadek, Thales VP Research, Technology & Innovation in charge of Artificial Intelligence.

Thales and AI

As the Groups defence and security businesses address critical requirements, often with safety-of-life implications, Thales has developed an ethical and scientific framework for the development of trusted AI based on the four strategic pillars of validity, security, explainability and resp
See more stories from thales

More from Thales


Thales and Thaicom establish the foundation to lead the Unmanned Traffic Management deployment in Thailand

Facebook Twitter LinkedIn Thales and Thaicom, through Thaicom's subsid...


From post-quantum cryptography to seamless connectivity and 5G security, find out how Thales enables a secure connected world

Facebook Twitter LinkedIn Thales actively contributes to making the world ...


Saab and Thales sign ATM MoU for Singapore

Facebook Twitter LinkedIn Saab and Thales announced today at Singapore Airshow 2024 the signature of a Memorandum of Understanding (MoU) to support the lo...


Merah Putih-2 telecommunications satellite successfully launched

Facebook Twitter LinkedIn Set to provide more capacity over the Indonesian archipelago Cape Canaveral, Florida, February 21, 2024 - Merah Putih-2 telecom...


Malaysia Airlines Partners with Thales to Optimize Flight Management and Surveillance Solutions

Facebook Twitter LinkedIn 20 Airbus A330neo aircraft in Malaysia Airlines&...


Thales Alenia Space launches MARSBalloon to offer students the chance to fly experiments and test technologies that could one day be destined for Mars

Facebook Twitter LinkedIn Bristol, February 20th, 2024 - Thales Alenia Space, a joint-venture between Thales (67%) and Leonardo (33%), harness students...


Thales Strengthens Avionics and Digital Capabilities in Singapore

Facebook Twitter LinkedIn Thales reinforces its Avionics industrial base in Singapore, through a Centre of Excellence that will work on industrialising ne...


Air India selects Thales' AVANT Up In-Flight Entertainment to Elevate Experience for its Passengers

Facebook Twitter LinkedIn Air India will equip 51 of its aircraft with Tha...


Thales supplies SurfSAT-L satellite communication solution for F126 German frigates

Facebook Twitter LinkedIn The German Navy will be equipped with SurfSAT-L ...


Thales to equip Swiss rail operator BLS with a lower power consumption electronic interlockings

Facebook Twitter LinkedIn Thales has been awarded a frame contract to equi...


Thales Alenia Space will provide communication equipment to NASA's NEO Surveyor mission

Facebook Twitter LinkedIn The equipment produced at Thales Alenia Space si...


123 GRIFFON and 22 JAGUAR: 2023 DGA scheduled delivery successfully completed by Nexter, a KNDS company, Arquus and Thales

Facebook Twitter LinkedIn The temporary company grouping (Groupement Momen...


Thales wins 1.8 Billion contract to maximise the days-at-sea for the Royal Navy'

Facebook Twitter LinkedIn The UK Ministry of Defence has today announced a...


Cubris to provide new energy saving system for London's Elizabeth line

Facebook Twitter LinkedIn Alstom has chosen to implement Cubris' driver advisory system on Transport for London's Elizabeth line. TheGreenSpee...


Korea's KASS satellite navigation system certified by national authorities and enters operational service

Facebook Twitter LinkedIn The KASS regional system was developed by Thales...


Thales air data solution to enable the smooth and safe flight of Eve Air Mobility's eVTOL aircraft

Facebook Twitter LinkedIn The strategic partnership between Thales and Eve...


Thales updates its financial agenda

Facebook Twitter LinkedIn Thales (Euronext Paris: HO) updates its financial agenda for 2024. Date Event Release 5 March 2024 Full Year 2023 results ...


IIT Madras and Thales announce fourth edition of Carbon Zero Challenge (CZC 4.0), invite applications across India

Facebook Twitter LinkedIn As part of its CSR efforts in India, Thales has ...


Thales and Caju take cards produced with sustainable raw materials to more than 30 000 companies in Brazil

Facebook Twitter LinkedIn Caju is now issuing recycled PVC cards from Th...


Thales and Quantinuum Launch Starter Kit to help Enterprises prepare for Post-Quantum Cryptography

Facebook Twitter LinkedIn The new solution - the PQC Starter Kit - offer...


EU launches Nostradamus and prepares Europe for a quantum world

Facebook Twitter LinkedIn Deutsche Telekom-led consortium responsible for building EUs quantum communications testing infrastructure. Partners are Thales...


La sonde de dtection rseau des cyberattaques de Thales renouvelle sa qualification par l'ANSSI pour sa version 2.0.X

Facebook Twitter LinkedIn Le Groupe propose ses clients, tats et op rat...


Thales appoints Christoph Ruffner as CEO for Thales Germany

Facebook Twitter LinkedIn Thales (Euronext Paris: HO), a global leader in the Defence & Security, Aeronautics & Space and Digital Identity & Security sect...


Thales Ground Master 200 Multi-Mission Compact radar to strengthen Lithuania's counter battery operations

Facebook Twitter LinkedIn The Lithuanian Defence Material Agency (DMA) sig...


Thales will digitalise depot signalling equipment for Singapore's oldest MRT lines (North-South and East-West)

Facebook Twitter LinkedIn Thales signed a new contract with Land Transport...


SK Telecom and Thales collaborate on post-quantum cryptography to enhance users' protection on 5G network

Facebook Twitter LinkedIn Leading Korean mobile operator SK Telecom and Th...


Thales Alenia Space has selected the UK's National Satellite Test Facility for FLEX satellite first test campaign

Facebook Twitter LinkedIn December 15, 2023 - Thales Alenia Space, a join...


Thales delivers the new integrated surveillance system to Melilla's Command Headquarters

Facebook Twitter LinkedIn The joint venture formed by Thales and Trab...


Trust Bank and Thales Launch Singapore's First Ocean Plastic Card

Facebook Twitter LinkedIn Trust Bank Singapore Limited (known as Trust'), is offering its customers sustainable credit and debit cards made from recy...


Thales' new Ground Master 400 reinforcing air surveillance and protection for the Royal Malaysian Air Force

Facebook Twitter LinkedIn In the framework of the defence cooperation betw...


Thales Alenia Space signs a multi-satellite contract with PT Len Industri to provide radar and optical imagery

Facebook Twitter LinkedIn A state-of-the-art constellation of innovative a...


Happy 40th Birthday to Thales Alenia Space Rome and L'Aquila sites

Facebook Twitter LinkedIn 40 years of passion, innovation, and technology at the heart of the global space ecosystem Rome, December 4, 2023 - Thales Alen...


Thales completes the acquisition of Imperva, creating a global leader in cybersecurity

Facebook Twitter LinkedIn With the integration of Imperva starting in Janu...


SFEIR and S3NS sign a formal partnership agreement to help organisations transition to the trusted cloud

Facebook Twitter LinkedIn Neuilly-sur-Seine, 22 November 2023 - As part of...


Physical hacking: Thales warns of new challenge to AI systems

Facebook Twitter LinkedIn At the 30th edition of the ACM Conference on Computer and Communications Security, which took place in Copenhagen from 27-30 Nov...


Thales Expands Footprint in India with a Second Office in Bengaluru

Facebook Twitter LinkedIn The new state-of-the-art facility in Bengaluru reaffirms India's position for Thales as a strategic hub for innovation and g...


Thales SEC Transport awarded a contract to deliver signaling system for Nanchang Metro Line 1 and Line 2 extensions

Facebook Twitter LinkedIn Thales SEC Transport was awarded a contract to d...


Thales - Elettronica joint company SIGEN aboard the Horizon frigates for the modernization of the Electronic Warfare system

Facebook Twitter LinkedIn Following the signing of the contract between OC...


Thales to provide Alexandria metro with modern solutions in Egypt

Facebook Twitter LinkedIn Thales signs a new contract with the Colas Rail / Orascom joint venture to supply the Egypt's National Authority for Tunnels...


Centaurium UAS and Thales join forces to open Swiss skies to long-range drone operations

Facebook Twitter LinkedIn Centaurium UAS and Thales have signed a cooperat...


Thales Alenia Space to provide Communication Transponder for Turkey's first lunar mission

Facebook Twitter LinkedIn Madrid, November 21, 2023 - Thales Alenia Space ...


GCAA ,Tawazun Council and Thales sign MoU to establish an ATM Innovation Lab in the UAE

Facebook Twitter LinkedIn MOU provides for establishing an Air Traffic Man...


Thales and StandardAero unlock a new level of safety with StableLight, the new 4-axis autopilot system

Facebook Twitter LinkedIn Thales and StandardAero are thrilled to announce...


Thales unveils Nexium SafeCore, an ANSSI-certified secure infrastructure solution for network function virtualisation

Facebook Twitter LinkedIn At Milipol on 16 November, Thales, a global tech...


Moro Hub and Thales Join Forces to Spearhead Innovation in Physical and Cyber Security Services

Facebook Twitter LinkedIn The agreement aims to ensure uncompromised data ...