SafeAI@EPFL
Artificial Intelligence is the ability of an algorithm to solve by a machine, a problem that was thought to be only solvable by a human. When Blaise Pascal invented the Pascaline to perform additions, his machine was viewed as an AI. Several centuries later, when the Deep Blue computer was able to beat Kasparov in Chess, it was called AI. For a little while. In the last twenty years, computers could beat the best champions in Jeopardy, Go, and Poker, as well as draw Rembrandt like pictures and produce Beethoven like symphonies. The power of those AI algorithms has lied on their deployment on large networks of machines and their ability to leverage massive amount of data to improve their strategies, namely to learn.
Whilst keys to their power, the wide network distribution together with the large amount of data managed are also sources of the fragility of these AI algorithms. Essentially, the more machines and data are involved, the higher the chances of bugs, faulty behavior and potential attacks. These issues can lead the algorithms to take bad decisions or leak information. This can be anecdotal in the context of games but today AI is entering areas that are critical to our lives such as medicine and transportation.
The goal of our work has been in the recent years to devise AI algorithms that do guarantee some levels of privacy and robustness as well as inform the AI users of the very limitations of these levels. Privacy means here the ability to hide crucial data from curious users. Robustness means the ability to tolerate the misbehavior of a fraction of the underlying network. Typically, we assume that less than 1/3 (or sometimes less than 1/2) of the machines of the networks are controlled by an adversary which sole goal is to make the AI algorithm take decisions that would harm the user.
We have published a series of papers on the topic:
Publications & Preprints
Rachid Guerraoui, Nirupam Gupta, Rafael Pinot.
Byzantine Machine Learning: A Primer. ACM Computing Surveys 2023
Sadegh Farhadkhani, Rachid Guerraoui, Nirupam Gupta, Le Nguyen Hoang, Rafael Pinot, John Stephan.
Robust Collaborative Learning with Linear Gradient Overhead. ICML 2023
Youssef Allouah, Rachid Guerraoui, Nirupam Gupta, Rafael Pinot, John Stephan.
On the Privacy-Robustness-Utility Trilemma in Distributed Learning. ICML 2023
El-Mahdi El-Mhamdi, Sadegh Farhadkhani, Rachid Guerraoui, Le Nguyen Hoang.
On the Strategyproofness of the Geometric Median. AISTATS 2023
Youssef Allouah, Sadegh Farhadkhani, Rachid Guerraoui, Nirupam Gupta, Rafaël Pinot, John Stephan.
Fixing by Mixing: A Recipe for Optimal Byzantine ML under Heterogeneity. AISTATS 2023
Sadegh Farhadkhani, Rachid Guerraoui, Nirupam Gupta, Rafaël Pinot, John Stephan.
Byzantine Machine Learning Made Easy By Resilient Averaging of Momentums. ICML 2022
Sadegh Farhadkhani, Rachid Guerraoui, Le Nguyen Hoang, Oscar Villemaud.
An Equivalence Between Data Poisoning and Byzantine Gradient Attacks. ICML 2022
El-Mahdi El-Mhamdi, Rachid Guerraoui, Lê Nguyen Hoang and Alexandre Maurer.
Removing Algorithmic Discrimination (With Minimal Individual Error). Theoretical Computer Science 2022
El-Mahdi El-Mhamdi, Rachid Guerraoui, Sébastien Rouault.
Distributed Momentum for Byzantine-resilient Stochastic Gradient Descent. ICLR 2021
Rachid Guerraoui, Nirupam Gupta, Rafael Pinot, Sebastien Rouault, John Stephan.
Differential Privacy and Byzantine Resilience in SGD: Do They Add Up?. PODC 2021
El Mhamdi El Mahdi, Farhadkhani Sadegh, Guerraoui Rachid, Guirguis Arsany, Hoang Le Nguyen, Rouault Sébastien.
Collaborative Learning in the Jungle (Decentralized, Byzantine, Heterogeneous, Asynchronous and Nonconvex Learning). NeurIPS 2021
Georgios Damaskinos, Celestine Mendler-Dünner, Rachid Guerraoui, Nikolaos Papandreou, Thomas Parnell.
Differentially Private Stochastic Coordinate Descent. AAAI 2021
Georgios Damaskinos, Rachid Guerraoui, Anne-Marie Kermarrec, Vlad Nitu, Rhicheek Patra, Francois Taiani.
FLeet: Online Federated Learning via Staleness Awareness and Performance Prediction. Middleware 2020
Rachid Guerraoui, Arsany Guirguis, Anne-Marie Kermarrec, Erwan Le Merrer.
FeGAN: Scaling Distributed GANs. Middleware 2020
El-Mahdi El-Mhamdi, Rachid Guerraoui, Arsany Guirguis, Lê Nguyen Huang, Sébastien Rouault.
Genuinely Distributed Byzantine Machine Learning. PODC 2020
El-Mahdi El-Mhamdi, Rachid Guerraoui, Sébastien Rouault.
Fast and Robust Distributed Learning in High Dimension. SRDS 2020
Karim Boubouh, Amine Boussetta, Yahya Benkaouz, Rachid Guerraoui.
Robust P2P Personalized Learning. SRDS 2020
Rachid Guerraoui, Alex Kogan, Virendra J. Marathe, Igor Zablotchi.
Efficient Multi-Word Compare and Swap. DISC 2020
Bellet Aurélien, Guerraoui Rachid, Hendrikx Hadrien.
Who Started This Rumor? Quantifying the Natural Differential Privacy of Gossip Protocols. DISC 2020
Rachid Guerraoui, Anne-Marie Kermarrec, Olivier Ruas, François Taïani:
Smaller, Faster & Lighter KNN Graph Constructions. WWW 2020: 1060-1070
Rachid Guerraoui, Anne-Marie Kermarrec, Olivier Ruas, François Taïani:
Fingerprinting Big Data: The Case of KNN Graph Construction. ICDE 2019: 1738-1741
El Mahdi El Mhamdi, Georgios Damaskinos, Rachid Guerraoui Arsany Guirguis; Sebastien Rouault:
AggregaThor: Byzantine Machine Learning via Robust Gradient Aggregation SysML 2019
Aurélien Bellet, Rachid Guerraoui, Mahsa Taziki, Marc Tommasi:
Personalized and Private Peer-to-Peer Machine Learning. AISTATS 2018: 473-481
Antoine Boutet, Florestan De Moor, Davide Frey, Rachid Guerraoui, Anne-Marie Kermarrec, Antoine Rault:
Collaborative Filtering Under a Sybil Attack: Similarity Metrics do Matter! DSN 2018: 466-477
Georgios Damaskinos, El Mahdi El Mhamdi, Rachid Guerraoui, Rhicheek Patra, Mahsa Taziki:
Asynchronous Byzantine Machine Learning (the case of SGD). ICML 2018: 1153-1162
El Mahdi El Mhamdi, Rachid Guerraoui, Sébastien Rouault:
The Hidden Vulnerability of Distributed Learning in Byzantium. ICML 2018: 3518-3527
Rachid Guerraoui, Anne-Marie Kermarrec, Tao Lin, Rhicheek Patra:
Heterogeneous Recommendations: What You Might Like To Read After Watching Interstellar. Proc. VLDB Endow. 10(10): 1070-1081(2017)
El Mahdi El Mhamdi, Rachid Guerraoui:
When Neurons Fail. IPDPS 2017: 1028-1037
Peva Blanchard, El Mahdi El Mhamdi, Rachid Guerraoui, Julien Stainer:
Machine Learning with Adversaries: Byzantine Tolerant Gradient Descent. NIPS 2017: 119-129
El Mahdi El Mhamdi, Rachid Guerraoui, Hadrien Hendrikx, Alexandre Maurer:
Dynamic Safe Interruptibility for Decentralized Multi-Agent Reinforcement Learning. NIPS 2017: 130-140
Rachid Guerraoui, Anne-Marie Kermarrec, Rhicheek Patra, Mahammad Valiyev, Jingjing Wang:
I Know Nothing about You But Here is What You Might Like. DSN 2017: 439-450
Georgios Damaskinos, Rachid Guerraoui, Rhicheek Patra:
Capturing the Moment: Lightweight Similarity Computations. ICDE 2017: 747-758
Rachid Guerraoui, Anne-Marie Kermarrec, Mahsa Taziki:
The Utility and Privacy Effects of a Click. SIGIR 2017: 665-674
El Mahdi El Mhamdi, Rachid Guerraoui, Sebastien Rouault:
On the Robustness of a Neural Network. SRDS 2017: 84-93
Antoine Boutet, Davide Frey, Rachid Guerraoui, Arnaud Jégou, Anne-Marie Kermarrec:
Privacy-preserving distributed collaborative filtering. Computing 98(8): 827-846 (2016)
Rachid Guerraoui, Anne-Marie Kermarrec, Rhicheek Patra, Mahsa Taziki:
D2P: Distance-Based Differential Privacy in Recommenders. Proc. VLDB Endow. 8(8): 862-873 (2015)