The lab is teaching the following courses:

• Concurrent Computing (CS-453) (theory & practice)
• Distributed Algorithms (CS-451) (theory & practice)

The lab taught in the past the following courses:

• Information, Calcul et Communication
• Introduction à la Programmation Orientée Objet
• Secure Distributed Computing (Fall semester 2009)

DCL offers master projects in the following areas:

• Cryptocurrencies: We have several project openings as part of our ongoing research on designing new cryptocurrency systems. Please contact Prof. Rachid Guerraoui.

• Accelerating Safe ML Systems: ML has been a hot topic for so long. Now with LLMs, it is getting even more attractive for everyone in the research community as well as industry (e.g., Google, Meta, etc.). In particular, training large models with massive data makes the need for distributed computing (i.e., distributing tasks among machines) non questionable, which leads to two main challenges. First, how to do it fast? Second, how to do it safe (e.g., secure collaborative training, robust ML, etc.)? At the heart of these two challenges is how to communicate with other machines in a fast and a secure way? This leads us to Remote Direct Memory Access (RDMA) technology which is becoming increasingly important in the field of machine learning (ML), particularly for distributed training of large models and handling massive datasets. RDMA enables high-throughput, low-latency data transfers between servers without involving the CPU, which significantly reduces the overhead associated with traditional networking methods. This is crucial for ML tasks that require rapid synchronization and communication among multiple nodes. Now the question is how to use RDMA efficiently to build fast and secure ML systems? If interested, contact Beatrice Shokry for more information.

• Tackling data heterogeneity in Byzantine-robust ML: Context: Distributed ML is a very effective paradigm to learn collaboratively when all users correctly follow the protocol. However, some users may behave adversarially and measures should be taken to protect against such Byzantine behavior [ 1, 2 ]. In real-world settings, users have different datasets (i.e. non-iid), which makes defending against Byzantine behavior challenging, as was shown recently in [ 3, 4 ]. Some defenses were proposed to tackle data heterogeneity, but their performance is suboptimal on simple learning tasks. Goal: Develop defenses with special emphasis on empirical performance and efficiency in the heterogeneous setting. Contact Youssef Allouah for more information.

• Benchmark to certify Byzantine-robustness in ML: Context: Multiple attacks have been proposed to instantiate a Byzantine adversary in distributed ML [ 1, 2 ]. While these attacks have been successful against known defenses, it remains unknown whether stronger attacks exist. As such, a strong benchmark is needed, to go beyond the cat-and-mouse game illustrating the existing research. Ideally, similar to other ML subfields such as privacy-preserving ML or adversarial examples, the desired benchmark should guarantee that no stronger attack exists. Goal: Develop a strong benchmark for attacks in Byzantine ML. Contact Youssef Allouah for more information.

• Evaluating Distributed Systems: By nature, distributed systems are hard to evaluate. Deploying real world systems and orchestrating large scale experiments require dedicated software and expensive infrastructure. As a result, many widespread distributed systems are not properly evaluated, tested on uncomparable or irreproductible setups. Projects of this category aim to build efficient and scalable evaluation tools for distributed systems. Diablo-related projects involve building a test harness for evaluating blockchains (skills required: network programming, blockchain, Go, C++). Another set of projects focus on creating large networks simulators able to emulate hundreds of powerful machines from a single physical server (skills required: system programming, virtualization, C, C++). Contact Gauthier Voron for more information.

• Smart Contracts and Decentralized Software: Smart contracts are one of the key innovations brought by blockchains, enabling users to deploy codes that get executed transparently, autonomously and in a decentralized fashion. However, the applicability of smart contracts is hampered by their limited performance. Projects of this category aim to build runtime environments for fast and efficient execution of smart contracts. The first set of projects address the challenge of deterministic parallelism, or how to use several threads to execute a smart contract while guaranteeing a deterministic result (skills required: compiler principles, Rust). The second set of projects explores the concept of non-transactional smart contracts, a way to remove the notion of gas in smart contracts (skills required: system programming, C, Rust). The last set of projects focus on high-throughput cryptographic primitives: how to use hardware acceleration to speed up transaction authentication (skills required: cryptography principles, GPU programming, C, Assembly). Contact Gauthier Voron for more information.

• Safe and Scalable Consensus: Decentralized systems like cryptocurrencies rely on the concept of consensus. This component is critical as it dictates how performant, safe and scalable a distributed system is. Over the last years, the DCL has pushed the performance of consensus algorithms to unprecedented levels but the practical safety and scalability are yet to be addressed. Projects of this category focus on designing and implementing distributed consensus algorithms which are safer against cyberattacks or adverse environments and work with higher number of participants. On one side, some projects explore new consensus designs with good theoretical guarantees and practical behaviors (skills required: distributed algorithms, network programming, Go). On the other side, some projects focus on ensuring the correctness of existing consensus algorithms through model checking at various levels (skills required: distributed algorithms, Rust, TLA+). Contact Gauthier Voron for more information.

• Robust mean estimation: In recent years, many algorithms have been proposed to perform robust mean estimation, which has been shown to be equivalent to robust gradient-based machine learning. A new concept has been proposed to define the performance of a robust mean estimator, called the averaging constant (along with the Byzantine resilience). This research project consists of computing the theoretical averaging constant of different proposed robust mean estimators, and to study their empirical performances on randomly generated vectors. Contact Sadegh Farhadkhani for more information.

• Accelerate Byzantine collaborative learning: Our recent NeurIPS paper proposed algorithms for collaborative machine learning in the presence of Byzantine nodes, which have been proved to be near optimal with respect to optimality at convergence. However, these algorithms require all-to-all communication at every round, which is suboptimal. This research consists of designing a practical solution to Byzantine collaborative learning, based on the idea of a random communication network at each round, with both theoretical guarantees and practical implementation. Contact Sadegh Farhadkhani for more information.

• Probabilistic Byzantine Resilience: Development of high-performance, Byzantine-resilient distributed systems with provable probabilistic guarantees. Two options are currently available, both building on previous work on probabilistic Byzantine broadcast: (i) a theoretical project, focused the correctness of probabilistic Byzantine-tolerant distributed algorithms; (ii) a practical project, focused on numerically evaluating of our theoretical results. Please contact Matteo Monti to get more information.

• Microsecond-scale dependable systems. Modern networking technologies such as RDMA (Remote Direct Memory Access) allow for sub-microsecond communication latency. Combined with emerging data center architectures, such as disaggregated resources pools, they open the door to novel blazing-fast and resource-efficient systems. Our research focuses on designing such microsecond-scale systems that can also tolerate faults. Our vision is that tolerating network asynchrony as well as faults (crash and/or Byzantine) is a must, but that it shouldn't affect the overall performance of a system. We achieve this goal by devising and implementing novel algorithms tailored for new hardware and revisiting theoretical models to better reflect modern data centers. Previous work encompasses microsecond-scale (BFT) State Machine Replication, Group Membership Services and Key-Value Stores (OSDI'20, ATC'22 and ASPLOS'23). Overall, if you are interested in making data centers faster and safer, contact Athanasios Xygkis and Antoine Murat for more information.

If the subject of a Master Project interests you as a Semester Project, please contact the supervisor of the Master Project to see if it can be considered for a Semester Project.

EPFL I&C duration, credits and workload information are available on https://www.epfl.ch/schools/ic/education/master/semester-project-msc/.