Differences

This shows you the differences between two versions of the page.

--- education [2021/09/08 10:00]
fablpd
+++ education [2022/12/20 10:30]
fablpd
@@ Line 28: / Line 28: @@
   * **[[cryptocurrencies|Cryptocurrencies]]**: We have several project openings as part of our ongoing research on designing new cryptocurrency systems. Please contact [[rachid.guerraoui@epfl.ch|Prof. Rachid Guerraoui]].
-  * **On the design and implementation of scalable and secure blockchain algorithms**: Consensus has recently gained in popularity with the advent of blockchain technologies. Unfortunately, most blockchains do not scale due, in part, to their centralized (leader-based) limitation. We recently designed a promising fully decentralised (leader-less) algorithm that promises to scale to large networks. The goal of this project is to implement it in rust and compare its performance on AWS instances against a traditional leader-based alternative like BFT-Smart whose code will be provided. Contact [[https://people.epfl.ch/vincent.gramoli|Vincent Gramoli]] for more information.
+  * **Tackling data heterogeneity in Byzantine-robust ML**: Context: Distributed ML is a very effective paradigm to learn collaboratively when all users correctly follow the protocol. However, some users may behave adversarially and measures should be taken to protect against such Byzantine behavior [1, 2]. In real-world settings, users have different datasets (i.e. non-iid), which makes defending against Byzantine behavior challenging, as was shown recently in  [3, 4]. Some defenses were proposed to tackle data heterogeneity, but their performance is suboptimal on simple learning tasks. Goal: Develop defenses with special emphasis on empirical performance and efficiency in the heterogeneous setting. Contact [[https://people.epfl.ch/youssef.allouah?lang=en|Youssef Allouah for more information.
-  * **Making Blockchain Accountable**: Abstract: One of the key drawback of blockchain is its lack of accountability. In fact, it does not hold participants responsible for their actions. This is easy to see as a malicious or Byzantine user typically double spends in a branch of blocks that disappears from the system, hence remaining undetected. Accountability is often thought to be communication costly: to detect a malicious participants who has sent deceitful messages to different honest participants for them to disagree, one may be tempted to force each honest participant to exchange all the messages they receive and cross-check them. However, we have recently designed an algorithm that shares the same communication complexity as the current consensus algorithms of existing blockchains. The goal of this project is to make blockchains accountable by implementing this accountable consensus algorithm and comparing it on a distributed set of machines against a baseline implementation. Contact [[https://people.epfl.ch/vincent.gramoli|Vincent Gramoli]] for more information.
-  * **GAR performances on different datasets**: Robust machine learning on textual data and content recommendation is critical for the safety of social media users (harassment, hate speech, etc.), but also for the reliability of scientific use of natural language processing such for processing computer programs, chemistry and drug discovery. Text datasets are known to have long-tailed distributions, which poses specific challenges for robustness, while content recommendation datasets may feature clusters of similar users. The goal of this project is to better understand the properties of different datasets, and what makes a gradient aggregation rule (e.g. Krum, trimmed mean...) better than another, given a specific text dataset (conversational chatbots, translation, github code etc.). Contact [[https://people.epfl.ch/le.hoang|Lê Nguyên Hoang]]  for more information.
+  * **Proof systems for Byzantine systems**: Cryptographic proof systems enable the rapid verification of computation between mutually distrustful parties. Recent advances in proof systems include (1) recursive proofs, transition proofs and accumulators which are of prime interest to shrink long chains of computation and/or their associated storage, and (2) zero-knowledge scalable proofs useful for privacy-preserving systems. Motivated by cryptocurrencies, the goal of this project is to devise and implement Byzantine-resilient systems that incorporate new cryptographic proof systems for efficiency and/or privacy. Contact Pierre-Louis Roman <pierre-louis.roman@epfl.ch> for more information.
+  * **Hybrid ordering for cryptocurrencies**: Most cryptocurrencies nowadays rely on total order broadcast to maintain a blockchain that represents an agreed-upon log of events. Total order broadcast may be required for some applications, such as smart contracts, but the simpler and easy to parallelize reliable broadcast suffices for payments. The goal of this project is to devise and implement Byzantine-resilient broadcast algorithms with hybrid ordering guarantees that only order events when required. Contact Pierre-Louis Roman <pierre-louis.roman@epfl.ch> for more information.
+  * **Topology-aware mempool for cryptocurrencies**: The mempool is a core component of cryptocurrency systems. It disseminates user transactions to the miner nodes before they reach consensus.Current mempools assume an homogeneous network topology where all machines have the same bandwidth and latency.This unrealitic assumption forces the system to progress at the same speed as the slowest node in the system. This project aims at implementing a mempool which exploits the heterogeneity of the network to speed up data dissemination for cryptocurrency systems. This is a practical project which requires good knowledge in network programming, either Go or C++, distributed algorithms. Contact Gauthier Voron <gauthier.voron@epfl.ch> for more information.
+  * **Robust mean estimation**: In recent years, many algorithms have been proposed to perform robust mean estimation, which has been shown to be equivalent to robust gradient-based machine learning. A new concept has been proposed to define the performance of a robust mean estimator, called the [[https://arxiv.org/abs/2008.00742|averaging constant]] (along with the Byzantine resilience). This research project consists of computing the theoretical averaging constant of different proposed robust mean estimators, and to study their empirical performances on randomly generated vectors. Contact [[https://people.epfl.ch/sadegh.farhadkhani?lang=en|Sadegh Farhadkhani]] for more information.
+  * **Accelerate Byzantine collaborative learning**: [[https://arxiv.org/abs/2008.00742|Our recent NeurIPS paper]] proposed algorithms for collaborative machine learning in the presence of Byzantine nodes, which have been proved to be near optimal with respect to optimality at convergence. However, these algorithms require all-to-all communication at every round, which is suboptimal. This research consists of designing a practical solution to Byzantine collaborative learning, based on the idea of a random communication network at each round, with both theoretical guarantees and practical implementation. Contact [[https://people.epfl.ch/sadegh.farhadkhani?lang=en|Sadegh Farhadkhani]] for more information.
-  * **Strategyproof collaborative filtering**: In collaborative filtering, other users' inputs are used to generalize the preferences of a given user. Such an approach has been critical to improve performance. However, it exposes each user to being manipulated by the inputs of malicious users, which is arguably currently occurring on social medias. In this theoretical project, we search for Byzantine-resilient and strategyproof learning algorithms to perform something akin to collaborative filtering. This would also have important applications for implicit voting systems on exponential-size decision sets. Contact [[https://people.epfl.ch/le.hoang|Lê Nguyên Hoang]] for more information.
   * **Probabilistic Byzantine Resilience**:  Development of high-performance, Byzantine-resilient distributed systems with provable probabilistic guarantees. Two options are currently available, both building on previous work on probabilistic Byzantine broadcast: (i) a theoretical project, focused the correctness of probabilistic Byzantine-tolerant distributed algorithms; (ii) a practical project, focused on numerically evaluating of our theoretical results. Please contact [[matteo.monti@epfl.ch|Matteo Monti]] to get more information.
-  * **Distributed coordination using RDMA.** RDMA (Remote Direct Memory Access) allows accessing a remote machine's memory without interrupting its CPU. This technology is gaining traction over the last couple of years, as it allows for the creation of real-time distributed systems. RDMA allows for communication to take place close to the μsec scale, which enables the design and implementation of systems that process requests in only tens of μsec. Current research focuses on achieving real-time failure detection through a combination of novel algorithm design, latest hardware and linux kernel customization. Fast failure detection over RDMA brings the notion of availability to a new level, essentially allowing modern systems to enter the era of 7 nines of availability. Contact [[https://people.epfl.ch/athanasios.xygkis|Athanasios Xygkis]] and [[https://people.epfl.ch/antoine.murat|Antoine Murat]] for more information.
+  * **Microsecond-scale dependable systems.** Modern networking technologies such as RDMA (Remote Direct Memory Access) allow for sub-microsecond communication latency. Combined with emerging data center architectures, such as disaggregated resources pools, they open the door to novel blazing-fast and resource-efficient systems. Our research focuses on designing such microsecond-scale systems that can also tolerate faults. Our vision is that tolerating network asynchrony as well as faults (crash and/or Byzantine) is a must, but that it shouldn't affect the overall performance of a system. We achieve this goal by devising and implementing novel algorithms tailored for new hardware and revisiting theoretical models to better reflect modern data centers. Previous work encompasses microsecond-scale (BFT) State Machine Replication, Group Membership Services and Key-Value Stores (OSDI'20, ATC'22 and ASPLOS'23). Overall, if you are interested in making data centers faster and safer, contact [[https://people.epfl.ch/athanasios.xygkis|Athanasios Xygkis]] and [[https://people.epfl.ch/antoine.murat|Antoine Murat]] for more information.
@@ Line 46: / Line 55: @@
 \\
-  * **Byzantine-resilient heterogeneous GANs**: Byzantine-resilient federated learning has emerged as a major theme over the last couple of years, in grand part due to the need to distribute machine learning across many nodes, due to performance and privacy concerns. Until now it has focused on training a single model across many workers and many parameter serves. While this approach has brought on formidable results - including in GAN training, the topic of efficient, distributed and byzantine-resilient training of heterogeneous architectures remain relatively unexplored. In the context of Generative adversarial networks (GANs), such learning is critical to training light discriminators that can specialize in detecting specific featuers of generator-generated images. The goal of this project will be to investigate the potential for GAN training process poisonning by malicious discriminators and generators and investigate efficient protocols to ensure the training process robustness. You will need to have experience with scientific computing in Python, ideally with PyTorch experience, and notions of distributed computing. Contact [[https://people.epfl.ch/andrei.kucharavy|Andrei Kucharavy]] for more information.
-\\
-  * **GANs with Transformers**: Since their introduction in 2017, the Transformer architecture revolutionized the NLP machine learning models. Thanks to the
-scalability of self-attention only architectures, the models can now
-scale into trillions of parameters, allowing human-like capacities of
-text generation. However, they are not without their own shortcomings,
-notably due to their max-likelihood training mode over data that
-contains potentially undesirable statistical associations.
-An alternative approach to generative learning - Generative Adversarial
-Networks (GANs) - perform remarkably well when it comes to images, but
-have until recently struggled with texts, due to their sequential and
-discrete nature that is not compatible with gradient back-propagation
-they need to train. Some of those issues have been solved, but a major
-one - their scalability due to usage of RNNs instead of pure
-self-attention architectures.
-Previously, we were able to show that it is impossible to trivially
-replace RNN layers with Transformer layers
-(https://arxiv.org/abs/2108.12275, presented in RANLP2021). This project
-will be building on those results and attempting to create stable
-Transformer-based Text GANs based on the tricks known to stabilize
-Transformer training or to attempt to theoretically demonstrate the
-inherent instability of Transformer-derived architectures in adversarial
-regime.
-You will need a solid background knowledge of linear algebra,
-acquaintance with the theory of machine learning, specifically neural
-networks, as well as experience with scientific computing in Python,
-ideally with PyTorch experience. Experience with NLP desirable, but not
-required.