Differences
This shows you the differences between two versions of the page.
Both sides previous revision Previous revision Next revision | Previous revision Next revision Both sides next revision | ||
education [2016/03/25 10:27] fablpd |
education [2019/05/27 12:09] fablpd |
||
---|---|---|---|
Line 3: | Line 3: | ||
====== Education ====== | ====== Education ====== | ||
+ | \\ | ||
The lab is teaching the following courses: | The lab is teaching the following courses: | ||
\\ | \\ | ||
+ | |||
+ | * [[education/ca_2018|Concurrent Algorithms]] (theory & practice) | ||
+ | * [[education/da|Distributed Algorithms]] (theory & practice) | ||
\\ | \\ | ||
- | * [[education/ca_2015|Concurrent Algorithms]] | + | The lab taught in the past the following courses: |
- | * [[education/da|Distributed Algorithms]] | + | |
* <html><a href="http://moodle.epfl.ch/course/view.php?id=14044">Information, Calcul et Communication</a></html> | * <html><a href="http://moodle.epfl.ch/course/view.php?id=14044">Information, Calcul et Communication</a></html> | ||
* <html><a href="http://cowww.epfl.ch/proginfo/wwwhiver/">Introduction à la Programmation Orientée Objet</a></html> | * <html><a href="http://cowww.epfl.ch/proginfo/wwwhiver/">Introduction à la Programmation Orientée Objet</a></html> | ||
Line 20: | Line 24: | ||
===== Master Projects ===== | ===== Master Projects ===== | ||
- | LPD offers master projects in the following areas: | + | DCL offers master projects in the following areas: |
- | " **Dynamically Distributed Spatial Indexing"": a project here would consist in studying existing spatial index data structures and algorithms, e.g., simple grids, Quadtrees, R-Trees etc., and how they may be dynamically distributed for indexing a large number of moving objects; please contact [[mailto:benoit.garbinato@unil.ch|Benoit Garbinato]] to get more information. | + | * **Probabilistic Byzantine Resilience**: Development of high-performance, Byzantine-resilient distributed systems with provable probabilistic guarantees. Two options are currently available, both building on previous work on probabilistic Byzantine broadcast: (i) a theoretical project, focused the correctness of probabilistic Byzantine-tolerant distributed algorithms; (ii) a practical project, focused on numerically evaluating of our theoretical results. Please contact [[matteo.monti@epfl.ch|Matteo Monti]] to get more information. |
- | * **Multicore computing**: a project here would consist for instance in designing and implementing efficient lock-based or lock-free shared objects; please contact [[http://people.epfl.ch/vasileios.trigonakis|Vasileios Trigonakis]] or [[http://people.epfl.ch/tudor.david|Tudor David]] to get more information. | ||
- | * **Dynamic distributed computing**: a project here would consist for instance in designing and implementing applications that would run in a simulation of a cloud with high churn, but possibly robust to arbitrary behavior of some of its components; please contact [[http://people.epfl.ch/matej.pavlovic|Matej Pavlovic]] to get more information. | + | * **Distributed computing using RDMA and/or NVRAM**: contact [[https://people.epfl.ch/igor.zablotchi|Igor Zablotchi]] for more information. |
+ | |||
+ | * **[[Distributed ML|Distributed Machine Learning]]**: contact [[http://people.epfl.ch/georgios.damaskinos|Georgios Damaskinos]] for more information. | ||
+ | |||
+ | * **Robust Distributed Machine Learning**: With the proliferation of big datasets and models, Machine Learning is becoming distributed. Following the standard parameter server model, the learning phase is taken by two categories of machines: parameter servers and workers. Any of these machines could behave arbitrarily (i.e., said Byzantine) affecting the model convergence in the learning phase. Our goal in this project is to build a system that is robust against Byzantine behavior of both parameter server and workers. Our first prototype, AggregaThor(https://www.sysml.cc/doc/2019/54.pdf), describes the first scalable robust Machine Learning framework. It fixed a severe vulnerability in TensorFlow and it showed how to make TensorFlow even faster, while robust. Contact [[https://people.epfl.ch/arsany.guirguis|Arsany Guirguis]] or [[https://people.epfl.ch/sebastien.rouault|Sébastien Rouault]] for more information. | ||
+ | |||
+ | * **Stochastic gradient: (artificial) reduction of the ratio variance/norm for adversarial distributed SGD**: One computationally-efficient and non-intrusive line of defense for adversarial distributed SGD (e.g. 1 parameter server distributing the gradient estimation to several, possibly adversarial workers) relies on the honest workers to send back gradient estimations with sufficiently low variance; assumption which is sometimes hard to satisfy in practice. | ||
+ | One solution could be to (drastically) increase the batch-size at the workers, but doing so may as well defeat the very purpose of distributing the computation. | ||
+ | In this project, we propose two approaches that you can choose to explore (also you may propose a different approach) to (artificially) reduce the ratio variance/norm of the stochastic gradients, while keeping the benefits of the distribution. | ||
+ | The first proposed approach, speculative, boils down to "intelligent" coordinate selection. | ||
+ | The second makes use of some kind of "momentum" at the workers. | ||
+ | [1] "Machine Learning with Adversaries: Byzantine Tolerant Gradient Descent" (https://papers.nips.cc/paper/6617-machine-learning-with-adversaries-byzantine-tolerant-gradient-descent) | ||
+ | [2] "Federated Learning: Strategies for Improving Communication Efficiency" (https://arxiv.org/abs/1610.05492) | ||
+ | |||
+ | * **Consistency in global-scale storage systems**: We offer several projects in the context of storage systems, ranging from implementation of social applications (similar to [[http://retwis.redis.io/|Retwis]], or [[https://github.com/share/sharejs|ShareJS]]) to recommender systems, static content storage services (à la [[https://www.usenix.org/legacy/event/osdi10/tech/full_papers/Beaver.pdf|Facebook's Haystack]]), or experimenting with well-known cloud serving benchmarks (such as [[https://github.com/brianfrankcooper/YCSB|YCSB]]); please contact [[http://people.epfl.ch/dragos-adrian.seredinschi|Adrian Seredinschi]] or [[https://people.epfl.ch/karolos.antoniadis|Karolos Antoniadis]] for further information. | ||
- | * **Recommender systems**: Projects in context of recommender systems can be found here: __[[List of Projects]]__. If you are interested in any project, please contact [[http://people.epfl.ch/rhicheek.patra|Rhicheek Patra]]. | ||
- | * **Distributed and Fault-tolerant algorithms**: projects here would consist in designing failure detection mechanisms suited for large-scale systems, real-time systems, and systems with unreliable communication or partial synchrony. This task also involves implementing, evaluating, and simulating the performance of the developed mechanisms to verify the achievable guarantees; please contact [[http://people.epfl.ch/david.kozhaya|David Kozhaya]] to get more information. | ||
- | * **Consistency in global-scale storage systems**: We offer several projects in the context of storage systems, ranging from implementation of social applications (similar to [[http://retwis.redis.io/|Retwis]], or [[https://github.com/share/sharejs|ShareJS]]) to recommender systems, static content storage services (à la [[https://www.usenix.org/legacy/event/osdi10/tech/full_papers/Beaver.pdf|Facebook's Haystack]]), or experimenting with well-known cloud serving benchmarks (such as [[https://github.com/brianfrankcooper/YCSB|YCSB]]); please contact [[http://people.epfl.ch/dragos-adrian.seredinschi|Adrian Seredinschi]] for further information. | ||
\\ | \\ | ||
+ | |||
===== Semester Projects ===== | ===== Semester Projects ===== | ||
Line 39: | Line 54: | ||
If the subject of a Master Project interests you as a Semester Project, please contact the supervisor of the Master Project to see if it can be considered for a Semester Project. | If the subject of a Master Project interests you as a Semester Project, please contact the supervisor of the Master Project to see if it can be considered for a Semester Project. | ||
- | EPFL I&C duration, credits and workload information are available [[http://ic.epfl.ch/page-17123.html|here]]. Don't hesitate to contact the project supervisor if you want to complete your Semester Project outside the regular semester period. | + | EPFL I&C duration, credits and workload information are available [[https://www.epfl.ch/schools/ic/education/|here]]. Don't hesitate to contact the project supervisor if you want to complete your Semester Project outside the regular semester period. |