Distributed and Parallel Computing (M2 DiPaC)

M2 DiPaC students will gain competency and skills in:

• Parallel programming models and performance engineering on CPUs/GPUs and distributed clusters/supercomputers.
• Design and analysis of distributed algorithms with theoretical guarantees.
• Scalable data analysis and AI skills: training, inference, large-scale distributed data analytics.
• Hybrid HPC+quantum simulation through exact and approximation techniques.
• Scheduling and load balancing of parallel workloads on clusters and clouds.
• Code profiling, tracing, bottleneck diagnosis, performance tuning techniques.
• Software engineering for HPC: modern C++, testing, CI, reproducibility.
• Version control with Git and rigorous documentation.
• Technical communication and project scoping in research and industry contexts.
• Responsible, reliable, and sustainable computing practices.

Computer systems are evolving toward higher efficiency and richer functionality across three major, interconnected scientific fields:

• Distributed systems deliver connectivity and dependable operation across the Internet, clouds, clusters, and sensors, addressing hard problems in synchronization, security, concurrency, and robustness.
• High-performance and parallel computing (HPC) tackles intensive workloads in science and AI by exploiting supercomputing architectures and rigorous performance engineering.
• Quantum computing provides algorithms and hardware that exploit quantum parallelism to achieve gains unreachable by classical paradigms.

Building on the foundations that M1 DiPaQ establishes in HPC, distributed systems, and quantum computing, M2 DiPaC specializes on advanced topics in HPC and distributed algorithms for large-scale systems. Students learn to design fast, scalable, and robust solutions for applications in AI, big data analytics, scientific simulations, and quantum-enabled workflows. Students have the opportunity to specialize either in high performance data analysis (HPDA) or hybrid HPC/quantum computing (HQI) through elective courses.

Knowledge objectives:

• Parallel programming models and performance engineering on modern supercomputers and accelerators.
• Large-scale distributed algorithms and systems: replication, consensus, consistency, mobile agents, and nature-inspired algorithms with robustness and performance guarantees.
• Big data analysis, machine learning, and AI algorithms with massive computational challenges (HPDA)
• Quantum algorithms and simulation using both classical/HPC or quantum workflows (HQI)

Skill objectives:

• Building high-quality parallel and distributed algorithms and software that meet latency, throughput, and performance targets on target HPC architectures.
• Developing and analyzing scalable, robust algorithms with theoretical guarantees on scalability, consensus, termination, and fault tolerance.
• Optimizing HPC code across the stack: complexity, memory locality, vectorization, accelerator use, communication, I/O, and networks.

Resources and practice:

• Access to university clusters and partner supercomputers for hands-on labs, course projects, code development, and tuning.
• Use of open-source toolchains and libraries widely adopted by the HPC community.
• Acquiring modern HPC software engineering practices with advanced C++, IDEs, version control (git), documentation, and continuous integration.