Explore

Fundamentals of Parallelism on Intel Architecture

Approx. 15 hours to complete

Save Course

Go to Course

Course Summary

Learn how to write parallel programs using modern parallel architectures, such as multicore CPUs and GPUs.

Key Learning Points

Understand the fundamental concepts of parallelism and how to apply them in programming.
Learn how to write efficient and scalable parallel programs using OpenMP and CUDA.
Explore real-world applications of parallelism in fields such as image processing and scientific computing.

Learning Outcomes

Develop a solid understanding of parallelism and its applications
Write efficient and scalable parallel programs using OpenMP and CUDA
Apply parallelism to real-world problems in image processing and scientific computing

Prerequisites or good to have knowledge before taking this course

Basic knowledge of programming in C/C++ or Python
Familiarity with computer architecture and operating systems

Course Difficulty Level

Intermediate

Course Format

Online self-paced
Video lectures
Assignments and quizzes
Programming assignments

Similar Courses

Parallel programming in Java
Parallel programming in Python

Related Education Paths

Notable People in This Field

John Hennessy
David Patterson

Related Books

Description

This course will introduce you to the multiple forms of parallelism found in modern Intel architecture processors and teach you the programming frameworks for handling this parallelism in applications. You will get access to a cluster of modern manycore processors (Intel Xeon Phi architecture) for experiments with graded programming exercises.

This course can apply to various HPC and datacenter workloads and framework including artificial intelligence (AI). You will learn how to handle data parallelism with vector instructions, task parallelism in shared memory with threads, parallelism in distributed memory with message passing, and memory architecture parallelism with optimized data containers. This knowledge will help you to accelerate computational applications by orders of magnitude, all the while keeping your code portable and future-proof. Prerequisite: programming in C/C++ or Fortran in the Linux environment and Linux shell proficiency (navigation, file copying, editing files in text-based editors, compilation).

Outline

Modern Code
1.0 Introduction
1.1 Why this course?
1.2 How Computers Get Faster
1.3 Intel Architecture
1.4 Modern Code
1.5 What You Are Going To Learn
1.6 Remote Access
Modern Code
Modern code

Vectorization
2.1 Vector Operations
2.2 Vectorizing Your Code
2.3.1 Automatic Vectorization
2.3.2 Will This Vectorize?
2.4 Guided Automatic Vectorization
2.8.1 Stencil Introduction
2.8 Stencil
2.5 SIMD-Enabled Functions
2.6 Vector Dependence
2.7 Strip Mining
Numerical Integration Introduction
2.9 Integral Vectorization
2.10 Learn More
Vectorization
Code Download
Code Download
Vectorization

Multithreading with OpenMP
3.1 Cores and Threads
Demo: Forks
3.2 Creating Threads
3.3 Variable Sharing
3.4 Parallel Loops
3.5 Data Races Mutexes
3.7 Parallel Reduction
Stencil Introduction
Stencil Demonstration
Learn More
Multithreading with OpenMP
Code Download
Code Download
OpenMP

Memory Traffic
4.1 Cheap Flops
4.2 Memory Hierarchy
4.3 High Bandwidth Memory
4.4 Memory Allocation
Stencil Introduction
Demo: Stencil with numactl
Demo: Stencil with Memkind
4.5 Bypassing Caches
Stencil Demonstration-Nontemporal
Stencil Demonstration-Char
4.6 Locality in Space
4.7 Locality in Time
Integral Introduction
Integral Demonstration
Memory Traffic
Code Download
Code Download
Memory traffic

Clusters and MPI
5.1 Computing Clusters
5.2 Message Passing Interface
5.3 Programming with MPI
5.4 Compiling and Running with MPI
5.5 Peer-to-Peer Messaging
5.6 Collective Communication
Stencil Introduction
Stencil Demonstration-MPI
Integral Introduction
Integral Demonstration
Learn More
Clusters and MPI
Code Download
Code Download
MPI

Summary of User Reviews

This course on parallelism is highly recommended for anyone interested in improving their knowledge in this area. The course has received positive reviews from many users due to its comprehensive content and hands-on approach.

Key Aspect Users Liked About This Course

The hands-on approach of the course was praised by many users as it allowed them to apply the concepts they learned in real-world scenarios.

Pros from User Reviews

Comprehensive content that covers a wide range of topics related to parallelism
The course is well-structured and easy to follow, even for beginners
The instructors are knowledgeable and provide clear explanations
The course provides ample opportunities for practice and application of concepts
The course is updated regularly to reflect the latest trends and technologies in the field

Cons from User Reviews

Some users found the course to be too theoretical and would have preferred more practical examples
Some users felt that the course could have been more challenging, especially for those with prior experience in parallelism
The course requires a significant time commitment, which may not be feasible for everyone
Some users found the course to be somewhat repetitive, especially in the early modules
The course may not be suitable for those looking for a quick introduction to parallelism

Recommended for you

Java Multithreading, Concurrency & Performance Optimization

Become an expert in Multithreading, Concurrency & Parallel programming in Java, with strong emphasis on high performance Description Teaching PhilosophyThe course is designed to teach you "how to fish"....

Save Course

Computer System Design: Advanced Concepts of Modern Microprocessors

Learn about advanced computer design concepts, including how to make modern multicore-based computers both fast and energy efficient. Additionally, you will learn how design decisions affect energy efficiency and performance....

Save Course