7.2.3. Data Parallelism vs. Task Parallelism

Try these first

1. 1.

   What is data parallelism?

   Hint

   Think about how tasks can be split between multiple processors.

2. 2.

   What is task parallelism?

   Hint

   Consider processors handling distinct functions in an AI system.

3. 3.

   What defines data parallelism?

   • Different tasks are executed concurrently.
   • The same operation is performed on different data subsets.
   • Only one processor is used for all tasks.

   Hint

   Focus on the operation and the data.

4. 4.

   Is task parallelism useful in AI applications?

   • True
   • False

   Hint

   Think about the need for multiple functions in AI.

5. 5.

   Design an AI system that utilizes both data and task parallelism. Describe its architecture and how the two types of parallelism would enhance performance.

   Hint

   Think of how each part can contribute to maximizing computational resources.

6. 6.

   Consider the limitations discussed in class regarding synchronization overhead in task-based systems. How might you mitigate these issues in a proposed system architecture?

   Hint

   Focus on reducing the need for constant communication between processors.