7.5. Challenges in Achieving Parallelism for AI Applications

Try these first

1. 1.

   What does synchronization overhead mean?

   Hint

   Think about communication effects on performance.

2. 2.

   Explain Amdahl’s Law in simple terms.

   Hint

   Consider the parts that can’t be parallelized.

3. 3.

   What is synchronization overhead in parallel systems?

   • True
   • False

   Hint

   Remember how communication affects performance.

4. 4.

   Does Amdahl's Law suggest that adding more processors always increases performance?

   • Yes
   • No
   • Sometimes

   Hint

   Think about the balance between parallelism and serial tasks.

5. 5.

   Propose a design solution that minimizes synchronization overhead in a parallel processing system used for AI applications. Explain why this would be effective.

   Hint

   Consider how to restructure tasks for independence.

6. 6.

   Given a set of data processing tasks, calculate the maximum expected speedup based on Amdahl’s Law if 90% of the tasks can be parallelized. Explain your calculation.

   Hint

   Remember to identify the serial fraction and the number of processors.