Embarrassingly parallel
In parallel computing, an embarrassingly parallel workload or problem is one where little or no effort is required to separate the problem into a number of parallel tasks. This is often the case where there exists no dependency (or communication) between those parallel tasks.[1]
Embarrassingly parallel problems (also called "perfectly parallel" or "pleasingly parallel") tend to require little or no communication of results between tasks, and are thus different from distributed computing problems that require communication between tasks, especially communication of intermediate results. They are easy to perform on server farms which do not have any of the special infrastructure used in a true supercomputer cluster. They are thus well suited to large, Internet-based distributed platforms such as BOINC, and do not suffer from parallel slowdown. The diametric opposite of embarrassingly parallel problems are inherently serial problems, which cannot be parallelized at all.
A common example of an embarrassingly parallel problem lies within graphics processing units (GPUs) for the task of 3D projection, where each pixel on the screen may be rendered independently.
Contents
Etymology of the term
The genesis of the phrase "embarrassingly parallel" is not known;[citation needed] it is a comment on the ease of parallelizing such applications, and that it would be embarrassing for the programmer or compiler to not take advantage of such an obvious opportunity to improve performance. "Because so many important problems remain unsolved mainly due to their intrinsic computational complexity, it would be embarrassing not to develop parallel implementations of polynomial homotopy continuation methods."[2] Contrastingly, the term may refer to parallelizing which is, "embarrassingly easy".[3] It is first found in the literature in a 1986 book on multiprocessors by MATLAB's co-founder Cleve Moler.[4] Moler claims to have invented this term.[5]
An alternative term, "pleasingly parallel," has gained some use, perhaps to avoid the negative connotations of embarrassment in favor of a positive reflection on the parallelizability of the problems. "Of course, there is nothing embarrassing about these programs at all."[6]
Examples
Some examples of embarrassingly parallel problems include:
- Distributed relational database queries using distributed set processing
- Serving static files on a webserver to multiple users at once.
- The Mandelbrot set, Perlin noise and similar images, where each point is calculated independently.
- Rendering of computer graphics. In computer animation, each frame may be rendered independently (see parallel rendering).[dubious ]
- Brute-force searches in cryptography. Notable real-world examples include distributed.net and proof-of-work systems used in cryptocurrencies.
- BLAST searches in bioinformatics for multiple queries (but not for individual large queries) [7]
- Large scale face recognition that involves comparing thousands of arbitrary acquired faces (e.g. a security or surveillance video via closed-circuit television) with similarly large number of previously stored faces (e.g., a "rogues gallery" or similar watch list).[8]
- Computer simulations comparing many independent scenarios, such as climate models.
- Genetic algorithms and other evolutionary computation metaheuristics.
- Ensemble calculations of numerical weather prediction.
- Event simulation and reconstruction in particle physics.
- The Marching squares algorithm
- Sieving step of the quadratic sieve and the number field sieve.
- Tree growth step of the random forest machine learning technique.
- Discrete Fourier Transform where each harmonic is independently calculated.
Implementations
- In R (programming language) – The snow (Simple Network of Workstations) package implements a simple mechanism for using a collection of workstations or a Beowulf cluster for embarrassingly parallel computations.
See also
- Amdahl's law defines value P which would be almost or exactly equal to 1 for an embarrassingly parallel problem.
- Map (parallel pattern)
References
- ↑ Section 1.4.4 of: Lua error in package.lua at line 80: module 'strict' not found.
- ↑ Lua error in package.lua at line 80: module 'strict' not found.
- ↑ Matloff, Norman (2011). The Art of R Programming: A Tour of Statistical Software Design, p.347. No Starch. ISBN 9781593274108.
- ↑ Lua error in package.lua at line 80: module 'strict' not found.
- ↑ The Intel hypercube part 2 reposted on Cleve's Corner blog on The MathWorks website
- ↑ Kepner, Jeremy (2009). Parallel MATLAB for Multicore and Multinode Computers, p.12. SIAM. ISBN 9780898716733.
- ↑ SeqAnswers forum
- ↑ How we made our face recognizer 25 times faster (developer blog post)
External links
- Embarrassingly Parallel Computations, Engineering a Beowulf-style Compute Cluster
- "Star-P: High Productivity Parallel Computing"