Introduction to Parallel Processing

November 25, 2020 by

Howard Poston

Introduction to Parallel Processing

In the beginning, computers had a single processing core and were single-threaded.  While these computers would be able to run multiple applications “in parallel”, this was accomplished by splitting the available time on a particular processor between each of these applications.

Now, with multicore CPUs it is possible to have “true” parallel processing.  Different applications can run at the same time on multiple different cores.  Each of these cores can also be multiplexed to run even more applications than cores.

The ability to implement true parallel processing enables developers to design their applications to take full advantage of their platforms’ abilities.  The image above illustrates a simple example of this.  Both the parallel and serial processing applications shown achieve the same goal (i.e. filling the array with random strings), but the one on the left fills all four array slots at the same time, while the other builds up the array over multiple steps.

Learn Secure Coding

Build your secure coding skills in C/C++, iOS, Java, .NET, Node.js, PHP and other languages.

Learn More

Types of Parallel Processing

Parallel processing can be implemented in a couple of different ways.  It is possible to have parallelization using either threads or processes.  The choice between these determines what resources are shared between the different threads of execution and have impacts on their vulnerability to exploitation.

Threads

Parallel processing using threads makes it possible for multiple threads of execution to occur within the same application.  This enables a higher level of state sharing than implementing parallel processing using discrete processes.

The image above illustrates the difference between a single-threaded and multi-threaded application.  The left image is a single-threaded application, while the right image shows an application with multiple different threads.

In the multi-threaded application, threads share some resources and don’t share others.  As demonstrated, the different threads use the same application code and share low-level resources like data and files.  However, each thread has its own registers and stack.

This use of multiple different threads makes it relatively easily to coordinate different operations between threads.  Global variables are included in the shared “data”, and the threads have the same codebase, making it possible for them all to run identical functions if desired.  While this type of synchronization is possible across multiple different processes, it is more complex as it requires the use of files to store shared state.

Processes

A multi-threaded application is one that is designed to implement parallel processing.  However, an application does not necessarily have to be designed this way to run multiple different threads in parallel.

For example, it is entirely possible to run multiple different instances of the same application (like Microsoft Word) at the same time.  If this occurs, then it is possible to perform processing on multiple different cores at the same time.  An application may or may not be designed to support this (which is where an application could run into trouble).

When parallelizing an application using processes, it is more difficult to share state and coordinate across the different processes.  Unlike threads, processes do not have shared global variables and only share access to low-level resources like the file system.  This means that any coordination has to be implemented using shared files, etc.

Learn Secure Coding

Build your secure coding skills in C/C++, iOS, Java, .NET, Node.js, PHP and other languages.

Learn More

Parallel Processing and Security

The ability to perform parallel processing - whether using multi-threading or multiple processes -  is very helpful for developers.  Programs designed to be parallelized can run much more quickly, and are likely to be designed in a more modular fashion.

However, the use of parallel processing also has security implications.  Developing a parallelized program (or having one that could be parallelized by running multiple different instances at once) creates the potential for race condition vulnerabilities.

Sources

1. https://sebastianraschka.com/Articles/2014_multiprocessing.html
2. https://www.cs.miami.edu/home/geoff/Courses/CSC322-11S/Content/UNIXProgramming/UNIXThreads.shtml
3. https://www.veracode.com/security/race-condition