Although the speed of a single-core processor has increased considerably over the decades, further gains in computing power are possible through the use of multiple cores or processors. Using multiple cores or processors is called parallel computing. In parallel computing, certain computations are partitioned into independent smaller subcomputations, which are then processed on separate cores or processors simultaneously. Consumer-grade PCs and servers are often equipped with multicore processors, and configurations that consist of multiple processors are becoming relatively common and inexpensive. As a result, parallel computing is becoming increasingly important.

One type of parallel computing is multithreaded computing on a single machine; this execution mode is called single-machine mode. In single-machine mode, several threads use the processors of a single computer to work concurrently on subtasks. These threads share the random access memory (RAM) of that computer.

Another type of parallel computing is distributed computing, in which computation is parallelized over a number of processors on different computers. This mode is called distributed mode. In distributed mode, each computer owns an independent memory allocation, and computation can be multithreaded on each computer.