Overview

GraphicsMagick has been transformed to use OpenMP for the 1.3 release series. OpenMP is a portable framework for accelerating CPU-bound and memory-bound operations using multiple threads. OpenMP originates in the super-computing world and has been available in one form or another since the late '90s.

Since GCC 4.2 has introduced excellent OpenMP support via GOMP, OpenMP has become available to the masses. Microsoft Visual Studio Professional 2005 and later support OpenMP so Windows users can benefit as well. Any multi-CPU and/or multi-core system is potentially a good candidate for use with OpenMP. Unfortunately, some older multi-CPU hardware is more suitable for multi-processing than multi-threading. Modern multi-core chipsets from AMD, Intel and Sun/Oracle perform very well with OpenMP.

Most image processing routines are comprised of loops which iterate through the image pixels, image rows, or image regions. These loops are accelerated using OpenMP by executing portions of the total loops in different threads, and therefore on a different processor core. CPU-bound algorithms benefit most from OpenMP, but memory-bound algorithms may also benefit as well since the memory is accessed by different CPU cores, and sometimes the CPUs have their own path to memory. For example, the AMD Opteron is a NUMA (Non-Uniform Memory Architecture) design such that multi-CPU systems split the system memory across CPUs so each CPU adds more memory bandwidth as well.

For severely CPU-bound algorithms, it is not uncommon to see a linear speed-up (within the constraints of Amdahl's law) due to the number of cores. For example, a two core system executes the algorithm twice as fast, and a four core system executes the algorithm four times as fast. Memory-bound algorithms scale based on the memory bandwith available to the cores. For example, memory-bound algorithms scale up to almost 1.5X on my four core Opteron system due to its NUMA architecture. Some systems/CPUs are able to immediately context switch to another thread if the core would be blocked waiting for memory, allowing multiple memory accesses to be pending at once, and thereby improving throughput. For example, typical speedup of 20-32X (average 24X) has been observed on the Sun SPARC T2 CPU, which provides 8 cores, with 8 virtual CPUs per core (64 threads).

An approach used in GraphicsMagick is to recognize the various access patterns in the existing code, and re-write the algorithms (sometimes from scratch) to be based on a framework that we call "pixel iterators". With this approach, the computation is restricted to a small unit (a callback function) with very well defined properties, and no knowledge as to how it is executed or where the data comes from. This approach removes the loops from the code and puts the loops in the framework, which may be adjusted based on experience. The continuing strategy will be to recognize design patterns and build frameworks which support those patterns. Sometimes algorithms are special/exotic enough that it is much easier to instrument the code for OpenMP rather than to attempt to fit the algorithm into a framework.

Since OpenMP is based on multi-threading, multiple threads access the underlying pixel storage at once. The interface to this underlying storage is called the "pixel cache". The original pixel cache code (derived from ImageMagick) was thread safe only to the extent that it allowed one thread per image. This code has now been re-written so that multiple threads may safely and efficiently work on the pixels in one image. The re-write also makes the pixel cache thread safe if a multi-threaded application uses an OpenMP-fortified library.

The following is an example of per-core speed-up due to OpenMP on a four-core system. All the pixel quantum values have 30% gaussian noise added:

% gm benchmark -stepthreads 1 -duration 10 convert \
  -size 2048x1080 pattern:granite -operator all Noise-Gaussian 30% null:
Results: 1 threads 5 iter 11.07s user 11.07s total 0.452 iter/s (0.452 iter/s cpu) 1.00 speedup 1.000 karp-flatt
Results: 2 threads 10 iter 22.16s user 11.11s total 0.900 iter/s (0.451 iter/s cpu) 1.99 speedup 0.004 karp-flatt
Results: 3 threads 14 iter 31.06s user 10.47s total 1.337 iter/s (0.451 iter/s cpu) 2.96 speedup 0.007 karp-flatt
Results: 4 threads 18 iter 40.01s user 10.24s total 1.758 iter/s (0.450 iter/s cpu) 3.89 speedup 0.009 karp-flatt

Note that the "iter/s cpu" value is a measure of the number of iterations given the amount of reported CPU time consumed. It is an effective measure of relative efficacy since its value should ideally not drop as iterations are added. The karp-flatt ratio is another useful metric for evaluating thread-speedup efficiency. In the above example, the total speedup was about 3.9X with only a slight loss of CPU efficiency as threads are added.

According to the OpenMP specification, the OMP_NUM_THREADS evironment variable may be used to specify the number of threads available to the application. Typically this is set to the number of processor cores on the system but may be set lower to limit resource consumption or (in some cases) to improve execution efficiency. The GraphicsMagick commands also accept a -limit threads limit type option for specifying the maximum number of threads to use.

Results

A simple scheme was developed in order to evaluate the performance boost with varying numbers of threads. GraphicsMagick's built-in benchmark facility is used. The selected algorithm is executed repeatedly until a specified amount of time has elapsed. The input image is generated on the fly by tiling a small image over a large area using a specification like -size 4000x3000 tile:model.pnm. It is important to note that the time to generate the input image is included in the benchmark timings so that even if an algorithm achieves perfect linear scaling, the measured difference is likely to be less than the number of cores used and the impact could be substantial if image generation is slow. Many modern CPUs increase the core frequency substantially ("turbo mode") when only a few cores are being used and this unfairly penalizes the calculated per-thread speedup results which are based on the time to run with just one thread.

A typical benchmark command using the built-in benchmark facility (-stepthreads requires GraphicsMagick 1.3.13 or later) looks like:

gm benchmark -stepthreads 1 -duration 5 convert \
  -size 4000x3000 tile:model.pnm -median 2 null:

The first test executed is -noop since it does no work other than to generate the input image. This represents how fast it is possible to go based on how fast the input image may be generated.