Home

Awesome

Hetero-Mark

A Benchmark Suite for collaborative CPU-GPU computing.

Prerequisite

AMD ROCm

See ROCm on how to install the latest version of ROCm.

OpenCV Library

The Background Extraction benchmark will use OpenCV for video decoding and encoding. The benchmark suite will detect if your system has OpenCV installed or not. If OpenCV libraries are not found, CMAKE will skip compiling the BE benchmarks.

We use the following command to install OpenCV libraries.

sudo apt install libopencv-dev

Applications

Hetero-Mark is designed to model workloads that are similar to real-world applications, where the major part of the application is written in general-purpose programming languages, while only a small, performance-critical portion is written using GPU-accelerated libraries. So for each benchmark, we provide a base class that provides platform-independent functionalities, such as input data loading and result verification. For each GPU programming method (such as CUDA, HC, HIP), we extend the base class with a sub-class and implement the "Run" method.

Since the base classes are platform-independent, we use plain pointers for input and output data. Each benchmark must read from plain pointers and finally write the result into other plain pointers. We believe this behavior is closer to real-world scenarios since most programmers do not carry a platform-specific memory management system to the whole application and usually only use GPU programs as a library. This also suggests that the benchmarking time considers the data copy time between the CPU and the GPU memory.

All the benchmarks have a verification process where the GPU result is compared with the CPU result. Although we report the execution time of the verification process, the time is not meant to compare the CPU performance to GPU performance. The verification process can be very useful if the benchmark runs in simulators or if the validity of the platform is under evaluation.

Compiling the code

OpenCL

Use the following commands to compile the OpenCL benchmarks.

mkdir build
cd build
cmake -DHMARK_BUILD_OPENCL12=On -DHMARK_BUILD_OPENCL20=On ../
make

If OpenCL is properly configured in your system, the command above will use the system default compiler to compile OpenCL benchmarks.

This command will also use the HCC compiler to compile the OpenCL benchmarks.

CUDA Compilation

Use the following commands to compile CUDA benchmarks. Make sure your system has NVCC compiler installed.

mkdir build
cd build
cmake -DHMARK_BUILD_CUDA=On ../
make

HIP Compilation

Use the following commands to compile HIP benchmarks.

mkdir build
cd build
cmake -DHMARK_BUILD_HIP=On ../
make

HIP works for both CUDA platform and the ROCm platform.

Run the code

The executables are in the build folder under Hetero-Mark/build/src/<application name>/<environment> if you follow the default compile guide, where <application name> is the name of the application, such as, fir, be, bs etc and replace <environment> for cl12, cl20, cuda or hc.

The executables support the following arguments:

All benchmark executables have a -h option. The help documentation of each benchmark explains how to use the benchmark and what parameter is needed.

Input data

Download standard input data

You can download the standard data from the following URL https://heteromark.s3.us-east-2.amazonaws.com/hmark-data.zip.

Generate your own input data

Development Guide

Please raise issues on the GitHub page if you have any questions or problems using the benchmark suite.

We accept pull requests on GitHub if you want to contribute to the benchmark suite. If you have any questions or problems with HeteroMark, please file an issue in our GitHub repo.

Hetero-mark follows google c++ coding style in header files and source files. We also use make check to lint the source code using the clang-format tool and cpplint tool.

Citation