What if I don’t want to spend a couple of months reading these books, writing my own video card program, testing and debugging, and then find out that all this is not for me?

As I said, there are a large number of libraries that hide the complexities of GPU development: XGBoost, cuBLAS, TensorFlow, PyTorch and others, we will consider the thrust library, since it is less specialized than the other libraries above, but at the same time it implements the basic algorithms, for example, sorting, searching, aggregation, and with a high probability it can be applied in your tasks.

Thrust is a C ++ library that aims to “replace” standard STL algorithms with algorithms executed on the GPU. For example, sorting an array of numbers using this library would look like this on a video card:

thrust :: host_vector h_vec (size); // declare a regular array of elements
std :: generate (h_vec.begin (), h_vec.end (), rand); // fill with random values
thrust :: device_vector d_vec = h_vec; // send data from RAM to video card memory
thrust :: sort (d_vec.begin (), d_vec.end ()); // sort the data on the video card
thrust :: copy (d_vec.begin (), d_vec.end (), h_vec.begin ()); // copy the data back from the video card to RAM

(do not forget that the example must be compiled with the compiler from NVIDIA)

As you can see, thrust :: sort is very similar to the similar algorithm from the STL. This library hides a lot of complexity, in particular the development of a subroutine (more precisely, the kernel) that will be executed on a video card, but at the same time it does not provide flexibility. For example, if we want to sort several gigabytes of data, it would be logical to send a piece of data to the map to start sorting, and while sorting is in progress, send more data to the map. This approach is called latency hiding and allows more efficient use of the resources of the server card, but, unfortunately, when we use high-level libraries, such capabilities remain hidden. But for prototyping and measuring performance, they are just the right ones, especially with thrust, you can measure what kind of overhead the data transfer gives.

I wrote a small benchmark using this library that runs several popular algorithms with different amounts of data on the GPU, let’s see what the results are.