What restrictions does this architecture impose on the algorithms being executed:

If we perform calculations on a GPU, then we cannot allocate only one core, a whole block of cores will be allocated (32 for NVIDIA).
All cores execute the same instructions, but with different data (we'll talk about this further), such calculations are called Single-Instruction-Multiple-Data or SIMD (although NVIDIA is introducing its own specification).
Due to the relatively simple set of logical blocks and general registers, the GPU does not like branching very much, and in general, complex logic in algorithms.

What opportunities opens up:

Actually, the acceleration of those very SIMD calculations. The element-wise addition of matrices can serve as the simplest example, and let's analyze it.

Reduction of classical algorithms to SIMD representation

Transformation

We have two arrays, A and B, and we want to add an element from array B to each element of array A. Below is an example in C, although, I hope, it will be understandable for those who do not speak this language:

void func (float * A, float * B, size)
{
for (int i = 0; i <size; i ++)
{
A [i] + = B [i]
}
}

Classic loop traversal and linear execution time.

Now let’s see how this code will look for the GPU:

void func (float * A, float * B, size)
{
int i = threadIdx.x;
if (i <size)
A [i] + = B [i]
}