What does vectorization mean?

Is it a good idea to vectorize the code? What are good practices in terms of when to do it? What happens underneath?

Vectorization means that the compiler detects that your independent instructions can be executed as one SIMD instruction. Usual example is that if you do something like

for (i = 0; i < N; i++) {
    a[i] = a[i] + b[i];
}

It will be vectorized as (using vector notation)

for (i = 0; i < (N - N % VF); i += VF) {
    a[i : i + VF] = a[i : i + VF] + b[i : i + VF];
}

Basically the compiler picks one operation that can be done on VF elements of the array at the same time and does this N/VF times instead of doing the single operation N times.

It increases performance, but puts more requirement on the architecture.

As mentioned above, vectorization is used to make use of SIMD instructions, which can perform identical operations of different data packed into large registers.

A generic guideline to enable a compiler to autovectorize a loop is to ensure that there are no flow- and anti-dependencies b/w data elements in different iterations of a loop.

http://en.wikipedia.org/wiki/Data_dependency

Some compilers like the Intel C++/Fortran compilers are capable of autovectorizing code. In case it was not able to vectorize a loop, the Intel compiler is capable of reporting why it could not do that. There reports can be used to modify the code such that it becomes vectorizable (assuming it's possible)

Dependencies are covered in depth in the book 'Optimizing Compilers for Modern Architectures: A Dependence-based Approach'