To use Theano in multi-threaded mode, install OpenBLAS with option
OPENMP enabled, install py-numpy with OpenBLAS support and use this
minimal ~/.theanorc:

---------------------------
[global]
cxx = %%CXX%%
openmp = True

[blas]
ldflags = -lopenblasp
---------------------------

You can control the number of threads used with OMP_NUM_THREADS environment
variable.