Memory issues

It seems that the memory management of tensorflow sucks.

A simple VGG16 with a batch of 128 makes the application crash by OOM (with 11GB of memory available)

https://github.com/tensorflow/tensorflow/issues/492