How does tf.train.ExponentialMovingAverage work with tf.estimators?

Judging from the documentation looks like tf.train.ExponentialMovingAverage does nothing but to keep track of the average of an variable. However, this average value should be used in predict/eval time as originally intended. The way to do this is explained in the documentation, in https://github.com/tensorflow/tensorflow/issues/3460, and http://ruishu.io/2017/11/22/ema/ . There is also https://www.tensorflow.org/api_docs/python/tf/contrib/opt/MovingAverageOptimizer

From the looks of it, we do not do any of these solutions but changing apply_moving_averages to True or False in the Logits class changes our results.

So now I am wondering what the heck is going on!