Why is the KMeans training decreasing the log-likelihood value?
Created by: siebenkopf
I am running a KMeans+GMM training, and I observe an unusual behavior. While for the GMM training, the reported log-likelihood increases, the opposite is the case for KMeans:
(KMeans training)
bob.learn.em@2016-04-27 16:04:36,859 -- INFO: Iteration = 0/200
bob.learn.em@2016-04-27 16:04:39,823 -- INFO: log likelihood = 5.961308
bob.learn.em@2016-04-27 16:04:39,823 -- INFO: convergence value = 0.291013
bob.learn.em@2016-04-27 16:04:39,823 -- INFO: Iteration = 1/200
bob.learn.em@2016-04-27 16:04:42,786 -- INFO: log likelihood = 5.717406
bob.learn.em@2016-04-27 16:04:42,787 -- INFO: convergence value = 0.040914
bob.learn.em@2016-04-27 16:04:42,787 -- INFO: Iteration = 2/200
bob.learn.em@2016-04-27 16:04:45,750 -- INFO: log likelihood = 5.611370
bob.learn.em@2016-04-27 16:04:45,750 -- INFO: convergence value = 0.018546
bob.learn.em@2016-04-27 16:04:45,750 -- INFO: Iteration = 3/200
bob.learn.em@2016-04-27 16:04:48,718 -- INFO: log likelihood = 5.545547
bob.learn.em@2016-04-27 16:04:48,719 -- INFO: convergence value = 0.011730
bob.learn.em@2016-04-27 16:04:48,719 -- INFO: Iteration = 4/200
bob.learn.em@2016-04-27 16:04:51,683 -- INFO: log likelihood = 5.501309
bob.learn.em@2016-04-27 16:04:51,683 -- INFO: convergence value = 0.007977
...
(GMM training with ML_GMMTrainer)
bob.learn.em@2016-04-27 16:09:41,702 -- INFO: Iteration = 0/200
bob.learn.em@2016-04-27 16:09:57,604 -- INFO: log likelihood = -13.541919
bob.learn.em@2016-04-27 16:09:57,604 -- INFO: convergence value = 0.686622
bob.learn.em@2016-04-27 16:09:57,604 -- INFO: Iteration = 1/200
bob.learn.em@2016-04-27 16:10:12,833 -- INFO: log likelihood = -11.647493
bob.learn.em@2016-04-27 16:10:12,833 -- INFO: convergence value = 0.139893
bob.learn.em@2016-04-27 16:10:12,833 -- INFO: Iteration = 2/200
bob.learn.em@2016-04-27 16:10:28,326 -- INFO: log likelihood = -11.342057
bob.learn.em@2016-04-27 16:10:28,326 -- INFO: convergence value = 0.026223
bob.learn.em@2016-04-27 16:10:28,326 -- INFO: Iteration = 3/200
bob.learn.em@2016-04-27 16:10:43,907 -- INFO: log likelihood = -11.179415
bob.learn.em@2016-04-27 16:10:43,908 -- INFO: convergence value = 0.014340
bob.learn.em@2016-04-27 16:10:43,908 -- INFO: Iteration = 4/200
bob.learn.em@2016-04-27 16:10:59,492 -- INFO: log likelihood = -11.064207
bob.learn.em@2016-04-27 16:10:59,493 -- INFO: convergence value = 0.010305
bob.learn.em@2016-04-27 16:10:59,493 -- INFO: Iteration = 5/200
bob.learn.em@2016-04-27 16:11:15,094 -- INFO: log likelihood = -10.983832
bob.learn.em@2016-04-27 16:11:15,094 -- INFO: convergence value = 0.007264I assume that, whatever is reported to be the log-likelihood value of the KMeans training is actually something else...