Can we assign K-Means and GMM M-steps to different queue?
Instead of using
grid.training_queue for all
gmm-m-step, can we use say
The reason I am suggesting this that
initialization of KMeans and GMMs is usually much more demanding on memory (the features are loaded in memory) compared to
m-steps (statistics are loaded). So, assigning all these steps to the same machine with large memory (which is usually a bottleneck resource), looks like a waste of resources and makes the whole training slow. Since, typically,
grid.training_queue is memory demanding, while
grid.enrollment_queue is not so much, it seem to make sense to assign these steps accordingly.