The only R implementation of the kernel k-means I found online is that of kernlab package. According to the documentation of the method kkmeans, the method accepts a parameter called centers that is described as:
"Either the number of clusters or a matrix of initial cluster centers. If the first a random initial partitioning is used."
It was not clear to me what kind of centers I should pass to the method If I wanted to use my own initial cluster assignments. Passing centers in the original feature space does not make any sense since those centers are not necessarily good centers in the kernel space. According to my understanding, the proper way for passing initial clustering to kernel k-means is to specify the cluster Id of each data sample. The description above of the parameter centers was confusing to me. Moreover, the method returns the final centers of the clusters. The returned centers are of the same dimension as the original feature space which, to my knowledge, cannot be obtained unless an approximate (or an expensive full) decomposition is used. I posted my inquiry to R mailing list but, I did not get any useful answer.
Another limitation with the kkmeans method is that it does not allow specifying a convergence criteria (e.g. maximum number of iterations). Controlling both the initial centers and the convergence criteria is important indeed for comparing different variations. Moreover, I was getting error messages that I could not find any clue how to fix them. An example on such error messages is:
So, I had to write my own R implementation of the weighted kernel k-means.The code is available here.
In my implementation, I am handling the empty cluster problem by randomly picking a sample that was assigned to the empty cluster in the previous iteration and moving it back to that cluster. It might not be the optimal solution, but it is the cheapest in terms of memory and time requirements. The implementation trades off time for space. That is why I am not using the triangular inequality described in [1].
[1] Dhillon et. al. "Kernel k-means, Spectral Clustering and Normalized Cuts". In KDD'04.
"Either the number of clusters or a matrix of initial cluster centers. If the first a random initial partitioning is used."
It was not clear to me what kind of centers I should pass to the method If I wanted to use my own initial cluster assignments. Passing centers in the original feature space does not make any sense since those centers are not necessarily good centers in the kernel space. According to my understanding, the proper way for passing initial clustering to kernel k-means is to specify the cluster Id of each data sample. The description above of the parameter centers was confusing to me. Moreover, the method returns the final centers of the clusters. The returned centers are of the same dimension as the original feature space which, to my knowledge, cannot be obtained unless an approximate (or an expensive full) decomposition is used. I posted my inquiry to R mailing list but, I did not get any useful answer.
Another limitation with the kkmeans method is that it does not allow specifying a convergence criteria (e.g. maximum number of iterations). Controlling both the initial centers and the convergence criteria is important indeed for comparing different variations. Moreover, I was getting error messages that I could not find any clue how to fix them. An example on such error messages is:
"Error in crossprod(x[vgr[[i]], vgr[[i]], drop = FALSE], w[vgr[[i]]]) :
object 'vgr' not found"
So, I had to write my own R implementation of the weighted kernel k-means.The code is available here.
In my implementation, I am handling the empty cluster problem by randomly picking a sample that was assigned to the empty cluster in the previous iteration and moving it back to that cluster. It might not be the optimal solution, but it is the cheapest in terms of memory and time requirements. The implementation trades off time for space. That is why I am not using the triangular inequality described in [1].
[1] Dhillon et. al. "Kernel k-means, Spectral Clustering and Normalized Cuts". In KDD'04.
