TCA issue computing matrix

[etiennevandebijl]

Hi,

I am implementing TCA and saw an issue in line 126 of _tca.py file:
Implemented is the following:
a = np.eye(n+m) + self.mu * K.dot(L.dot(K))
However, according to the authors, it should be: K.dot(L.dot(K)) + self.mu * np.eye(n+m) (mu should be multiplied with the identity matrix)
I hope this helps.

[antoinedemathelin]

Hi @etiennevandebijl,
Thank you for your help!
Are you sure that the optimization should be K.dot(L.dot(K)) + self.mu * np.eye(n+m)? In the TCA paper, I see np.eye(n+m) + self.mu * K.dot(L.dot(K)) (cf. last paragraph of page 3).

But maybe the adapt documentation is wrong, it is written that "The larger mu is, the less adaptation is performed." but it may be the opposite...

[etiennevandebijl]

Hi @antoinedemathelin,

Thank you for looking into this. Indeed, in this unpublished version of the article proposing TCA, it is written as np.eye(n+m) + self.mu * K.dot(L.dot(K)). However, in the published peer-reviewed version of TCA it is K.dot(L.dot(K)) + self.mu * np.eye(n+m), see page 204 right above Algorithm 1. It doesn't unfortunately result in the same result, but I think it might be a choice of which work to implement...

[antoinedemathelin]

Hi @etiennevandebijl,
Thank you for noting this point. Indeed, this new version makes more sense, as mu is a regularization parameter.
Do you want to open a pull request to propose the modification?
Best,

[etiennevandebijl]

Hi @antoinedemathelin, to continue on this thread, I progressed with working on TCA and saw by experimentations that the eigenvectors and eigenvalues derivation on sol in line 130 of tca file is done with linalg.eigh function. This function assumes that sol is symmetric. However, even though a and b are symmetric in lines 126 and 127, this does not necessarily mean the `sol' in line 128 is. I realized this by computing multiple times the values. My suggestion would be to calculate the matrix with linalg.eig as it does not make the wrong symmetry assumption. When computing self.vectors.T @b@self.vectors_, the solution makes more sense.