Retrieving loss info in MTL

[trunghng]

Great paper. I wonder if there is any way to retrieve the loss information in multi-task learning?
In particular, in the example of MTL, do you still have to add loss=loss1+loss2 into the script?

[ValerianRey]

Hi! In multi-task learning, we typically consider each loss independently, as they each represent a different task. You may still want to track the average loss, or the sum of losses, by recording the value of loss=loss1+loss2 for instance. But note that with JD, this loss is not necessary anymore for training, as you use the individual losses instead.
Also, note that loss1 and loss2 may have very different scales (e.g. loss1 could be typically around 1e-5 while loss2 could be around 1e-2). In this example, summing them could thus give a value very close to loss2, which would give little information on loss1.

So yes, you can do that, but be careful!