Question regarding online clustering between modalities.

[giuseppeCruised]

Hello,
In the paper of concerto i was wondering how exactly you did the "cross modality online clustering" criterion in table 6 (a). On what targets did you do the cross entropy? as far as i know the DINOv2 Projector+clustering weights are not available?

[Yujia-Zhang0913]

Hi, the code for online clustering refers to https://github.com/Pointcept/Pointcept/blob/main/pointcept/models/concerto/concerto_v1m1_base.py#L34. I use separate heads for cross-modality learning. As for the MLP for DINOv2 online clustering part, I tried directly copying from the point encoder part and the EMA version, together with the EMA update for the teacher encoder. Other parts are learning through training.