Congratulations! VIPL's 1 paper is accepted by ICML (International Conference on Machine Learning) 2024! ICML is a top-tier international conference on machine learning. In this year, ICML will be held in Vienna Austria on July 21 through the 27. The paper is summarized as follows:
ReconBoost: Boosting Can Achieve Modality Reconcilement (Cong Hua, Qianqian Xu, Shilong Bao, Zhiyong Yang, Qingming Huang)
Current paradigms of multi-modal learning tend to explore multi-modal features simultaneously. The resulting gradient prohibits further exploitation of the features in the weak modality, leading to modality competition, where the dominant modality overpowers the learning process. To address this issue, we study the modality-alternating learning paradigm to achieve reconcilement. Specifically, we propose a new method called ReconBoost to update a fixed modality each time. Herein, the learning objective is dynamically adjusted with a reconcilement regularization against competition with the historical models. By choosing a KL-based reconcilement, we show that the proposed method resembles Friedman's Gradient-Boosting (GB) algorithm, where the updated learner can correct errors made by others and help enhance the overall performance. The major difference with the classic GB is that we only preserve the newest model for each modality to avoid overfitting caused by ensembling strong learners. Furthermore, we propose a memory consolidation scheme and a global rectification scheme to make this strategy more effective. Experiments over six multi-modal benchmarks speak to the efficacy of the proposed method.
Figure 1. The overview of proposed ReconBoost
Download: