Deep Patch Representations with Shared Codebook for Scene Classification,

Abstract

Scene classifcation is a challenging problem. Compared with object images, scene images are more abstract,which are composed of objects. Object and scene images have different characteristics with different scales andcomposition structures. How to effectively integrate the local mid-level semantic representations includingboth object and scene concepts needs to be investigated, which is an important aspect for scene classifcation.In this paper, the idea of sharing codebook is introduced by organically integrating deep learning, concept feature and local feature encoding techniques. More specifcally, the shared local feature codebook is generated from the combined ImageNet1K and Places365 concepts (Mixed1365), using convolutional neural networks. As the Mixed1365 features cover all the semantic information including both object and scene concepts, we can extract shared codebook from the Mixed1365 features which only contains a subset of the whole 1365 concepts with the same codebook size. The shared codebook can not only provide complementary representations without additional codebook training, but also be adaptively extracted towards different scene classifcation tasks. A method of fusing the encoded features with both the original codebook and the shared codebook is proposed for scene classifcation. In this way, more comprehensive and representative image features can be generated for classifcation. Extensive experimentations conducted on two public datasets validate the effectiveness of the proposed method. Besides, some useful observations are also revealed to show the advantage of shared codebook.

• Shuqiang Jiang, Gongwei Chen, Xinhang Song, and Linhu Liu. 2018. Deep Patch Representations with Shared Codebook for Scene Classifcation. ACM Trans. Multimedia Comput. Commun. Appl. , , Article （Accepted), 17 pages.