Modeling Restaurant Context for Food Recognition.

Abstract

Food photos are widely used in food logs for diet monitoring and in social networks to share social and gastronomic experiences. A large number of these images are taken in restaurants. Dish recognition in general is very challenging, due to different cuisines, cooking styles, and the intrinsic difficulty of modeling food from its visual appearance. However, contextual knowledge can be crucial to improve recognition in such scenario. In particular, geocontext has been widely exploited for outdoor landmark recognition. Similarly, we exploit knowledge about menus and location of restaurants and test images. We first adapt a framework based on discarding unlikely categories located far from the test image. Then, we reformulate the problem using a probabilistic model connecting dishes, restaurants, and locations. We apply that model in three different tasks: dish recognition, restaurant recognition, and location refinement. Experiments on six datasets show that by integrating multiple evidences (visual, location, and external knowledge) our system can boost the performance in all tasks.

• L. Herranz, S. Jiang, R. Xu, “Modeling Restaurant Context for Food Recognition”, IEEE Transactions on Multimedia, vol. 19, no. 2, pp. 430-440, Feb. 2017.

• L. Herranz, R. Xu, S. Jiang, “A probabilistic framework for food recognition in restaurants”, Proc. International Conference on Multimedia and Expo 2015 (ICME15), pp. 1-6, Torino, Italy, June 2015 (earlier conferene version)