Sign language team in VIPL won the first place in 2016 ChaLearn lap Large-Scale continuous Gesture recognition Challenge. The task of competition is to identify 4042 unaligned RGB-D continuous sign language videos under the condition of given training set and verification set.
We adopted the technical route of hand detection, sequential segmentation and sign language recognition, and set up a two-stream RNN framework to achieve sign language recognition. A paper on the work was published in ICPRW2016 *. In our method, hand motion information and hand shape information are both considered to describe the words. RGB and depth information are computed in parallel in SRNN-layer, and then combined in fusion layer. Finally, LSTM is used for recognition.
Among the total of 50 teams all over the world, our team received the highest recognition scores.
- Xiujuan Chai, Zhipeng Liu, Fang Yin, Zhuang Liu, Xilin Chen. Two Streams Recurrent Neural Networks for Large-Scale Continuous Gesture Recognition. ICPRW 2016.
Download: