Search-By-Example in Multilingual Sign Language Databases

Ralph Elliott, Helen Cooper, John Glauert, Richard Bowden and Fran├žois Lefebvre-Albaret


The Search-By-Example tool presented in this paper is a proof-of-concept prototype that has been constructed to show how sign recognition and sign synthesis can be combined in a simple lookup tool. The user signs in front of the Kinect(TM) device and is presented with animations of one or more signs recognised as similar to the sign performed. A chosen sign can be presented in all of the four sign languages supported so that a user can view and compare signs in different languages. We present the sign recognition process and the sign synthesis process in some detail. We then describe to architecture and graphical interface of the prototype tool. Finally we discuss results and user evaluation of the system.


Main Topic:  USABILITY


Requirements for signing avatar technology
Realism and acceptability of signing avatars
Use of corpora to inform animation
Evaluation results for practical translation systems


[Paper (PDF)]  

Supported by START Conference Manager