The Search-By-Example tool presented in this paper is a proof-of-concept prototype that has been constructed to show how sign recognition and sign synthesis can be combined in a simple lookup tool. The user signs in front of the Kinect(TM) device and is presented with animations of one or more signs recognised as similar to the sign performed. A chosen sign can be presented in all of the four sign languages supported so that a user can view and compare signs in different languages. We present the sign recognition process and the sign synthesis process in some detail. We then describe to architecture and graphical interface of the prototype tool. Finally we discuss results and user evaluation of the system.
Main Topic: USABILITY
Requirements for signing avatar technology
Realism and acceptability of signing avatars
Use of corpora to inform animation
Evaluation results for practical translation systems
[Paper (PDF)]
Supported by START Conference Manager