Revisiting Concatenative Video Synthesis with Relaxed Constraints

Sangyong Gil and Jong C. Park


A number of sign language translation systems have recently been proposed with the shared goal of providing improved accessibility for the deaf people. However, they are still largely in the stage of achieving correctness in a very limited domain, and much further work is needed to achieve naturalness with full emotion and non-manual signals, as compared to human signers and, equivalently, full video. Concatenative video synthesis has earlier been proposed to address scalability of full video, but it has become less popular with associated shortcomings. In this paper, we propose to improve this method by relaxing certain constraints, such as on signer and background, and see how the proof-of-concept videos are perceived by the deaf people. The result of this study shows that the revised concatenative video synthesis may provide a complementary solution for accessibility with adequate naturalness until sign language translation systems become fully mature.




Requirements for signing avatar technology
Flexibility in animation of facial gestures and mouthing
Realistic animation of manual and bodily gestures


[Paper (PDF)]  

Supported by START Conference Manager