ABSTRACT

We present Conversational Gesture Synthesizer, an online method for automatically generating and synthesizing gesture animations whose intensity and style are driven by live spoken speech, as well as a specified conversational attitude. Body gestures are adapted in such a way to have a believable strength relation between speech and gestures, whereas gesturing style is matched with the current conversational attitude. The method is data-driven and uses pre-recorded mocap motions to generate new ones. The pipeline is made up of three stages, the preprocessing stage, the online generation stage and the postprocessing stage. The preprocessing stage is responsible of segmenting all the mocap motions and creating a motiongraph structure, and it is the only stage that is offline of the three. The online generation stage takes the speech input, extracts prosody features out of it, and uses the constructed motiongraph to select appropriate motion segments, while the postprocessing stage concatenates the selected segments together and creates the final animation.