Seminar Conversational Animation for Virtual Humans:
Virtual humans are widely used in many research and commercial applications, be it as an assistant on the Ikea website, in (serious) games, or as assistants in the homes of the elderly. By allowing interaction in the form of a conversation, virtual humans provide human computer interfaces that are intuitive and pleasant to work with. Nonverbal social behavior plays a crucial role in communicating information in conversations.
In this course you will be working on designing and implementing algorithms that generate -in real time- conversational animations such as gestures, head nods and facial expressions. Techniques that can be explored include (but are not limited to) motion capture editing, procedural animation, corpus based machine learning.
Isy project: The next generation GUI – combining keyboard, mouse, speech and virtual assistants: Human-like virtual agents can provide intuitive and entertaining user-interface elements. Classical interfaces - such as mouse/keyboard and buttons on a machine - are less entertaining and intuitive, but provide more accurate and more efficient interaction, especially for trained users. In this project, you will work on an a framework that allows one to explore the ideal mix between those two approaches in a user-friendly manner.
Seminar “Konversation am Lagerfeuer” – Verhaltenssimulation für virtuelle humanoide
In diesem Seminar sollen Methoden untersucht und implementiert werden, um virtuelle
Charaktere wirklich “lebendig” zu machen. Es gibt dazu in der Literatur zu vielen
Fragestellungen theoretische Modelle, teils mit ersten Umsetzungen. Wir wollen uns einige
ansehen, kritisieren, und dann in eigenen Implementationen umsetzen.
In unserem Szenario macht ein virtueller Agent mit einem Nutzer einen Campingurlaub in
der Wildnis. Abends, am Lagerfeuer, findet eine Unterhaltung statt. Wir haben vier
interessante Herausforderungen herausgearbeitet, die zur Wahl stehen und jeweils in
einer Gruppe bearbeitet werden sollen.
Below: some suggestions for bachelor or master projects. Of course these suggestions may be modified to your liking and you can bring your own ideas for projects as well. I'm especially interested in supervising projects on multi-modal behaviour realization and on real-time computer animations for virtual humans.
Automatic generation of beat gestures: In co-speech gestures, hand movement is aligned to the rhythm of
speech. This is especially clear in so-called beat gestures (for example in political
speeches). Other gestures (iconics, metaphorics, pointing gestures) may be overlaid by
such rhythmical movement. Ultimately, we would like to automatically generate beat
gestures synchronized to speech, given a text a virtual character is about to speak.
Teaching Billie to move his hands:We move our hands to grab a cup of coffee, to drive a car, to scratch the back of our
head or to gesture. All these tasks require us to position our hand somewhere in space,
while avoiding collision of our arms with the rest of our body and other objects. In this project you will learn our virtual character Billie to move his hands to achieve
Humans are capable of very rapid turn-taking in conversation.
Typically, one interaction participant starts speaking immediately after (or even before)
the previous speaker finishes his turn. However, sometimes such fluent turn taking
breaks down, for example because both parties want the turn or because one of the
parties incorrectly assumed the other was finished speaking. Several strategies are
employed by speakers to indicate that they want to keep the turn, including increasing
volume and/or pitch of ongoing speech, changing their speech rate in overlap), or reuttering
the turn so far. In this project, you will design two virtual characters that have a
conversation with one other. Each of them is to have a turn-taking strategy that takes into consideration the
speech of the other character.
Designing a movement repertoire for a virtual human:
A Virtual Human is expected to react to others and the world by selecting an
appropriate movement (gesture, locomotion, …) from a big set of movements he is capable of –
this set is called a movement repertoire. In this project you will be designing a subrepertoire for a
certain type of movement of a virtual human. You are
free to select what movement type you want to model. Examples include head and eye
movement for gaze, arm gestures for pointing, walking, etc.
The movement model used to generate the motion could be based on models from
biomechanics or behavior science, or could consist of clever combination and adaptation of
Understanding Spoken User Interface Commands:
Speech interaction provides a natural way to interact with a user interface. However,
designing user interfaces that use speech input is still challenging. One challenge is
that there are many ways to provide the same information to the interface, which should
all be understood by it. For example, sentences like "Ok", "Continue", "That's fine", "Go
on", could all mean to go to the next screen in a particular user interface, or "I would like
to make a new appointment", "new appointment", "please set up a new appointment",
etc. could all mean to set up a new appointment in a calendar user interface.
In this project, you will be using and evaluating speech recognition to understand
spoken commands to control a user interface.