Generation of facial nonverbal behavior for SIA:

A convolutional generative adversarial approach

Pdf
📚 Table of Contents

This page provides supplementary material for the thesis Generation of facial nonverbal behavior for Socially Interactive Agents: videos, evaluation websites, and source code.

Chapter 3 - Dataset Trueness

Examples of 4 interactions from the Trueness dataset.
We edited the videos so that the two participants appear side-by-side, with their recordings respectively on the left and on the right. The perpetrator of the discriminatory behavior always appears on the right. This participant is the one receiving the instruction regarding which social attitude to enact. The participant on the left, the victim in context scenes or the witness in conflict scenes, adapts their behavior accordingly.

4 examples are provided in the playlist, illustrating the following settings:

  • Context scene
  • Conflict scene with conciliatory attitude
  • Conflict scene with hot anger attitude
  • Conflict scene with cold anger attitude

Watch the videos

Examples of feature extraction replayed on a virtual agent
4 examples are provided in the playlist, illustrating the following settings:

  • Without preprocessing (Greta SIA)
  • With preprocessing (Greta SIA)
  • With preprocessing (Retorik SIA)

Watch the videos

Chapter 4 - FaceGen

Evaluation

Visualisation of the four excerpts for each condition (Greta)
We edited the videos so that the speech of both participants is clearly audible. For each condition, four examples are provided, corresponding to the four excerpts used in the evaluation.

GTS m1 m2 m3

Evaluation Website
We provide the link to the online evaluation platforms used for the user study.

Evaluation

Code


The full implementation of our model (FaceGen v1) is available in the GitHub repository below.

Code

Chapter 5 - Objective evaluation

Evaluation

Visualisation of the four excerpts for each condition (Retorik)
We edited the videos so that the speech of both participants is clearly audible. For each condition, four examples are provided, corresponding to the four excerpts used in the evaluation

Best Comp. Score Best DTW Best AUFGD Random

Evaluation Website
We provide the link to the online evaluation platforms used for the user study.

Evaluation

Chapter 6 - FaceAttGen

Affective-oriented evaluation

Visualisation of the four excerpts for each condition (Retorik)
We edited the videos so that the speech of both participants is clearly audible. For each condition, four examples are provided, corresponding to the four excerpts used in the evaluation.

GTS FaceAttGen - Implicit
FaceAttGen - Anger conditioned FaceAttGen - Concilatory conditioned

Evaluation Website
We provide the link to the online evaluation platforms used for the user study.

Evaluation

Believability and coordination-oriented evaluation

Visualisation of the four excerpts for each condition (Retorik)
We edited the videos so that the speech of both participants is clearly audible. For each condition, four examples are provided, corresponding to the four excerpts used in the evaluation.

GTS FaceAttGen FaceGen

Evaluation Website
We provide the link to the online evaluation platforms used for the user study.

Evaluation

Code


The full implementation of our models (FaceGen V3 & FaceAttGen) is available in the GitHub repository below.

Code

Chapter 7 - FairGenderGen

Evaluation

Visualisation for each condition (Greta)
We provide several videos where both conditions, FaceGen V2 and FairGenderGen, are shown side by side in the same clip to facilitate comparison.

GTS FaceGen V2 vs. FairGenderGen

Evaluation Website
We provide the link to the online evaluation platforms used for the user study.

Evaluation

Code


The full implementation of our models (FairGenderGen & FaceGen V2) is available in the GitHub repository below.

Code