detecting human emotions from videos
|
emotions from facial expressions |
|
The sentiment detections (labels) are detected automatically. The video is unconstrained, downloaded from youtube. The analysis of sentiments is complementary information to the long term activities detection.
For the demo, we consider 6 different classes/emotions: happy, angry, sad, neutral, closed eyes, and not_a_face. We have collected a dataset composed of 20412 images, where each class is trained with 3402 images. The classification error rate is 6% in a frame by frame basis. |
emotions in the wild |
|
|
We have built deep learning algorithms to understand facial emotions in wild videos. The video (left) displays the results of the detections. The emotions: sad, happy, surprised, closed eyes. The numbers on the side of the label is the confidence of the detection.
While the results are acceptable, understanding the context and combining the face with the context seems to be an interesting way to understand the overall frame -as a collective moment- . |
Understanding happiness |
|
Can an algorithm understand a happy moment ? We trained a deep neural network to understand happy moments. And applied the algorithm to extract video chunks that were understood as a happy moment. The interesting thing is that for this algorithm NO HACKS were used like combinations of features (e.g. face detection + human pose ). Check the original and result video. This is the first prototype of the algorithm, interesting enough is how important pose and social interactions are. Attend closely to the green/red progress bar, the bar represents the confidence of the algorithm. The bar takes a "green color" when the certainty is over 30%. The most happy moments are including social interactions (see result video).
ORIGINAL VIDEO |
Result videoOur algorithm takes the original video and extracts the parts that are detected as happy moments.
|
COntext + Face |
|
Below we show qualitative results of different deep neural networks to understand facial emotions and the context (e.g. selfie, group, etc.).