skip to content

Department of Computer Science and Technology

Principal lecturer: 
MPhil ACS, Part III
Lent term
Course code: 
Python programming skills (other programming languages are also OK but not always ideal), basic background in signal/image processing and/or machine learning desirable
Class limit: 


Understanding, (automatically) analysing and modelling people's affective and social behaviour is very important for multiple domains such as enhancing human-computer and human-agent interaction, improving gaming technology and players’ experience, behavioural analytics for the banking sector, etc.

Accordingly, the aim of this module is to impart knowledge and ability needed to make informed choices of models, data, and techniques for sensing, recognition, and expression of human affective and social behaviour (e.g., smile, frown, head nodding/shaking, agreement/disagreement), and its use in the design of innovative interactive technology (e.g., interaction with virtual agents, robots, and games; single and multi-user smart environments, e.g., in a car; implicit (multimedia) tagging; clinical and biomedical studies, e.g., autism, depression, pain).


The following list provides a representative list of topics:

  • Introduction, definitions, overview and applications
  • Emotion theories
  • Sensing: from multiple modalities of vision, audio, bio signals, text
  • Data acquisition and annotation
  • Signal processing and feature extraction
  • Automatic recognition, prediction and evaluation
  • Synthesis: Affect and expression synthesis and generation
  • Emotional design frameworks
  • Advanced topics and Ethical considerations
  • Hands-on programming work (i.e., practicals and mini-project)


On completion of this module, students will:

  • Understand the challenges in human-human affective and communicative interaction (e.g. not what is said but how it is said – using the body, head, face, intonation, etc.) and its implication to Human-Computer Interaction;
  • Demonstrate knowledge in current theories and trends in designing emotionally and socially sensitive interactive technology, as well as recent advances in human audio/visual/bio signal processing, and recognition using machine learning and pattern recognition techniques;
  • Comprehend and apply (appropriate) methods for collection, analysis, representation and evaluation of human affective and communicative behaviour data;
  • Demonstrate ability to computationally analyse, recognise and evaluate human affective and social behaviour;
  • Enhance programming skills for human affect and behaviour analysis and understanding;
  • Demonstrate critical thinking, analysis and synthesis while making a decision on 'when' and 'how' to incorporate emotions and social signals in a specific application context, and gain practical experience in proposing and justifying computational solution(s) of suitable nature and scope.


Practicals: 15%
Seminars: 20%
Mini-Project: 65% (written report, code and presentation)

Coursework will include:

  • Practical 1 report (written report)
  • Practical 2 report (written report)
  • Practical 3 report (written report)
  • Seminar presentation (PDF of presentation slides)
  • Mini project report, code and PDF of presentation slides

NOTE: This module has a practical element. If the module is run remotely due to COVID-19 restrictions, changes to the practical work and assessment will be required 

Recommended reading

Picard, R. (2000). Affective Computing. MIT Press.

Jeon, M. (2017). Emotions and Affect in Human Factors and Human-Computer Interaction. Academic Press.

Calvo, R., D'Mello, S., Gratch, J. and Kappas, A. (2014) The Oxford Handbook of Affective Computing. Oxford University Press.


  1. IEEE Transactions on Affective Computing

Conference proceedings:

  1. ACII: Affective Computing and Intelligent
  2. ICMI: ACM International Conference on Multimodal Interaction
  3. FGR: IEEE Conference on Automatic Face and Gesture Recognition

Further Information

Due to COVID-19, the method of teaching for this module will be adjusted to cater for physical distancing and students who are working remotely. We will confirm precisely how the module will be taught closer to the start of term.