Conference language

The conference language is English and sign language interpreters will be available.

We encourage you to check out the related International Congress of Linguists conference also in Cape Town.

Gesture form analysis

Introduction and hands-on ELAN coding training of an open-data corpus

Trainer
Dr Julius Hassemer
Research Fellow of the German Research Foundation (DFG)
Universidade de São Paulo; Humboldt-Universität zu Berlin

Students and researchers are invited to participate in a short practical course on “form” in gesture conceptualisation and how to systematically code it. The driving question in this course is “How do (manual) movements translate to meaning?” One approach to answering this question is to tease apart which spatial information is derived from specific patterns of configuration and motion of the body.

The theoretical framework of gesture form analysis (Hassemer, 2016; Hassemer & McCleary, subm.; see also Hassemer et al., 2011; Hassemer & Winter, 2016; subm.) has significant overlap with recent work on gesture semantics by Talmy (2018). It integrates insights about the spatial conceptualisation of gesture, including work on “gestural modes of representation” (Müller, 2014), metonymy (Mittelberg & Waugh, 2009), “profiling” (Sowa, 2006; Langacker, 2008), as well as “sketching” and “stamping” (Mandel, 1977) in a single framework.

In the course, after giving an introduction to gesture form analysis, the focus is on the practical analysis of gesture in ELAN with a lot of hands-on coding and comparing in class. Other issues such as coding for a specific hypothesis, setting up your own controlled vocabularies, use of open-data sources are touched upon.

Info

  • Date: July 9 and 10, after the ISGS8 in Cape Town (morning: Course; afternoon: Free coding training/discussion outside of class)
  • Location: University of Cape Town
  • Costs: We are trying to offer this course free of charge, but we may charge a nominal fee of about 50€ to compensate for our costs (discounts for African students), depending on current funding efforts.
  • Please contact g@juliushassemer.de to register or if you have any questions

Requirements for participants:

1. Bring a laptop

  • with the newest version of Elan installed

2. Download the open data corpus “NM-MoCap”
(https://www.researchgate.net/publication/324168684_Corpus_objectDescription_Video_Mocap_Manuals_Templates_version2) including the coding manual (in English or Portuguese) and the ELAN coding template file (GFAv06_template.eaf).

  • Read the coding manual
  • Copy the coding template file into the folder of participant 01 and link it to the video p01_Trial011.66694118.mp4 (detailed instructions in the coding manual).
  • Make yourself familiar in annotating gestures following the coding system of gesture form analysis. In case you have problems, contact me.

3. Optional: send me your own research interest, hypotheses, data, or motivation for gesture coding so that I can addressed this in the course.

References

Hassemer, J. (2016). Towards a Theory of Gesture Form Analysis. Imaginary forms as part of gesture conceptualisation, with empirical support from motion-capture data. Dissertation, RWTH Aachen University.
Hassemer, J., Joue, G., Willmes, K., & Mittelberg, I. (2011). Dimensions and mechanisms of form constitution: Towards a formal description of gestures. In Proceedings of Gesture and Speech in Interaction (GESPIN) 2011. Bielefeld.
Hassemer, J., & McCleary, L. (subm.). The multidimensionality of pointing. Gesture.
Hassemer, J., & Winter, B. (2016). Producing and perceiving gestures conveying height or shape. Gesture, 15(3), 404–424.
Hassemer, J., & Winter, B. (subm.). Decoding gestural iconicity. Cognitive Science.
Langacker, R. W. (2008). Cognitive Grammar. A basic introduction. Oxford/NewYork: Oxford University Press.
Mandel, M. A. (1977). Iconic devices in american sign language. In L. A. Friedman (Ed.), On the other hand: new perspectives on american sign language (pp. 57–107). New York: Academic Press.
Mittelberg, I., & Waugh, L. R. (2009). Metonymy first, metaphor second: A cognitive-semiotic approach to multimodal figures of thought in co-speech gesture. In C. Forceville & E. Urios-Aparisi (Eds.), Multimodal Metaphor (pp. 329–356). Berlin/New York: Mouton de Gruyter.
Müller, C. (2014). Gestural Modes of Representation as techniques of depiction. In C. Müller, A. Cienki, E. Fricke, S. H. Ladewig, D. McNeill, & J. Bressem (Eds.), Body – Language – Communication: An International Handbook on Multimodality in Human Interaction, Vol. 2 (pp. 1687–1702). Berlin/Boston: Mouton de Gruyter.
Sowa, T. (2006). Understanding Coverbal Iconic Gestures in Shape Descriptions. Dissertation, University of Bielefeld.
Talmy, L. (2018). The targeting system of language. MIT Press.

Gesture form analysis of Drawing

Gesture form analysis of Drawing

Coding in Elan

Coding in Elan