keyvisual

Jelena Rosic & Pia Tikka
Annotation of film content for a neurocinematic analysis: Implications for embodied approaches to filmmaking

 

Today’s advanced neuroimaging methods allow studying intersubjective correlations of participants’ neural responses to continuous film stimuli. However, despite the high promises of the neurocinematic approach (Lahnamäki et al. 2012), without a careful annotation of the film content, the results of the neuroimaging studying may remain difficult to interpret. There is a growing interest for developing systematic annotation methods amongst the film research community (see, e.g. Bateman 2012). In this presentation we will take a closer look at the practical aspects of the annotation process by describing the manual annotation of the short experimental film “At Land’ by Maya Deren, using annotation tool ELAN by Max Planck Institute (http://tla.mpi.nl/tools/tla-tools/elan/).
The content of the film has been organized, on one hand, around the formal elements of cinematic expression (such as framing, angles, mise-en-scène, camera movements, continuity/ discontinuity, actors eye gaze and movements, composition, speed, POV, and so on). On the other hand, we have looked at body gestures, facial expressions and body situatedness in detail. The indicators assumedly reveal the character’s enactment with the diegetic world. In the case of “At Land” this apparently works even without explicit goal-orientation and in the absence of verbal interaction.Each set of annotations is synchronized on the timeline of the film so that any section of the film can be assessed with a detailed description of its content. This data is further compared with the results of the eye tracking data collected during neuroimaging (fMRI, MEG) experiments. In the latter, the analysis focuses to the spatial synchrony of the subjects’ gaze, in attempt to uncover similarities and differences, or expected and unexpected eye fixations, along thetemporal line of the annotated content.
Since the film “At Land” applies experimental filmmaking methods, which typically violate classical (narrative) cinematic conventions, special focus will be put on the annotated formal elements or features usually described as lyrical or dreamlike. We discuss these elements from the points of view of cognitive neuroscience, embodied mind, and, in particular, neurocinematics. The process of annotation has been informed through the professional expertise of the authors (film editor and cinematographer). In addition, a retrospective look at the writings of Maya Deren (McPherson 2005) and her contemporary Slavko Vorkapich (1959) allows us to extract the core ideas of avant-garde filmmakers around the notions of kinesthesia and synesthesia. The practice-based knowledge of these filmmakers will be extrapolated to the more recent discussion on embodiment in neurosciences (Freedberg & Gallese 2007) and cognitive film studies (Grodal 2009). Finally, we will discuss how the practical knowhow of filmmakers will eventually contribute to the neuroscientific interpretation, not only in terms of annotations of particular artistic aspects of the films under scrutiny but also in terms of the more general understanding of human behavior, motivations, and meaning-making that are indispensably present in the filmmaking process.

 

References

Bateman J. 2012. "What counts? Generating fine-grained hypotheses concerning filmic organisation for empirical studies" A presentation in SCSMI2012 conference, NY, US.

Freedberg D., and Gallese V. 2007. Motion, emotion and empathy in esthetic experience. Trends in Cognitive Sciences 11: 197-203.
Grodal T. 2009. Embodied Visions: Evolution, Emotion, Culture and Film. New York, NY: Oxford University Press.

Lahnakoski J. M., Salmi J., Jääskeläinen I. P., Lampinen J., Glerean E., Tikka P., and Sams M. 2012. Stimulus-Related Independent Component and Voxel-Wise Analysis of Human Brain Activity during Free Viewing of a Feature Film. PLoS ONE 7(4): e35215.

McPherson B. R. (ed.). 2005. Essential Deren: Collected Writings on Film. Kingston, N.Y.: Documentext.

Vorkapich S. 1959. “Toward True Cinema,'' in Film Culture (New York), April 1959; reproduced in Film: A Montage of Theories, ed. Richard Dyer MacCann (New York: Dutton Paperbacks, 1966).

 

CV

Jelena Rosic & Pia Tikka, Neurocine-Team, Aalto University Helsinki, Finland
School of Arts, Design and Architecture / Department of Motion Picture, Television and Production Design

Jelena Rosic, Master student, Netherlands Film and Television Academy; previously Faculty of Dramatic Arts, Belgrade, dept. Film and Television Editing. Attended Neuroaesthetics of Film, University of Amsterdam, 2012. Visiting researcher at the research team NeuroCine, Aalto University, fall 2012.

Pia Tikka, PhD, filmmaker, has directed features Daughters of Yemanjá (Brazil-Finland 1996), Sand Bride (Finland 1998), and worked in a number of international film productions. The author of Enactive Cinema: Simulatorium Eisensteinense (2008), Enactive Cinema project Obsession (2005) awarded with Möbius Prix Nordic prize, and co-author of interactive film-game Third Woman exhibited in Galapagos Art Space, NY (2011), she is also a founding member of research project Enactive Media (2009-2011), Aalto University Finland. Currently, Tikka is affiliated in the research project aivoAALTO. Her research team NeuroCine (www.neurocine.net) combines filmmaking practice with the methods of neuroimaging in order to study neural basis of cinematic imagination and filmmaker’s expertise.