Neurocognition and ­Action - Biomechanics

Action event discrimination and sensorimotor interlingua

Project duration: 2015/12 - 2018/12
Project funding: 230.000 Euro

The combined study of cognition through action, embodiment and language is a unique challenge. The idea is that a lot of issues concerning motor event conception and representation in combination with their labeling ?possible in different languages- could be better understood. The scope of this work is event segmentation of a motion continuum through the isolation of the minimum elements that distinguish an action, in order to create a motion id for each individual. These separate motion events will be named in different languages to form a sensorimotor interlingua. Labelling of sensorimotor data emerges as a necessity for robotics. The state-of-the-art claim is that robots better identify events when they are taught their names (assuming the availability of multi-modal input) that is, when they couple a sensorimotor experience with a symbol (any word, phrase, etc); it follows robots that perceive a continuous series of motion events will be in a position to better identify them if they are taught their names. So far, robots and other cognitive systems have been taught only one language (basically, English) and issues of ?perspective? have not turned up. It is clear that our languages enable us to communicate a lot of things, when we have the intention to do that, but (i) they do not enable us to communicate everything and (ii) they somehow filter what we communicate. In fact, some linguists have gone so far as to claim that our languages eventually filter (or express the filter that affects) the way we perceive reality. There is an ongoing debate on this issue: recent studies in cognitive linguistics suggest that sensorimotor input is the same for all humans and that language-independent categorization of events converges among native speakers of typologically different languages (despite the fact that the corresponding linguistic description does not). Rather than filtering perception, it is suggested that languages favour some available perspectives of viewing events.

Expected outcomes:

• Event segmentation and sensorimotor interlingua
• the cognitive mechanism of learning and recognizing motion events and verbs,
• the critical moments that distinguish actions,
• the individual?s criteria of each word?s selection
• the influence of lexicalization pattern of path and manner languages in recognition memory and similarity judgements.

Knowledge diffusion:

Event discrimination in motor action and a sensorimotor intelingua may benefit cognitive psychology, neuroinformatics and intelligent tutoring systems? community by presenting robust learning mechanisms based on sensorimotor visualisations. Except from the final outcome, motion intelligence and situated communication will be enriched with valuable information such as the critical moments that distinguish actions, the individual?s criteria of each word?s selection. On the other hand, cognitive linguistics may promote grounding of language to action to higher levels, search the importance of non-verbal elements (actions and gestures) in the concept formation and study the influence of lexicalization pattern of path and manner languages in recognition memory and similarity judgements.