Universität Bielefeld

© Universität Bielefeld

MovVis

A movement visualization toolbox

 

MovVis Toolbox for Matlab

On this page you will find the Matlab/Octave code of our movement visualization toolbox MovVis.

  • MovVis creates videos and animated gifs of input movement data.
  • MovVis can display movements at any speed.
  • MovVis has three options for aligning multiple movements.
  • MovVis can display movements on a chosen background image.
  • MovVis can display movements with desired visual attributes.
  • The toolbox was conducted with Matlab 2013b. The toolbox requires Matlab's statistics and image processing toolboxes (commercially available from MathWorks), but should work without any further Matlab toolboxes installed. The toolbox was adapted to Octave 4.4.1 (open source) and requires Octave's statistics, image processing, and video packages. MovVis has only been tested under Windows.

     

    Tutorial

    Download the MovVis toolbox. Unzip the files and set them to your Matlab path.

    Open Matlab or Octave. Open MovVis_Script.m by double clicking or typing

    >>open MovVis_Script.m;

    This file explains the MovVis toolbox and contains code for example visualizations on the example movement data file mymovedata.txt. All movement data you want to visualize with MovVis must have the same format as the example data. Each row contains one movement sample. The first column contains the timestamps. In a variable number of following columns, coordinates can be tagged by categories to which they belong (e.g., subject in column 2, trial in column 3, and condition in column 4). The next column (2+number of category columns) marks to which sub-movement unit the sample belongs. The next column (3+number of category columns) marks to which event the sample belongs. The last two columns contain the x- and y-coordinates, respectively.

    Data columns: timestamp, category 1-n, unit, event, x, y.

    Specify the data by constructing a variable called 'data' containing either a string of the full path to your movement data file or is a matrix containing the data.

    >>data='C://User/data/mymovedata.txt';

    Visualize the data with default options by typing in the Command Window:

    >>MovVis(data);

    Alternatively, you can specify your desired visualization options. Maximal specification is achieved by

    >>[xy,xyA]=MovVis(data,picpath,speed,vissize,alignment,prefix,drawtrace,missub,marker,faceCol,edgeCol,traceCol,traceLine,markerSize);

    You can choose as many specifications as you like. Note, however, that they need to have the right name and format as explained in MovVis_Instructions.txt and MovVisScript.m in the MovVis folder.

     

    Citation and Contact

    When publishing data visualized with MovVis or results interpreted with the help of MovVis, cite:

    Foerster, R. M. (2019). MovVis: A movement visualization toolbox for Matlab and Octave. CITEC, Bielefeld University. doi:10.4119/unibi/2934028

    In case of problems or questions contact Rebecca M. Foerster

     

    Example data visualizations

    MovVis_Script also contains eight example visualizations on the example movement data 'mymovedata.txt' in the MovVis folder. The example data are described in detail in

    Foerster, R. M. (2016). Task-irrelevant expectation violations in sequential manual actions: Evidence for a 'check-after-surprise' mode of visual attention and eye-hand decoupling. Frontiers in Psychology, 7, 1-12. https://doi.org/10.3389/fpsyg.2016.01845

    The example data contains cursor (event 9) and eye (event 23) movements defined in column 5, recorded from 2 trials (65 & 66) defined in column 3, belonging to 20 subjects (1-20) defined in column 2, while subjects performed a sequential number clicking task (computerized TMT-A). In this task, subjects have to click as fast as possible and in ascending sequence on numbered circles displayed on a computer screen (here 1-9). The movements until a click on a specific number are tagged by the number identity (unit 1 for all movements until click on number 1). The file contains 100 eye and 30 cursor movement samples per second. A constant visuospatial number-configuration as can be viewed from the ScreenPrechange.jpg picture in the MovVis folder was completed for 65 trials (prechange trials). Unannounced to the subjects, in trial 66, the font of number 4 changed from Arial font (ScreenPrechange.jpg) to the font MVBoli as can be seen in the ScreenChange.jpg picture in the MovVis folder.

    Run the examples in MovVisScript.m for getting acquainted with MovVis.

    The following ten example visualizations will be stored as gif and avi video.

     

    Example 1: Default starts-aligned visualization

    animated gif
    By default, all movements are visualized as black 10-point circles moving continuously on a grey background. Click on the gif to open the video.

     

    Example 2: Fully specified units-aligned visualization

    animated gif
    Eye movements are visualized as 6 point-sized diamonds and leave a broken trace, while cursor movements are visualized as 10 point-sized circles and leave a solid trace. All moving markers have white marker edge color. The marker face color of all movements belonging to the prechange trial 65 are visualized in high contrast blue (0 0 1) and leave a lower contrast blue trace (0.4 0.4 0.8). The marker face color of all movements belonging to the change trial 66 are visualized in high contrast red (0.6 0.2 0.2) and leave a lower contrast red trace (0.4 0.2 0.2). All movements are superimposed on the display with the deviant font of number 4 (ChangeScreen.jpg). The temporal resolution of all movements is upscaled to 150 Hz, and the missing interim data points of the higher resolution are substituted by the next point, thereby creating a smooth slow-motion visualization when replayed with 25 Hz (typical display speed). As not only starts, but also ends are aligned per sub-unit, movements are streched to match a unit's slowest 150-Hz upscaled movement. The spatial resolution of the visualization is reduced to half the size of the original picture size [height of 384, and width of 512]. Inspecting the visualization shows nicely that during the last prechange trial (blue), participants scan the display sequentially with their eyes (diamonds and broken lines) followed by the cursor (circles and solid lines) with a slight delay. During the first change trial (red), the eyes (diamonds and broken lines) initially scan the display sequentially. However, after having completed the changed number 4, the eyes look back at it occasionally without much of an effect on the cursor movements (circles and solid lines). This indicates that they noticed the change and checked for it, however, without letting it deteriorate their cursor performance too much. This was only possible by occasionally decoupling eye and cursor (hand) movements. Click on the gif to open the video.

     

    Example 3: Starts-aligned visualization with selected specifications.

    animated gif
    By defining only the background picture (picpath), the marker styles(marker), their face color (faceCol), and their size (markerSize) as in example 2, defaults are used for speed (real-time), alignment (starts-aligned), missub (substitution active), edge color (edgeCol=faceCol), and the trace specifications (no trace). The spatial resolution of the visualization is reduced to half the size of the original picture size [384,512]. Thus eye movements are visualized by small red diamonds and cursor movements by big blue circles, both without leaving a trace. Markers belonging to the prechange trial 65 are blue and markers belonging to the change trail 66 are red. The temporal resolution of all movements is reduced to 25 Hz and only their starts are aligned, so that they are diplayed in real-time. Click on the gif to open the video.

     

    Example 4: Ends-aligned visualization with individual colors per subject

    animated gif
    The movements are visualized with randomly chosen individual colors per subject. Note that all four movements (eye and cursor in prechange and change trial) belonging to a specific subject have the same face color. Example 4 visualizes eye movements as diamonds and cursor movement as circles. Movement markers belonging to the prechange trial have black edge colors, while movements belonging to the change trial have white edge colors. Movements are again superimposed on the half-sized change display. Starts and ends of all trials are aligned ignoring unit bounderies. Click on the gif to open the video.

     

    Example 5: Selected movements - prechange trial

    animated gif
    Only the movements of the prechange trial are visualized. Eye movements are visualized as green diamonds leaving green broken traces and cursors move as black circles leaving black solid traces on the half-sized prechange display, all with white marker edge color. Starts and ends are alinged per unit after cursor movements were upscaled to 100 Hz. The example shows nicely how movement data with different temporal resolutions can be handeled by MovVis. Each position shift of the cursor markers is accompanied by three position shifts of the eye markers. Click on the gif to open the video.

     

    Example 6: Selected movements - change trial

    animated gif
    Only the movements of the change trial are visualized. Eye movements are visualized as green diamonds leaving green broken traces and cursors move as black circles leaving black solid traces on the half-sized change display, all with white marker edge color. Starts and ends are alinged per unit after cursor movements were upscaled to 100 Hz. The example shows nicely how movement data with different temporal resolutions can be handeled by MovVis. Each position shift of the cursor markers is accompanied by three position shifts of the eye markers. Click on the gif to open the video.

     

    Example 7: Eye and cursor movement of subject 3 in the change trial

    animated gif
    Only two movements are visualized, namely the eye and cursor movement of the third subject in the change trial. Eye movements are visualized as green diamonds leaving a green broken trace and the cursors move as black circles leaving a black solid trace on the half-sized change display. Movements are down-scaled to the default 25 Hz and only starts are aglined. From the visualization it can be viewed nicely that subject 3 fixates on the number with the deviant font four times. The first fixation is followed by a cursor movement, while the three refixations are not. Click on the gif to open the video.

     

    Example 8: Unit-averaged eye movements of subject 3

    animated gif
    Only two movements are visualized, namely the unit-averaged eye movement during the prechange and the change trial of subject 3. Eye movements of the prechange trial are visualized as blue solid traces and eye movements of the change trial are visualized as red solid traces on the half-sized change display. Each data point is displayed for 200 ms. From the visualization it can be viewed nicely that the averaged scanpaths of subject 3 are highly similar in the prechange and the change trial despite the font change. Click on the gif to open the video.

     

    Example 9: Units-aligned averaged movements (average observer)

    animated gif
    Based on the units-aligned data (extracted by examples 5 and 6), mean x- and y-coordinates of all subjects' movements are calculated per event and trial. The resulting data matrix contains four movement paths. An average path of the eye movements during the prechange trial. An average paths of the cursor movements during the prechange trial. An average path of the eye movements during the change trial. An average path of the cursor movements during the change trial. Eye movements are visualized as diamonds leaving broken traces and cursors move as circles leaving solid traces on the half-sized change display. Movement markers belonging to the prechange trial are blue and leave blue traces. Movement markers belonging to the change trial are red and leave red traces. Movements are starts-aglined and visualized with 100 Hz. From the visualization it can be viewed that the average cursor paths have relatively straight sequential connections between the ascending numbers. The eye movement traces are slightly curved towards number 4 when considering connections of numbers higher than 3 due to the individual refixations of number 4. However, the average eye movement path touches number 4 only once, namely when it is followed by a cursor movement. This reveals that the different subjects refixate on the number 4 at different times within the seqeunce, so that averaging brings the paths more in line with the required clicking sequence. The example shows nicely how averaging can cover individual patterns (all participants refixate on number 4) and pretend smooth equal patterns (all participants scan sequentially with some curvature to number 4) across participants that can lead to wrong conclusions (no inspection of the surprising number 4). Click on the gif to open the video.

     

    Example 10: Successive movements of all subjects

    animated gif
    The movements of all subjects are visualized successively. Eye movements are visualized as diamonds and cursor movements as circles. Movements belonging to the prechange trial are visualized in blue and movements belonging to the change trial are visualized in red. All movements are visualized on the half-sized change display. A pause of 1,000 ms separates the successive movements. From the visualization it can be seen how eye and cursor movements of each individual subject differ in the prechange and the change trial. Click on the gif to open the video.

     

    Acknowledgement

    This research was supported by the Cluster of Excellence Cognitive Interaction Technology CITEC (EXC 277) at Bielefeld University, which is funded by the German Research Foundation (DFG).