Neurocognition and Action - Biomechanics
© Universität Bielefeld
Adaptive and Mobile Action Assistance in Daily Living Activities (ADAMAAS)
Adaptive technical systems - for an intuitive interaction between humans and complex technologies
Project duration: 05/2015 - 042018
Project funding: 1.600.000 Euro
Share (Bielefeld University): 1.200.000 Euro
The award-winning ADAMAAS project focused on the research and development of intelligent glasses as cognitive task assistance systems (Fig. 1). It combines techniques from human memory research, eye tracking and physiological measurements (such as breathing rate or heart rate), object and action recognition (Computer Vision), Augmented Reality (AR), and advanced cognitive assessment and intervention techniques. In 2018, the initiative "Land of Ideas", founded by the German government and the Federation of German Industries, praised ADAMAAS as an innovative and forward-looking project that benefits the country and its inhabitants, and honored it as one of 100 "Landmarks in the Land of Ideas" (out of approx. 1,500 candidates).
Almost everyone knows at least one of the following situations: you want to bake a cake but cannot remember the exact recipe; you do not know how to use a high-tech kitchen stove to prepare a delicious meal or simply how to repair a bicycle. In such everyday situations it may be helpful to receive unobtrusive and intuitive support from an adaptive technical system that operates along in a largely unnoticed and restriction-free manner. The ADAMAAS system shall be able to identify actual action processes, to anticipate mistakes and react when errors are made, as well as to display situation- and context-dependent assistance in textual, 2D and 3D in-situ format superimposed on a transparent virtual plane in users' field of view. ADAMAAS aims to provide support for people to be able to live a self-sufficient life in an age-appropriate way according to their mental and physical capabilities. Thus, the system will be able to suggest new action options and to induce selective learning processes. ADAMAAS is therefore not a stationary, but a mobile assistance system.
In order to allow for age-appropriate, individual assistance in everyday situations, cognitive action sequences will be diagnosed in a first step using the structural-dimensional analysis of mental representations (SDA-M; Schack, 2012). The method applied is well-established in cognitive psychology and captures users' individual mental representation structures (Fig. 2). The system is then able to identify user-specific problems in the execution of actual action sequences (Strenge et al. 2019). Based on this information, the current visual focus and the detected objects in the environment, the system is able to provide adequate help, i.e. by displaying situation- and user-specific feedback and hints.
A particular problem in the context of technical assistance systems is the handling (e.g., the recording and storing) of sensitive data. Therefore, we will pay particular attention to privacy and data security. The recorded data shall be transferred via a secured data connection on a local host system, which is installed on-site. Therefore, no displaced person can access or distribute this potentially sensitive data.
Several research groups based in Bielefeld University and external companies will work together to achieve the project goals. The v. Bodelschwingh Foundation Bethel will contribute their expertise in diagnosis and action assistance, as well as taking care of disabled and elderly people. Furthermore, they will provide an ideal test environment and provide us with access to different target groups for the system evaluation stage of the project. Bielefeld University is represented by three working groups: Sport Science, the Center of Excellence CITEC (Cognitive Interaction Technology) and the Faculty of Technology. Together they bring a wealth of experience in the areas of Image Processing/Computer Vision, Machine Learning, mental representation structures for diagnoses and action execution, feedback/prompting, sport and movement science, Augmented Reality (AR), artificial systems, character animation, man-machine interaction, user experience, ELSI (ethical, social and legal implications), and speech processing. The eye tracking company SensoMotoric Instruments Gesellschaft für innovative Sensorik mbH (SMI), based in Berlin, will provide the necessary hardware (see Fig. 1) and Velamed GmbH Science in Motion (Medizintechnik und Biomechanische Konzepte), based in Cologne, integrates the sensors for measuring the vital parameters. The iTIME (Institute for Technological Innovation, Market Development and Entrepreneurship) provide advice in economic matters, to conduct market analysis and the preparation of a marketing concept. The initiative Bielefeld 2000plus - Research Projects for the Region will establish a working group on the topic "Intelligent Assistance Systems for Elderly People" to foster the dialog between academia and representative from society.The Hettich Holding GmbH & Co. oHG will evaluate the AR-Eyetracking Glasses in the context of the acquisition and support in a complex assembly process.
For realizing diagnostic and action assistance we draw on Bethels comprehensive experience in the field of disabled and elderly persons care. The v. Bodelschwingh Foundation Bethel are involved in diagnostic processes and assistance of action in a real world context (workshop and living unit). They also contribute to the system evaluation for people with special needs (elderly and handicapped people).
In a nutshell, the aim of this project is to go beyond stationary status diagnosis systems by developing a mobile and adaptive action support and monitoring system that is able to anticipate issues and provide individualized assistance for different activities.
For more information, see [ here ] .
Figure 1: ADAMAAS provides situation, user, and action specific feedback in a textual, visual or avatar based format on a transparent virtual plane superimposed on user's field of view.
Figure 2: Basic principles of the ADAMAAS system: Detection of the cognitive action structure (left). Supportive prompting on the AR plane (right).
Related Publications
Web Releases
Intelligente Brille: Bielefelder Forscher entwickeln neues Assistenzsystem für Senioren. Neue Westfälische (02.08.2016).
Bethel und Uni Bielefeld entwickeln intelligente Brille. V. Bodelschwinghsche Stiftungen Bethel (25.11.2015).
54 Smart Glasses Of The Future For 2016. The Optical Vision Side (16.12.2015).
SMI Shows First Eye Tracking Upgrade for Augmented Reality Glasses. The Optical Vision Side (16.12.2015). Stefanie Gehrke (SMI), MarketWatch.com (06. August 2015).
SMI Shows First Eye Tracking Upgrade for Augmented Reality Glasses. Stefanie Gehrke (SMI), PR Newswire (06.08.2015).
Intelligente Datenbrille als individuelles Assistenzsystem. Kersten Auel, Heise Online (KW 27 2015).
ADAMAAS smart glasses to assist elderly and disabled in everyday tasks. Emily Price, Gizmag Online (05.07.2015).
Wearable Technology: A Game Changer for the disabled. Keith White, Parker White - brand interactive (30.07.2015)
ADAMAAS Assistive Glasses for Elderly & Disabled. Disabled Worlds - Towards tomorrow (06.07.2015).
ADAMAAS Smart glasses Can Assist The Elderly And Weak In Their Day-to-Day Activities. Talha Dar, wonderfulengineering.com (07. Juli 2015).
The ADAMAAS Smart Glasses Assist the Elderly in Everyday Tasks Rahul Kalvapalle, trendhunter.com (09.07.2015).
Researchers during a Cluster of Excellence are building a new mobile assistance system. globalnewsconnect.com (09.07.2015).
AR-Projekt: Die Welt durch eine Brille sehen. Virtual Dimension Center / vdc-fellbach.de (07.2015).
Uni Bielefeld - Das Projekt ADAMAAS. Augment3d - Augmented & Virtual Reality (12.07.2015).
ADAMAAS Smart Glasses Explain Basic Tasks. ideaconnection - Build on the Genius of Others (09.07.2015).
ADAMAAS-Projekt konzentriert sich auf das Entwickeln der beweglichen Unterstützungsanlage, die Benutzer in den täglichen Situationen unterstützt News Medical - Life Sciences & Medicine (02.07.2015).
IHK Innovationsnachrichten Juli 2015
"Умные" очки ADAMAAS
Inteligentne okulary ADAMAAS pomagaja chorym i niepelnosprawnym
radiozet.pl
Adamaas umnye ochki prizvannye pomogat pojilym lyudyam i invalidam
json.tv
ADAMAAS智能眼镜可协助老年人和残障人士完成日常任务
cnBeta.com (06.07.2015, Translation: gizmag)
Daha cok yasliv ve engellileri hedef alan akilli gozluk
Press Releases
Personalized Cognitive Assistance (and ethical system development) Pan European Networks, Congleton, U.K, 2017
Unendliche Aktenschränke und lernende Brillen
Bielefeld University, H1 (2), S. 30-31, 01.02.2016, Hermwille, Andreas
Intelligente Brille als Alltagshelfer
Forscher entwickeln neues Assistenzsystem für Senioren
Westfälische Nachrichten Ibbenbürer Volkszeitung, 03.08.2016.
Brille der Zukunft unterstützt Senioren im Alltag
Neue Westfälische, 0308.2016
Brille zeig mir was Du weißt!
BMBF Projektgalerie 2015, S. 16 - 17, 07.07.2015
Bethel und Universität entwickeln intelligente Brille
Der Ring
S. 14-15, Burg, Robert, 01.01.2016
Brille hilft beim Kaffeekochen
Digitale Brille unterstützt alte und behinderte Menschen im Alltag
Westfalen Blatt, Nr. 20, S. 2, 25.01.2016
Intelligente Brille hilft bei Einschränkungen
Neues System: Gemeinschaftsprojekt der Uni und Bethel
Neue Westfälische, 30.11.2015
Eine Brille für alle Fälle
Neue Westfälische, 03.07.2015
AR-Glasses Help to Guide the Elderly
Allen, Tom, Mobile Display Monitor from Meko, Vol. 22, Nr. 9, S. 25., 07.2015
Der Bewegung auf der Spur
Wirtschaftswoche, 01.07.2015
New Perspectives on Human Performance
Pan European Networks
Science & Technology
Vol 16, S. 80-81, 01.09.2015
Television/Broadcasts
DieSuperbrille
WDR Lokalzeit Sendung, 01.04.2016, 19.30 Uhr
Interview with Benjamin Strenge at GENIALE “Hertz 87,9” Campusradio, 24 October 2017
ADAMAAS - Eine unterstützende Brille für den Alltag
YouTube https://youtu.be/lr6-LDAOqOo
Imagefilm: Bethel-150 für Menschen da sein.
YouTube http://www.youtube.com/watch?v=yLN3wyDFD2M
V. Bodelschwinghsche Stiftungen Bethel, 14.02.2017
Workshops
Multimodal Learning and Social Interaction – Assistive Technology for Preventing Human Errors
CITEC Summer School 2016, 27th Sep. 2016, Bielefeld
Workshop Digitale Welt – IT‐Anwendungen für soziale Inklusion
Vision Summit, 11 November 2016, Campus Pariser Platz, Berlin
BBB-Fachtagung TeilhabeUNternehmen - Wider den Barrieren
Bildungszentrum des Handels e.V., Recklinghausen March 17 2017, Recklinghausen
Workshops on Ethical, Legal and Social Implications of ADAMAAS
Feb/May/June/September 2017, Bielefeld
Research stay of Dr. Kai Essig at the Institute for InfoComm Research
1 Fusionopolis Way, #20-10 Connexis North Tower, Singapur 138632, July 24-30 2017 (funded by the Singaporean partner)
Aim: discuss possibilities for an international cooperation
CITEC –I2R Mixed Reality for Human Enhancement Workshop at the Institute for InfoComm Research
1 Fusionopolis Way, #20-10 Connexis North Tower, Singapur 138632 (with speakers from Singapore, Germany and USA)
Bilateral workshop on Assistive Technologies at Rhine-Waal University of Applied Sciences
September 2018 with speakers from Singapore, Germany and Great Britain.
Fairs/Exhibitions
AUTOMATICA
21. - 26.06.2016, Munich
Wundertüte Arbeit 4.0, Kongress zur Zukunft der Arbeitswelt
29.09.2016 in Bad Oeynhausen
FMB Zuliefermesse Maschinenbau
09. - 11.11.2016, Bad Salzuflen
Hannover Messe, it’s OWL stand
24. - 28.04.2017, Hannover
GENIALE Science Festival
23.08.2017, Bielefeld
ZiF Konferenz: Arbeit 4.0 Potentiale, Probleme, Perspektiven
21.11.2017, Zentrum für Interdisziplinäre Forschung (ZiF), Bielefeld
Wissenschaftliche Leitung: Günter Maier, Joanna Pfaff-Czarnecka, Véronique Zanetti (Bielefeld)
Third-party funds
German Research Foundation (DFG) Grant for Initiation of International Cooperation
Topic: Design and Evaluation of a Modular Adaptive Cognitive Mobile Assistive System for flexible use in professional and private environments. Together with Dr. Ma Keng Teck, Science and Engineering Institutes (SCIE), A*-Star, Singapore.Awarded Funds 36.625 € for travel expenses and joint workshops.
Submitted:
BMBF-Antrag “Audiovisuelle Unterstützung durch ein kognitives und visuelles Assistenzsystem (avikom)” für die Fördermaßnahme:
Zukunft der Arbeit – Mittelstand (eingereicht am 31.08.2017).
Das Konsortium besteht aus der AG Neurokognition und Bewegung – Biomechanik und der Arbeits- und Organisationspsychologie (Uni Bielefeld), dem Institut für Systemtechnik und Mechatronik (FH Bielefeld) sowie 5 regionalen KMUs.