VRHack

VR-Hack Munich 2021

Context & challenge:

The first Virtual Reality Hackathon is part of the Neuroergonomics Conference Munich in September 2021.
Virtual Reality (VR) has the ability to immerse us into alternative worlds and to create experiences we would likely never encounter in everyday life. Although the idea of VR goes back into the early 20st century, especially in recent years, VR has become more affordable and sophisticated than ever before. With the release of modern Head-Mounted-Displays (HMDs), VR is increasingly used by consumers for gaming, sport, education or entertainment. In combination with motion tracking, HMDs let us naturalistically perceive and interact with virtual worlds, creating the illusion of being in the virtual place.

VR has been promised as an empathy machine, able to e.g., let healthy people experience what it is like to live with disabilities. However, many people with physical impairments are often unable to experience VR without assistance and some are not even able to make head or body movement, required to experience immersive VR.

The goal of the weekend-long hackathon is to bring scientists and professionals with various backgrounds together to develop new interfaces with neuroscientific methods to increase the accessibility of VR for disabled people.

Hardware:

During the event, you will be able to work with hardware such as Oculus Quest 2, Mentalab and other cool stuff.

Hackathon Team:

Michael Wiesing:

Farbod Nosrat Nezami:

Marius Klug:

Talk:

Objective: Freezing of gait in people with Parkinson’s disease (PwP) is associated with executive dysfunction and motor preparation deficits. We have recently shown that electrophysiological markers of motor preparation, rather than decision-making, differentiate PwP with freezing of gait (FOG+) and without (FOG-) while sitting. To examine the effect of locomotion on these results, we measured behavioural and electrophysiological responses in PwP with and without FOG during a target response time task while sitting (single-task) and stepping-in-place (dual-task).

Methods: Behavioural and electroencephalographic data were acquired from 18 PwP (eight FOG+) and seven young controls performing the task while sitting and stepping in place.

Results: FOG+ had slower response times while stepping compared with sitting. However, response times were significantly faster while stepping compared with sitting for controls. Electrophysiological responses showed no difference in decision-making potentials (Centroparietal Positivity) between groups or conditions but there were differences in neurophysiological markers of response inhibition (N2) and motor preparation (Lateralized Readiness Potential, LRP) in FOG+ while performing a dual-task.

Conclusion: This suggests that the addition of a second complex motor task (stepping-in-place) impacts automatic allocation of resources in FOG+, resulting in delayed response times. The impact of locomotion on the generation of the N2 and LRP potentials, particularly in freezers, indirectly implies that these functions compete with locomotion for resources.

Significance: In the setting of multiple complex tasks or cognitive impairment, severe motor dysfunction may result, leading to freezing of gait.

Bio:

John S. Butler is a Lecturer in the School of Mathematical Sciences in Technological University Dublin. He has published over 60 papers in diverse venues from Cerebral Cortex to the International Journal for Numerical Methods in Fluids. In these papers he utilized numerical methods to analyse and help interrupt and predict large and complex datasets. John received his PhD in Numerical Analysis from the School of Mathematics, Trinity College Dublin under the supervision of Prof John Miller. Following this, he was a post-doctoral fellow from 2006-2009 at the Max Planck Institute for Biological Cybernetics in Tuebingen, Germany. During this time he collaborated with Prof Heinrich Bülthoff on research in self-motion perception using a variety of methods from low level psychophysics to complex closed loop tasks to electrophysiological recordings. From 2009 to 2013 he worked with Prof John Foxe and Prof Sophie Molholm at the Albert Einstein College of Medicine, Bronx, New York where he expanded the scope of his work to include a translational research component, specifically investigating the neural developmental differences in multisensory integration between typically developing children and children with Autism Spectrum disorder. From 2013 to 2015 he worked with Prof Richard Reilly at the Trinity Centre for Bioengineering in Trinity College Dublin on movement disorders such as Parkinson’s disease and Dystonia.

Talk: Mobile Brain/Body Imaging in Neuroergonomics

The report of Phineas Gage; the young man who survived an iron bar passing through his head, is one of the most famous clinical cases in the field of neuropsychology. However, this case is actually also a neuroergonomics one, as the promising young and intelligent Gage was prevented from continuing his work as railway construction supervisor due to the consequences of the severe traumatic brain injury. Gage was earlier described as ‘the most efficient and capable’ and ‘as a shrewd, smart businessman, very energetic and persistent in executing all his plans of action’, but lost due to the brain injury the ‘ability to anticipate the future and plan accordingly in a complex social environment’, and thus could not resume his previous work. This case is considered a historical milestone study exploring the relationship between brain functioning, behaviour, and cognition, and represents a classical example of executive dysfunctioning. What is the current status of the latest methods and theories from neuroscience to understand how the brain implements executive functions in everyday life?
This talk explores how neuroscientific knowledge regarding healthy interindividual differences in brain anatomy, executive functions and their relation to everyday life performance contributes to current challenges in the real world. First, the definitions of executive functions and their theoretical and methodological frameworks are presented. This includes the well-established unity and diversity model of Miyake (Miyake et al., 2000, Karr et al., 2018), the dual process framework of Braver (Braver et al., 2012), and the cognitive continuum approach from Dehais (Dehais et al., 2019). Second, the current neuroscientific knowledge regarding the development of neurofeedback trainings to enhance executive functioning is discussed (Enriquez-Geppert et al., 2014). The relevance of executive functions and its enhancement is discussed using examples from aviation (ID 149, ID 164).

Talk: From Phineas Gage to current research on executive function in the field of neuroergonomics

References:

ID 149 Theta Neurofeedback and Pilots’ Executive Functioning. Lafont, A., Enriquez-Geppert, S., Roy, R., Leloup, V., Dehais, F.
ID 164: Executive functions and flying performance in pilots. Smit, D., Enriquez-Geppert, S. Daneshnia, N., Lafont, A., Dehais, F.
Braver, T. S. (2012). The variable nature of cognitive control: a dual mechanisms framework. Trends in cognitive sciences, 16(2), 106-113
Dehais, F., Hodgetts, H. M., Causse, M., Behrend, J., Durantin, G., & Tremblay, S. (2019). Momentary lapse of control: A cognitive continuum approach to understanding and mitigating perseveration in human error. Neuroscience and Biobehavioral Reviews, 100, 252–262.
Enriquez-Geppert, S., Huster, R. J., Figge, C., & Herrmann, C. S. (2014). Self-regulation of frontal-midline theta facilitates memory updating and mental set shifting. Frontiers in behavioral neuroscience, 8, 420.
Harlow, J.M. (1868). Recovery from the passage of an iron bar though the head. Publications of the Massachusetts Medical Society, 2:327-47
Karr, J. E., Areshenkoff, C. N., Rast, P., Hofer, S. M., Iverson, G. L., & Garcia-Barrera, M. A. (2018). The unity and diversity of executive functions: A systematic review and re-analysis of latent variable studies. Psychological bulletin, 144(11), 1147.
Miyake, A., Friedman, N. P., Emerson, M. J., Witzki, A. H., Howerter, A., & Wager, T. D. (2000). The unity and diversity of executive functions and their contributions to complex “frontal lobe” tasks: A latent variable analysis. Cognitive Psychology, 41(1), 49-100.

Talk: Exoskeleton-supported stroke rehabilitation using embedded brain reading

Stroke rehabilitation has to address many factors that are impaired by brain lesion caused by brain hemorrhage. Patients show diverse symptoms reaching from movement disorders, impaired sensation and cognitive deficits, which must be addressed by therapy. To compensate movement disorders, exoskeletons can be applied (1-4). They can sense movements and support them to enable a patient with smallest remaining muscular activity to regain control over a disabled limb (Fig. 1). Electromyogram and force measurements can be used to adapt the support to the patients need (5). In case that no control of the limb is left, brain activity can be analyzed by embedded brain reading (eBR) to infer on the intention of the patient and to trigger movements implicitly (6,7). For a successful therapy not only “assist as needed” is required but further, patients must be able to follow instructions - mental stress must be avoided to assure optimal therapy conditions. Again, eBR can be used to detect A task load on a patient while he or she is performing an ongoing action, e.g., performing a specific arm movement to infer whether a patient (while performing the movement) is still able to understand instructions by the therapist or a serious game (8). Further, EEG signals can be used to adapt the behavior of the system to the subjective preferences of the patient. This talk will show new therapy approaches for upper limb rehabilitation made possible by means of an active exoskeleton with highly adaptive embedded control combined with movement intention recognition based on EEG, EMG and eye tracking data. In future our approach will be supported by task-load detection based on P300-related activity and ErrP-related activity evoked by subjective misbehavior of the system (9).

Figure 1: Stroke patient (2 weeks after stroke) with impairment of the right arm. A: movement support triggered by muscle activity. B and C: repetition accuracy of taught-in movements (B).

References:

(1) Platz T, Roschka S (2009) Rehabilitative Therapie bei Armparese nach Schlaganfall. Neurol Rehabil 15(2):81– 106
(2) Platz T (2011) Rehabilitative Therapie bei Armlähmungen nach einem Schlaganfall. S2-Leitlinie der Deutschen Gesellschaft für Neurorehabilitation. NeuroGeriatrie 3(4):104–116
(3) Nitschke J., Kuhn D, Fischer K, Röhl K (2014) Comparison of the usability of the ReWalk, Ekso and HAL. OrthOpädietechnik 9(14):22
(4) Kirchner, E. A., Will. N., Simnofske, M., Vaca Benitez, L. M., de Gea Fernández, J., Kampmann, P., Kirchner, F. (2019) Exoskelette und künstliche Intelligenz in der klinischen Rehabilitation. Editors: Mario A. Pfannstiel, Patrick
Da-Cruz, Harald Mehlich. In: Digitale Transformation von Dienstleistungen im Gesundheitswesen V, Springer Nature, chapter 21, pages 413-435, Aug/2019. ISBN: 978-3-658-23986-2.
(5) Kirchner, E. A., Albiez, J., Seeland, A., Jordan, J., Kirchner F. (2013), Towards Assistive Robotics for Home Rehabilitation, In Proceedings of the 6th International Conference on Biomedical Electronics and Devices, (BIODEVICES-13), 11.2.-14.2.2013, Barcelona, Feb/2013.
(6) Kirchner, E. A., Fairclough, S., Kirchner, F. (2019) Embedded Multimodal Interfaces in Robotics: Applications, Future Trends, and Societal Implications. Editors: S. Oviatt, B. Schuller, P. Cohen, D. Sonntag, G. Potamianos, A. Krueger. In: The Handbook of Multimodal-Multisensor Interfaces, Morgan & Claypool Publishers, volume 3, chapter 13, pages 523-576, 2019. ISBN: e-book: 978-1-97000-173-0, hardcover: 978-1-97000-175-4, paperback: 978-1-97000-172-3, ePub: 978-1-97000-174-7.
(7) Kirchner, E. A., Kim, S.-K., Straube, S., Seeland, A., Wöhrle, H., Krell, M.M., Tabie, M., Fahle M. (2013) On the Applicability of Brain Reading for Predictive Human-Machine Interfaces in Robotics. In PLoS ONE, Public Library of Science, volume 8, number 12, pages e81732, Dec/2013.
(8) Kirchner, E. A., Kim S.-K. (2018), Multi-Tasking and Choice of Training Data Influencing Parietal ERP Expression and Single-Trial Detection—Relevance for Neuroscience and Clinical Applications, In: Frontiers in Neuroscience, volume 12, pages 188, DOI: 10.3389/fnins.2018.00188
(9) Kim, S.-K., Kirchner, E. A., Stefes, A., Kirchner F. (2017). Intrinsic interactive reinforcement learning - Using error-related potentials for real world human-robot interaction. In Scientific Reports, Nature, volume 7: 17562, pages n.a., Dec/2017.

VRHack

VR-Hack Munich 2021

Hackathon Team:

Champion Sponsors

Sponsors

Supported by

Want to join us?

VR-Hack Munich 2021

Hackathon Team:​

Champion Sponsors

Sponsors

Supported by

Want to join us?

Talk:

Talk: BCI-assisted augmentation of group decision-making

Talk: Mobile EEG technology: portable, motion-tolerant and transparent?

Talk: Physiological Synchrony for monitoring attentional engagement in a group

Talk: Eye blink-related potentials. A valuable tool for neuro-ergonomics

Talk: Towards Theories of Mental Task-based Brain-Computer Interfaces User Training

Talk: A Neuroadaptive Gaming Prototype to Distract from Pain

Talk: NHANCE: Neurotechnology for Human AugmentatioN in Critical Environments

Talk: Mobile Brain/Body Imaging in Neuroergonomics

Talk: From Phineas Gage to current research on executive function in the field of neuroergonomics

References:

Talk: Harnessing embodied cognition to learn sign language in virtual reality

References:

Talk: Exoskeleton-supported stroke rehabilitation using embedded brain reading

References:

Talk: Can Neuromodulation Make Us Better: Changing Brain Activity with Wearable Brain Stimulation Devices

References:

Talk: Measuring mental workload and stress: a bioengineering perspective

References:

Talk: Is endowing robots with social signals always beneficial?

References:

Talk:

References:

Hackathon Team: