To evaluate the potential of the EduExo Pro as an educational and research platform, Auxivo, in collaboration with the University of Applied Sciences of the Grisons (FHGR), launched an innovation project. The goal was to develop a prototype for an exoskeleton-based virtual reality (VR) gaming experience, exploring how exoskeletons could function as control and haptic interfaces in video games.
Over the course of two semesters, four students, supervised by Prof. Christian Bermes, worked on building a fully functional prototype. The project was divided into two phases. Initially, three engineering students developed the core setup, integrating the EduExo Pro into the system and establishing the game architecture. Later, a single Bachelor student expanded the project by designing a more complex game and incorporating VR-display integration for an immersive experience.
The final prototype integrates two EduExo Pros alongside additional peripherals and display elements, forming an exoskeleton-based arcade machine. The result is a system that offers an interactive gaming experience with novel control mechanisms and haptic feedback.
Hardware Setup
The setup incorporates several essential components:
-
Two EduExo Pro devices for tracking arm movements and providing haptic feedback.
-
A racing pedal set for additional foot-based input.
-
Two handheld push buttons integrated into 3D-printed handles, connected to open interfaces on the EduExo Pro.
-
A PC to run the games.
-
A screen to display the games.
-
An adjustable aluminum frame to support and connect the components, allowing modifications for different users.
-
Optional VR display: The Meta Quest 3 headset was used for an enhanced virtual experience.
A critical aspect of this setup was developing an interface to connect the exoskeletons with the PC game. This interface facilitated bidirectional communication between the exoskeletons, game environment, and integrated sensors.
A modified EduExo Pro control software allowed the exoskeletons to receive external motor control inputs. This feature enabled haptic feedback, making the virtual interactions more realistic by providing tactile sensations.
To capture detailed arm movements, an additional angle sensor (potentiometer) was installed in the exoskeleton’s shoulder joint, allowing precise tracking of shoulder and elbow movements.
Although not implemented in the prototype, additional sensors from the EduExo Pro, such as an IMU motion sensor and EMG muscle activity sensors, could further enhance the experience. These sensors could enable direct muscle-driven control, introducing a deeper level of immersion.
Game Development
The team chose Unreal Engine 5 for game development due to its advanced capabilities and adaptability. Since the EduExo handbook tutorial primarily focuses on Unity 3D, this project also aimed to test the compatibility of the EduExo Pro hardware with Unreal Engine.
The game worlds and levels were designed using the Unreal Engine editor, with additional assets sourced from the Unreal Marketplace to streamline development and enhance visual appeal.
First Game: Exoskeleton-Based Beer Pong
The first game was a Beer Pong-style experience where players controlled a virtual arm using the exoskeleton to throw balls into cups. This prototype served as a proof of concept, primarily testing the technical integration of the EduExo Pro with an Unreal Engine-built game. While interaction and immersion were limited, the game successfully demonstrated the feasibility of using exoskeletons as game controllers.
Second Game: Sci-Fi Exoskeleton Exploration
The second game featured a more complex and immersive experience. Players controlled a human avatar equipped with an exoskeleton, which included a gripper module on the left arm for picking up objects and a gun module on the right arm for shooting targets.
The game was set in a jungle environment, inspired by the Avatar movies. Players used the exoskeletons to mirror their real-world arm movements in the game, enhancing realism.
Handheld push buttons allowed players to activate the gripper or fire the gun, and haptic feedback simulated the recoil effect when shooting. The game also integrated a racing pedal system to control movement, adding another layer of engagement. Players earned points by shooting targets or transporting parcels between designated locations, with time-based scoring to introduce an additional challenge.
VR Display Integration
To maximize immersion, the project integrated a Meta Quest 3 VR headset to display the game world in virtual reality. This allowed for a fully immersive experience, where players could interact naturally with the game using the exoskeletons.
The Meta Quest 3 controllers also replaced the custom handheld push buttons and racing pedals, simplifying the control scheme while maintaining full functionality. This upgrade streamlined gameplay and improved the overall experience.
Summary and Conclusion
Project Nexus was a groundbreaking experiment that allowed the FHGR student team and Auxivo to explore innovative ideas in exoskeleton-based gaming. The prototype demonstrated that exoskeletons could serve as intuitive and engaging control interfaces in VR gaming.
Many aspects of the project worked exceptionally well, including the immersive gameplay and effective haptic feedback. However, there remains significant potential for future developments, particularly in integrating additional sensors such as the IMU and EMG, which could enable muscle-driven control mechanisms.
One key design consideration is the absence of leg exoskeletons, which limits natural movement. As a result, the most immersive game concepts are likely cockpit-based experiences, where players control robotic arms while seated. Future game ideas could include:
-
Operating a wheeled robot for planetary cleanup missions.
-
Piloting a submarine for deep-sea exploration.
-
Collecting pirate treasure using robotic arms.
-
Flying a spaceship to harvest asteroids.
For researchers and students interested in similar projects, the EduExo Pro provides an excellent starting point, offering a flexible and adaptable hardware platform for developing innovative exoskeleton-based applications.
This project was purely experimental and not a commercially available product. The hardware setup described here is not for sale, and no additional technical documentation beyond what is shared in this article is available. This initiative was conducted by students as a research project, and we share it in the hope of inspiring others to explore similar ideas.
