JARVIS: Ubiquitous Mixed Reality Fitness Platform

  • Ubiquitous Mixed Reality Screenshot of JARVIS Fitness App

This project envisions genuinely ubiquitous mixed reality experiences. The next generation of mixed reality devices needs to incorporate the capability to closely interact with objects in a user’s surrounding environment, as proposed in Keiichi Matsuda’s concepts of Hyper Reality. a potential approach to enable such capability, we propose to leverage the Internet of Things (IoT) technologies that integrate sensors and connectivity into everyday objects such as toys, home appliances, and shopping carts. These sensors capture a user’s interaction with the objects in her vicinity, expanding the capability of mixed reality platforms beyond tracking users’ head and whole-body movements.

We describe the challenges of realizing the vision of ubiquitous mixed reality, and then present our ongoing work on developing a virtual fitness coach and its supporting platform as a concrete example of ubiquitous mixed reality technology.

This work is in press for the quarterly ACM SIGMOBILE Mobile Computing and Communications Review (GetMobile) and under major revision for Proceedings of the ACM on
Interactive, Mobile, Wearable and Ubiquitous Technologies (IMWUT). Find the current draft of our vision paper here.

  • Date
    • 2015-Current
  • Role
    • Project Co-Lead, Frontend VR/AR Application Design and Development, User Experience Research
  • In collaboration with Mi Zhang (MSU), Youngki Lee (Singapore Management University).