Open the door to another (virtual) reality
Between the Worlds revolves around using Meta Quest camera passthrough to create a vast landscape hidden behind a regular, real world doorway. The player is given 3 gems, which swap the world behind the doorway. The player must recover artifacts from each of these locations to assist the ghostly remains of a knight in a kingdom long forgotten.
The first step of this project was testing if we could do directional occlusion for the doorway (Facing the back side of the door, you see a whole virtual landscape, from the other side, you see the world peaking through the doorway). When prototyped with greybox assets, the effect seemed to work. A virtual doorway was then modeled to the exact dimensions of the real door, and a calibration tool was built extending MRTK's manipulators to allow the user to adjust the position and rotation of the virtual door to match the real door.
At this point, heavy iteration was made on attempting to "track" the virtual door. Due to limitations of the Meta Quest, we had no way to tell if the real door was open or closed. The solution was to track the player's hand when they reach for the real door. Many special edge cases were accounted for due to the different ways I observed players opening and closing the door (ie, pushing the edge closed with the palm of their hand, rather than using the door knob). Desyncs of a few degrees of rotation often occured due to the tolerances for what was considered "attached" to the door handle, but these rarely proved noticable to players.
A few weeks after this project was completed, the Meta Quest Pro was released! Because our project made such heavy use of the Quest passthrough technology, it was a natural fit for the new platform. Porting the project over resulted in a much more powerful experience, as the depth of the reconstructed passthrough view made the illusion of a world beyond the door much more powerful.
This project was created with Unity using the Oculus-Unity Integration Plugin, targeting the Meta Quest 2 and Pro.
I handled systems programming and internal tool development.
Environment art by Shuozhi Shen .
Character art by Jiaxin Li.
Voice acting, audio, and music provided by Penghau Zhou.
Gameplay programming and additional VFX by Jacky Sun.