VR programming isn’t as hard as you think
Coming from a web development background, I initially assumed building a virtual reality experience would be a huge, and probably impossible, challenge for me to do. I knew nothing about VR and assumed I would be crunching complex algorithms to get this stuff to work.
When researching how to create a VR experience I first came across a framework called React 360. “Perfect!”, I thought. As the name implies it’s a VR framework built on top of React Native. Being a React developer I thought this was the ideal entry point. However, the library was shelved in 2019. Although still available to use, it’s creaky now, and has a habit of blowing up whenever you install anything via npm. Because of its age it doesn’t support Hooks and Context. Using it now is painful and frustrating to even get started with. I gave up after a couple of days.
As I was researching an alternative I kept reading about a library called a-frame. I bought myself a udemy course and gave it a try. For anyone dipping their toes in the water with VR I would highly rate this course by Danilo Pasquarillio. It’s accessible and well-paced and is really the perfect entry point into WebVR.
A-frame is an extremely powerful and accessible library built on html, and its documentation is first class. It uses an entity-component framework, and is built on top of three.js, exposing all of its underlying functionality. The documentation sums up this system very nicely;
“a component is a reusable and modular chunk of data that we plug into an entity to add appearance, behavior, and/or functionality. In A-Frame, components modify entities which are 3D objects in the scene.”
The thrill for me with using WebVR was the ability to build in a 3d virtual environment for the first time. Having been confined to 2d layouts in web development, it’s a lot of fun experimenting within a new spatial dimension.
You can do a lot of fun things with this library. You can drop 3d models into your scene and they appear like magic. The recommended and easiest way to do this is by using models in the glTF file format. Many free glTF 3d models can be found at Sketchfab, which is the best place to find these on the web.
It is recommended that simple 3d models are used as ones that are too complex can take forever and a day to load in a web browser. If your 3d model has thousands of faces and vertices it will probably take 20 seconds or longer to load. To compensate for this drawback you can use techniques like normal mapping and stereoscopic images to create a sense of 3d realism and depth.
A good technique for creating a realistic look and feel for your models whilst loading only simple 3d shapes is the use of normal-mapping, which allows you to add textures to any 3d object. Textures.com is the best place to find these. Simply create an account and you can download 15 images a day for free. For any given texture download the albedo image and the normal map to be combined in your scene over a surface. The way in which the light reflects off this surface will give it the illusion of 3d, and you can adjust the way that light bounces off the surface using the roughness attribute.
Stereoscopic images is the technique of presenting two different images, one displayed to your left eye and a different one displayed to your right eye, which when combined in the brain create depth perception. This is how classic 3d movies/glasses work; two overlapping scenes are played on the screen in different colours, red and blue, and the viewer wears a pair of glasses, with a red lense in one eye and blue in the other, filtering what’s on screen so each eye only recieves one of the images being played. These images then combine in our brains and create the illusion of depth. As different images must be displayed to each eye, this technique only works when wearing a VR headset. You can set a default image for when the user is not using a headset.
To interact with your environment you can either use gaze-based interactions, where you can set a cursor in the middle of the screen and set a timeout to trigger an event like a click when hovering over an object in the environment. This is my preferred method as it keeps the rules of interaction simple and consistent for the user, and is the method I have used in the this test project. Hover the cursor over the tree to the left of the scene and it will turn blue and begin playing music…
If you move away from the tree you will also see positional audio at work; the music will lessen in volume the further away you move from the object.
As you can see in this project you can walk through objects in the environment, which is how this library works by default. You can however use a navigation mesh to prevent this behaviour, which is a glTF shape that is wrapped around your scene and the objects within it, and confines the user to only move around this limited space, thereby blocking the ability to walk through walls.
You can also use the aframe-physics-system to set the mass of objects, the gravity in your environment, and collision behaviour between objects, as demonstrated in the test project when the ball hits the ground and bounces, rather than falling through the plane as would be the default behaviour.
As an alternative to gaze-based interactions you may choose to use the super-hands-component, which allows you to click and grab, drag and drop objects in your environment. Personally I like to give the user the full ability to explore the scene, and so like to keep the hand cursor free for rotating the camera.
This library really is a pleasure to work with and can do so much more than is listed in this short blog post. Take a look at these official example projects to get more of a sense of what a-frame can do.
