For developing my reading in VR app I wanted to prototype reactive environments. The goal was to test how manifesting sound as visuals in an immersive experience would feel. While possible to prototype physically, I knew the technology should be faster to execute in Unity and viewed in VR. I also had basic dialogs built from my course in VR UI Fundamentals which would integrate nicely for prototyping. Wanting to test as quickly as possible I found a Bad Racoons’ 3D Visualizer Spectrum Vu Meter on the Unity Asset store, since I knew it would work with VR.
Scene view of 3D visualizer in Unity
Experimenting with the visualizer I plugged in audio from Audibles’ recording of James Joyces’ A Portrait of an Artist as a Young Man, narrated by Collin Ferral. The particles move based on the intensity of the audio. If this was used when the user was reading, the experience would only move forward when the user spoke aloud.
Scene using remixed Cave Alley // Day 1 Creative Comons by Danny Bittman: https://poly.google.com/view/aMGO5kSJnMY
As a student I’m very thankful for creative commons assets. Danny Bittman has graciously shared his tilt brush scenes as CC, which I felt captured a mood I wanted to explore with the visualizer. Scenes can be imported directly into Unity using the Poly Toolkit. Bittman creates a tutorial series on YouTube for becoming a VR artist, and on this episode he explains how the Tiltbrush to Unity workflow works.
I’ve attached the visualizer as a child of the center camera in Oculus’ Integration SDK in Unity. This tracks the users gaze, similar to a class HUD in 2D screens.
Scene 1
Using Post-Processing in Unity, I’ve set up volumes to shift the hue and saturation as the user moves forward through the environment. I wanted to find a method of conveying mood of the text as the user progresses in the story, so I was quite happy with the simplicity of this prototype.
Scene 2
Testing non-linear camera paths proved tricky while testing in VR. Lateral rotations and heavy light flickering caused myself to experience heavy VR sickness. This is clearly an ethical concern for testing with other players. This inspired me to build comfort setting on a palette for your left controller. This allows turning off of audio reactivity, acceleration of user, environment itself for focus, and particle effects. There’s also quick toggles for focus or full immersion.
Audio used for testing is Collin Ferrals’ reading of James Joyces’ “A Portrait of an Artist as a Young Man” by Audible.
3D Visualizer Spectrum Vu Meter by Bad Racoon.
Scene 1 Environment: Cave Alley // Day 1 by Danny Bittman Creative Commons
Scene 2 Environment: Snow Melt by Danny Bittman Creative Commons