Post-Processing volumes in Unity
Rebound: Demystifying Literature in Virtual Reality
Interaction Design 2021 // Grad Project
Emily Carr University of Art & Design
Faculty Advisors: Haig Armen, Eugenia Bertulis & Daniel Wildberger
Concept Video
Rebound
An experimental adaptation of James Joyces’ A Portrait of the Artist as a Young Man. Designed from the ground up for Virtual Reality. Explore generative visuals in a techno-abstract world. Experience a classic that defined literature for the 20th century.
Problem Space
Finding Focus for Reading
Reading has become more important than ever for many since COVID-19. Finding the focus can be difficult through changes in our home lives and mobility. Virtual Reality offers an opportunity to rethink reading as an immersive experience.
The Infamy of Literature
Classic literature can be overwhelming to read. Often requiring large amount of context from outside of the book. Dated language and long text are common barriers to the average reader.
James Joyce
Joyce was born 1882 in Ireland and became one of the key writers of the modernist avante-garde movement of the early 20th Century. Best know for his complex, symbolic and lyrical writing. His writing is also notorious for being difficult to read.
Opportunity Space
These are affordances that Virtual Reality provides to rethink and adapt classic literature for our contemporary times.
Co-Creation Workshops
Generative Meaning
Collaborative Machine Learning
Who would Joyce be in the 21st Century? Stephen Daedalus, our protagonist is heavily based on Joyces’ own life. The story follows him from a child into a young adult. Through found photographs and an array of young Irish men and women, we can visualize a modern Joyce. This is powered by styleGAN collaborative machine learning (Artbreeder).
Stream of Conscious Landscapes
Every reader has their own mental image of a books world. Stephen Daedalus’ crisis and epiphanies are from within his own mind. Rebound amplifies Joyces’ stream of conscious style with fluid AI generated landscapes. Sweeping the reader between sections and chapters in parallel to the books’ dream-like prose.
Liminal Symbolism
Beauty can be found in the absence of comprehension. Embracing the bewilderment of Joyces’ prose, visuals are generated through a series of symbolic references found within the text. These are bred together through machine learning to create abstract yet familiar images, a liminal state of understanding.
Unity Development
Audio Reactive
Elements of the 3D environment react to audio input. Natural pauses in the reading create a visual rhythm. Intensity is expressed through particles and flares when channel thresholds are hit.
Immersive Reading
Danny Bittmans’ Tilt Brush paintings are used for abstract environments while reading. Post-Processing volumes shift the ambient mood as the user moves through a location. Hybrid experience of reading and listening.
Insight
Drifting through the wild while listening to an audiobook was effective in transporting the reader and creating a meditative state. Abstract environments devoid of symbol or language did not distract from audiobook listening.
Reflection
VR Product Design
This is only the beginning of understanding literature and new media. Joyces’ experimental writing provided a structure and reference point to create with an array of tools. Each tool provided it’s own quirks which I incorporated into the project. Thinking through making, similar to Joyces’ own stream of conscious style, allowed the project to be amorphous and wildly transformed through iterations. Delegating labor to machine learning allowed me to creative direct asset creation, independent of a support team. Design language for VR is still forming and currently embedded in complex tools like Unity. I’m excited to continue developing new workflows, tools, and communication strategies for XR applications.
Created with:
- Audio by Audible, Colin Ferrall narrating James Joyces’ “A Portrait of the Artists and a Young Man”
- Video from Pexel: Taryn Elliot, Cottonbro, Zen Chung, Nothing Ahead
- Unity: Oculus Integration SDK, 3D Visualizer Spectrum Vu Meter by Bad Racoon
- Poly Toolkit: Cave Alley // Day 1 (CC-BY) & Snow Melt (CC-BY) by Danny Bittman
- Machine Learning: Artbreeder