Spatial Telepresence Protoypes
Roles: Concept Development, UX design, prototyping, user testing
Technologies: React.js, webRTC (Daily API)
I created these prototypes while working at Bluecadet. Some of the prototypes described here were incorporated into an article about virtual events produced by a team a Bluecadet and discussed in this Fast Company article
Where are you in a Zoom meeting?
A few months into the pandemic, I started thinking about why platforms like Zoom work well in certain contexts--like business meetings--but struggle in others. Happy hours and game nights feel awkward (especially with more than 10 or so participants), and online conferences and workshops can be frustrating. Zoom and Google Meet are suddenly being called upon to support a whole spectrum of human interactions, which they weren't really designed to facilitate.
As I considered what was missing, I realized how important spatial relationships are for in-person interactions. I actually drew a short comic dramatizing the situation (click to enlarge the images):
Reintroducing Space
I decided to prototype a new type of telepresence application which included a spatial metaphor. First, I sketched a simple interface: dots on a virtual 2d space represent participants on a call. Each participant can move their dot using a keyboard, mouse, or touch input. Participants can only see and hear other participants whose dots are close by.
Prototyping with Code
I quickly realized that thinking about these experiences and making sketches would only get me so far: I needed to create some interactive live prototypes. Fortunately, some colleagues at Bluecadet introduced me to Daily, an API for creating custom web-based video chat applications. I'm not a web developer, but have enough experience with javascript and React.js to make a quick and dirty prototype.
I created an initial prototype where audio volume and video size vary continuously with distance. Since I was living by myself, my first tests involved running the app on devices placed in different locations throughout my house, with different music playing near each device.
For the next iteration I refined the UI to support a larger number of participants, which I tested out with a number of my colleagues:
I was gratified to see people using the app in ways I hadn’t anticipated: for example, a few games of ‘tag’ broke out, and another colleague discovered a way to ‘hide’ by taking his avatar offscreen. A particularly fun moment happened when a colleague joined the meeting a little late, and we all went to ‘swarm’ him:
Before sharing the prototype with a wider audience, my colleague Janet Lu created a much more polished interface:
Looking Ahead
Once you have the idea of a shared virtual space, lots of ideas for activities and interactions present themselves. Here are a few sketches:
I worked with some colleagues at Bluecadet to produce