Exploring potential design ideas for mixed reality and sharing with your team
Last month I kicked off a new project - a mixed reality video game for the Quest 3. I have spent a lot of time considering the narrative and world building aspects to assist with the prototype phase. I like to employ three specific techniques during this early stage.
Sketching
Using good old paper and pen, I create many sketches of potential mixed reality and hand interactions. I jot these in sketchpads, notebooks, post its, back of receipts - whenever and wherever I get inspiration! Then I set aside some time to review and collate these together in something like Figma or Google Slides, depending on how I need to share these with my team.
Rapid sketching of potential ideas
Sometimes I am tempted to redraw sketches (I am not a great illustrator) or to use AI to clean them up. But honestly I think the rougher the better as its just the suggestion of an idea or concept. It leaves something to the imagination. I liken it to reading a book that has an image of the protagonist on the cover (which I don’t like) because I would rather just get the idea of the character and let my imagination conjure up that image in my mind’s eye. In the same way, I want to present my team members with a core idea and let them use their own expertise to riff on that.
With high fidelity or refined sketches there’s also the potential problem that the viewer may fixate on something that isn’t necessarily relevant so early in development. You don’t want someone to be distracted by the colour of an object or put off by the style of the drawing. Those of you who are familiar with UX or digital design will be familiar with the process of creating wireframes which block content and interactions leaving later mockups that will feature more detail such as font, colour and specific images. For now, the focus should be on whether an idea could work or has potential.
Storyboarding
I find its useful to take some of these sketches and create storyboards. Not necessarily to create a linear narrative but to describe the before and after of a certain user interaction and quickly experiment with different potential environments or room layouts.
Storyboard to explore mixed reality interaction
It helps me consider what might be a potential issue or challenge and what the user may need to guide them. Sometimes this process triggers questions for other team members - potentially the software developer or technical artist, so I’ll make note of those too.
It’s also useful in identifying commonalities between potential scenarios (Is the user standing or sitting? Are they constantly looking up or down?) and what might be required later for testing sessions (furniture or props).
Again, I try to keep these quick and rough rather than spend too much time on how “nice'‘ they look. From personal experience I know that if I start to use digital tools to create these storyboards I will spend WAY too much time making them look good and start fussing over aligning elements, font size, the “perfect” prop image etc
Common reference points = common language
What both of these tools provide is a solid starting point for discussions between team members. I have experienced so many times where one team member will make an assumption that is very different to someone else. But why should it be the same? We’re all bringing our own experiences, bias, skills and perspectives so it’s actually rare that we would all have exactly the same approach.
Sketches and storyboards can help articulate a concept to someone else – which is especially important when we’re talking about spatial experiences.
Other team members can also help point out the obvious – for which you may be completely oblivious. For example, why is the user facing in a particular direction? Or that placing a virtual table in a specific location may occlude another object. Team members can also question any technical assumptions that are already inherent in your design.
The art of working with others is finding a common language or understanding. And whatever tools allow you to do this as quickly and simply as possible are key at the outset of a new project.