The Lost Bus: A Mixed Reality Concept

As XR headsets increase in popularity and become more ubiquitous in homes across the world, TV and film studios are becoming more interested in how existing traditional “flat screen” content can be adapted to a Virtual Reality or Mixed Reality environment.

This article documents my process for creating a mixed reality experience for the 2025 film The Lost Bus starring Matthew McConaughey and America Ferrera. Warning, spoilers ahead!

Narrative analysis and research

• Watched the entire movie several times and took notes throughout including: timestamps, scenes, emotional arcs, props, 3d elements.

• I mapped the film’s 3-act structure with the main story beats (potential immersive anchors) and overall emotional arc which gave me the idea of introducing a specific colour palette for each act via a passthrough colour filter (see image below).

• Conducted additional research including Frontline PBS documentary, news articles and online reports.

• Created storyboard mockups in Figma using an isometric view of the viewer’s physical environment.

Mapping Story Structure Against Immersive Elements. Image credit: Natalie Marinho.

My Narrative Design approach

Experience Goal

• This immersive XR experience is designed to be a narrative companion to the original film.

• The film remains the central focus. Immersive elements and environmental spatial design supports the overall narrative and does not distract or overshadow.

Core Story Themes

• “A new and ferocious kind of climate change-inflected wildfire” quote from Fire in Paradise, Frontline PBS documentary about the 2018 Camp fire in California.

• Bravery and resilience of everyday people who went above and beyond to help others.

• Explore what went wrong and factors that contributed to the scale of the tragedy.

Narrative Design Elements in This Design

• Immersive elements support narrative and highlight action, but used sparingly to maintain their impact.

• Start with simple effects. Layer elements thoughtfully and increase complexity as the story progresses.

• Extend 2D film elements beyond the screen frame: transform into 3D, coax into the viewer’s room and “bring them to life”.

• The movie often switches between “day and night” scenes, sometimes with a rapid cut. Rather than matching the passthrough colour filter to each scene (too jarring and reduces impact) 3 main colours are aligned with each story act.

• Translate film artefacts into immersive elements that are symbolic to characters, locations and events. e.g. The family photo wall represents connections between family members. In the event of catastrophic fires and emergencies, it is often the loss of irreplaceable photographs that people feel most intensely. They also emphasise the importance of family and memories over material possessions.

• I did not utilised user interactions for several reasons. The viewer is seated, making it difficult to reach distant content. Lack of viewer agency reflects onscreen characters’ situation.

• Deliberate decision not to show the bus’ movements and real time location on the map. Provides a sense of being lost and avoids giving away how close they are to escaping at the film’s climax.

• Provide an opportunity post-film for viewers to access behind the scenes immersive content, similar to DVD extras.

Storyboarding the Experience

Next step was to document the immersive experience. I created storyboards using Figma. Each board included

• 3D Visualisation of the viewer in their physical home environment

• Virtual 2D screen

• Film timestamp

• Short description of film events

• Immersive elements.

Sample storyboard for The Lost Bus, mixed reality experience. Image credit: Natalie Marinho.

Production notes and technical considerations

My previous experience working on XR projects including mixed reality for the Quest headset was invaluable when creating the narrative design for this concept. Once can come up with many ideas but without an understanding of the technical limitations, valuable time may be lost in ascertaining the feasibility of such ideas. Conversely, knowing the technical capabilities of mixed reality technologies allowed me to come up with creative ideas that supported the film’s overall narrative while being conscious of application optimisation.

• This immersive experience concept is designed for an Extended Reality (XR) headset i.e. Apple Vision Pro or Meta Quest 3.

• Immersive content includes XR elements such as passthrough filters (colour/noise/distortion), 3D models, 2D animations, 3D animations, video with alpha channel, particle effects, procedural generation and spatial audio.

• Viewer can choose to mute the “immersive layer” at any time during the experience via hand gesture triggering a subtle fade in/out effect.

• Conscious of overall application performance and application size by utilising particle effects, baked lighting, minimise 3D model tri count, and lightweight shaders.

• Repurpose props from the film as immersive elements. Either as 2D images or 3D models (low polys created from scratch or photogrammetry scan).

• There doesn’t have to be an immersive element for every scene in the film. It’s ok to let the room “breathe” especially as the overall film has increasing tension throughout.

• Too much activity can distract or become overwhelming. Elements should fade slowly before the next scene appears.

• Focused on visual elements but would love to collaborate with sound designers on potential for existing film audio to be integrated into experience as spatial audio.

This was such a fun project to sink my teeth into and I’d love the opportunity to do more. Especially with a technical team who are keen to push the boundaries of what is possible including audience interaction and participation. So do feel free to get in touch at hey@nataliemarinho.com