AR Recreation of Uberti's Abitare
Lighting is an essential quality for augmented reality. It is the unconscious perception and understanding of how light behaves that allows AR to trick the human brain into believing that a virtual object is really present in the real world.
AR Designers are cognisant of these “tricks” and seek to exploit these when creating an immersive AR experience. For example, AR objects can have shadows, reflections, shading and specular highlights. These properties make it appear as though the AR object is obeying the laws of physical reality. Thus, real.
Preferably these properties are consistent with the lighting that is present in the surrounding real world environment. This is possible through lighting detection and estimation. But even without this type of real time lighting analysis, static lighting elements can still add a significant sense of realism.
Lighting can also evoke a mood, atmosphere, place and story as demonstrated by visual artists who work specifically with light as a medium.
Uberti: Artificial light illuminating space and place
Massimo Uberti is an Italian visual artist known for his large-scale installations featuring artificial lighting. Specifically, fluorescent neon tubes and transformers. These are typically site-specific, designed for a particular room or outdoor location. Sometimes his work creates a sense of place or story in an otherwise anonymous room as seen in his 2015 commission Drawing of Drawing for British motor company Bentley.
Image from www.massimouberti.it
Other work by Uberti emphasises the surrounding environment. This 2019 work is called Fulgida (Refulgent) located at the Castle of Brescia, created for the CidneOn International Light Festival.
Another work is the mesmerising 2018 Citta Ideale (Ideal City) at the Doge’s Palace in Venice.
Image from www.massimouberti.it
Uberti has worked with this medium for nearly three decades. Constructing spaces and objects, exploring light and shadow. What has been his inspiration? Uberti has said: “I like to create architectures of light. I employ neon tubes to build places for poetical inhabitants, trying to create dream-like spaces that allow for reflection.”
I think Uberti’s work is very beautiful. It’s ethereal and transient, while also projecting strength through geometric forms. There are also many interesting interplays. For example, between the work and its surrounding environment.
In Abitare, the room can only be seen by the light emmitted by that work. Citta Ideale illuminates the palace in defined ways - everything else is plunged into darkness.
There is also a relationship between the work and the observer. You can only perceive an object by the light it emits or reflects and when that light reaches your eyes. If a room is in complete darkness, we can’t see any of the objects contained therein.
I wanted to explore some of these aspects by recreating one of Uberti’s works into augmented reality. But first, lets take a look at how lighting works in AR.
How an augmented reality scene is illuminated
Loooking specifically at projects created with the Unity platform, a virtual object can be illuminated with direct lighting (point, directional or spot), indirect lighting (reflecting from other surfaces) or emitting light itself.
What about lighting that responds to the real-world environment in real-time? ARCore has a lighting estimation API while ARKit has the ARLightEstimate class. Both analyse a given image for discrete visual cues and provides detailed information about the lighting in a given scene. You can then use this information when rendering virtual objects to light them under the same conditions as the scene they're placed in. The virtual object appears to be subject to the same lghting conditions as real objects in the same scene.
From a performance point of view, it is also important to consider whether lighting calculations are performed at runtime or in advance (baked lighting) or a combination (mixed lighting).
The laws of AR object lghting
AR objects can
Cast shadows on virtual objects.
Cast shadows on real objects (floor or wall).
Emit light that is reflected off other virtual objects.
Respond to changes in ambient lighting in the surrounding environment.
AR objects can’t
Emit light that is reflected off a real object.
Cast a reflection on the surface of a nearby real object.
Project: Abitare in AR
In this project, I recreated Uberti’s 1999 work Abitare in augmented reality.
Image from www.massimouberti.it
Abitare is the Italian word for “to live”. According to Google Translate it is a verb similar to dwell or to inhabit. This structure represents an action: this is something to live in. Interestingly this is not a site-specific work as it has been installed at various locations. It suggests that the notion of “something to live in” could be mobile and yet remain entirely intact.
The work appears to be installed only within other buildings or structures that have an interesting lighting environment in themselves (an assumption based on the images displayed on Uberti’s website). This creates some very interesting play with reflections in glass walls and reflective floor surfaces.
Image from www.massimouberti.it
Image from www.massimouberti.it
I recreated Uberti’s work as a 3D asset using Unity 2019.3.12f1. Cylindrical game objects were selected for the structural elements to replicate fluorescent neon tubes. The original dimensions of the physical structure were 200 x 200 x 220cm. The Unity project replicated this scale.
I then created a material that resembled blue neon style lighting by playing around with albedo, emission, specularity and reflection properties. Although the structure now emits light this only makes the tubes brighter, it will not actually cast light on to real objects in the surrounding envioronment.
I used ARFoundations 3.1.3 for AR plane detection and tracking. Horizontal plane detection was the only mode enabled as vertical detection was not required. Detected planes are indicated within the AR Scene by a yellow transparent panel. After building to an Android device I was then able to place the work in various outdoor locations.
Exploring how location adds context to a static work
The AR version of Abitare can be placed literally anywhere, allowing the work to be viewed from dfferent perspectives depending on physical location.
Due to current Covid-19 isolation measures, there are few accessible indoor locations close to where I live. A nearby car park was one of my options and turned out to be quite thought provoking.
The Abitare structure implies home and placing it within a carpark introduced another narrative. A carpark is a place that people can inhabit, in cars or within recessed doorways of adjacent buildings. And like this AR structure, those places are mobile and transient. The original installation and AR recreation offer no real protection from the elements – there are no walls, floor or roof. It is just the basic idea of a habitat. In an empty carpark, it feels like a mirage.
I also noted that the AR version can be viewed in broad daylight while the original work was generally placed in dark unlit rooms or viewed outdoors at night with little lighting. The AR version can also be viewed at night, although the plane detection may be unstable.
(A side by side comparison of Uberti’s original work and the AR versions below.)
But the most important difference between the original physical work and the AR version, is that the original emits light that is reflected off surrounding objects and surfaces. The real Abitare physically changes the world around it and therin lies its true strength.
Looking at the images below of Tendente Infinito 2008 (which reminds me of the forrest illumination in Alex Garland’s recent series Devs) - it would be impossible for these spaces to exist without the light emitted by Uberti’s work. They are lit in a very specific way, unique to that exact location, dramatically different from their daytime counterpart.
Uberti’s Ellisse 2018 (below) magically illuminates the ceiling of Ducal Palace in Mantua. Walking that corridor by day, how many people would have stopped to ponder and marvel at the intricate artwork that hung above?
Image from www.massimouberti.it
This project has revealed many points of difference between an original physical artwork and the recreation of a digital/virtual AR version, offering both opportunities and shortcomings.
Differences and similarities between original work and AR recreation
Changing properties
Once constructed, it is very difficult to change the original artwork without significant cost or effort. But altering the properties of a virtual AR object is simple. One can changes its scale, colour, and dimensions. These can be controlled by audience interactions work via the app on an external device. Although an artist may wish to create a work and leave it intact, the ability to change properties makes AR a useful prototpying and design tool. And provides an opportunity to explore a dynamic and changing work.
No physical constraints
Unlike the real installation, moving the AR version of Abitare is simple. There are no structural elements to pack and unpack. One does not need to ensure the structure is carefully handled and transported. Instructions on how to build the construction are not necessary. No powerpoints, cables or power boards are required.
Digtal constraints
Digital art work faces constrants that physical work does not.
Changing file formats and the ability to read those formats.
Reliance on specific software and hardware confgurations.
Accidental deletion.
In this respect, a physical art work stored in a box within a storage facility can be more permanant and long lasting for future generations to enjoy.
Original versus copies
Virtual art installations present an interesting situation for artists. Most art works are originals. There is only one unique instance which must be viewed in person.
Virtual objects on the other hand can be copied, are easily distributed and could be viewed by any number of people simultaneously. Of course, if you introduced blockchain technology you could have an “orginal digital” work. But that’s a topic for another blog post.
REFERENCES
Massimo Uberti. https://www.massimouberti.it/
ARLightEstimate, Apple. https://developer.apple.com/documentation/arkit/arlightestimate
2020, April 29. Lighting, Unity Manual. https://docs.unity3d.com/Manual/LightingOverview.html
2020, March 20. Using ARCore to light models in a scene. https://developers.google.com/ar/develop/java/light-estimation
2017, December 11. Real-Time Light Estimaton with Google ARCore. https://www.andreasjakl.com/real-time-light-estimation-with-google-arcore/