Flat World No More
Film school and making films taught us how to represent a non-flat reality on (or in?) a flat, carefully framed, rectangular surface. To that end, we learnt how to use lenses, camera angles, framing, movement of the camera and movement within the camera to control the viewable space. 360/VR/Immersive video presents a spherical representation of reality to the viewer and opens up the viewable space by giving the viewer greater agency to select the frame. As such, it calls on us to unlearn all those lessons learnt in film school and making films.
Can flat maps represent the earth as faithfully as spherical globes?
Map-making has long struggled with the problem of creating maps with limited knowledge of unexplored territories, the difficulties of surveying and charting terrain and most relevant to us – the impossibility of representing a spherical Earth in the rectangular frame of a map. Spheres and flat rectangles are not isomorphic and map-makers had to devise various means of representation for flattening out a sphere and representing it as a flat rectangle. One exercise for understanding the impossibility of this task is to place a balloon inside a cylinder and to blow it up until it is as tight as possible against the walls of the cylinder. No matter how tighly the balloon fits inside the cylinder, the top and botton will never touch the walls. Lets imagine that we had drawn a map of the Earth on the balloon and the ink imprinted itself on the walls of the cylinder upon touch. We can now unroll the cylinder and lay it down as a flat rectangle. The map of the Earth is now in the form of a rectangle. But there is significant distortion. The further we move away from the equator, the more the sphere has had to expand in order to touch the walls of the cylinder and transfer the ink. The polar regions, in fact, have never touched the cylinder walls and that is why they are always represented as running all along the top and bottom of a rectangular map.
For me, the analogy between traditional film and 360/VR/Immersive video is that of flat maps and spherical globes. The map lays out a geographical region in a precisely controlled, detailed, flat rectangular frame, just like traditional film. Whereas 360/VR/Immersive video presents the world like a globe, giving the user greater agency to spin, select their targets and move their heads to view. The viewer becomes an active participant, but what influence can the creator exert in order to guide user choices? How can narratives be created with guided participation?
While the similarities and differences between traditional film and 360/VR/Immersive video are numerous, for this particular journal I wanted to focus on the following three aspects:
- Movement within the frame
- Always in long shot/movement of the frame
- Mise en scene
1. Movement within the frame
360/VR/Immersive video surrounds the viewer and also provides them with agency to perform the functions of panning and tilting their head to direct their gaze and select the subjects from within the viewable area. However, in some 360/VR/Immersive it may be required to influence the viewer’s gaze to direct their attention. An onscreen narrator is one way to perform this task and a subject that stands out on their own is another way. But what are some of the implicit cues that can be used to target the viewer’s gaze? Can movement within the frame be used to direct the viewer towards an action that is occurring or is about to occur? The following two videos feature movement within the frame (cars and pigeons) and were shot to test the hypothesis that the viewer will pan and tilt in order to follow their self-selected subject when they are confronted with movement within the frame. Using the analogy of the map and the globe, how can we influence the viewer’s choices and make them spin the globe a particular way?
Pigeons @ Abdullah Shah Ghazi's mazaar – 360 Video
This is my first 360/VR/Immersive video so I was expecting it to be a complete disaster. Fortunately, the results turned out better than I expected.
I was visiting Karachi, Pakistan recently and knew I would be able to find some exotic sites for immersive videos. One of the sites on my list was the mazaar (shrine) of Abdullah Shah Ghazi – an eighth-century Muslim mystic and Sufi. His shrine is located close to the beachfront and overlooks the city of Karachi. It has always held a great fascination for me because it is a massive complex incorporating impressive aspects of Islamic architecture such as elegant domes, minarets and Sindhi tile-work.
This particular video was made before I even entered the shrine. I saw a flock of pigeons in the parking lot and thought it would be a great opportunity to test one of the questions I have about immersive video – how do we direct the viewer's gaze in the absence of an onscreen narrator? I thought the pigeons flying all around the viewable sphere of the immersive video would act as targets for the viewer's gaze thereby making them pan and tilt their head as they followed a pigeon or a flight of pigeons. I was using a Ricoh Theta V mounted on a small desktop tripod and placed the camera close to the edge of the main flock. The camera was soon surrounded by pigeons and I was glad to see that they were not intimidated by the contraption. This bodes well for future endeavours.
Having tested my hypothesis of directing the viewer's gaze with a limited sample (my wife, kids and a couple of friends) I can say that the birds did act as effective targets of the viewer's gaze and the viewer does experience the viewable sphere more fully due to the flight of the birds... how can this be effectively applied to immersive narratives? does a target have to be explicitly identified as a target or can implicit cues be used to direct the viewer's gaze?
Tariq Road (Liberty Chowk) – 360 Video
Still early in my 360/VR/Immersive journey I decided to place the camera on a traffic divider in the middle of oncoming traffic. This is a video of Tariq Road at Liberty Chowk. It is a popular shopping area and normally there is a lot of traffic (and many traffic violations) at this intersection. This particular night was a bit more quiet than usual. However, you will still see a fair bit of traffic.
Once again, the objective of this video was to influence the viewer to self-select subjects from within the frame and exercise their agency by panning and tilting their head. The viewer might want to follow a rickshaw, a car that made an illegal U-Turn or any other subjects in order to explore the viewable area.
Once again, the results were mixed, although I found the test subjects exploring the viewable space freely, I wasn’t able to find any definitive cues I could use for influencing them
2. Always in long shot/movement of the frame
Traditional film taught us the benefits of working with different lenses and moving the camera. The master shot discipline taught us how long, medium and close shots could be used to direct the viewer’s attention and direct the narrative. In a way, this was like looking at a map (or being shown a map, actually) where carefully framed layouts direct the viewer’s attention to a target. If you want to see Paris in the context of Europe, you could start with a map of Europe, then turn the page to Western Europe, then turn to France and then to Paris. However, 360/VR/Immersive video gives the viewer a globe, not an atlas and they spin it in order to see what they want. Can a globe provide the viewer the same context and level of detail that a map would provide? Can we influence how they spin the globe and (assuming a fixed lens) how can we give them the level of detail that they see in a map?
Frere Hall – 360 Video
This is a brief 360 video featuring Frere Hall – a colonial era building in Karachi, Pakistan. Frere Hall was built in 1865 as the town hall for Karachi. Since independence in 1947 it has served as a library, an exhibition space and and venue for artistic and cultural events.
There were a few challenges this video. Firstly, Frere Hall is a tall structure and I wasn't sure where to place the camera. I ended placing the camera about eight feet high more out of convenience than due to any artistic or technical considerations. As a result the viewer has to tilt their head up to see the top of the building. I guess this is a close approximation of reality.
Secondly, also with relation to camera placement, Frere Hall is a magnificent structure from every angle and once again, I chose an angle more out of convenience than artistic and technical considerations. I am still reluctant to move the camera for 360 video (for fear of disorienting the viewer) so I wasn't able to explore all sides of the building. The historical image that appears next to the current building actually shows it from a different angle, but with my reluctance to move the camera I couldn’t make the juxtaposition an exact match.
Christie Lake – 360 Video
This particular video was shot at Christie Lake in the early morning. The lake stretches no more than a 100 metres across at this point. With traditional film I would have placed the camera at the edge of the dock (since the boat rentals hadn’t opened yet, it was my only option) and shot across the lake with a longer lens (maybe a 150 mm). With 360/VR/Immersive video, my only option was to be in long shot.
Mise-en-scène literally translates as “staging” or “placing on stage”. In film, it refers to the visual theme or design of a frame in traditional film. That was the easy definition. In Film Art: An Introduction, David Bordwell and Kristin Thompson call mise-en-scène "everything that appears before the camera and its arrangement—composition, sets, props, actors, costumes, and lighting." I bet you can see where I am going with this. Traditional film, allowed us to create the frame and control it tightly. 360/VR/Immersive video opens up an all-encompassing frame for the viewer. Can traditional mise-en-scène principles still be applied?
The following two videos demonstrate my struggle with wanting to control the mise-en-scène. Having, found the perfect spot to view the fall colours of Spencer Gorge from Dundas Peak, I hid behind a rock while my camera captured its 360 footage. While the viewer sees Spencer Gorge (and as a bonus, a freight train) the way I intended, if they turn their head they will also see other hikers popping in to the frame.
Does a wider viewable space require greater control? Does that mean mise-en-scène has to be carefully crafted considering all possible views? Since the viewer is creating the frame for themselves at 24 fps, can this impossible task ever be achieved?
Final Thoughts: Maps or globes?
Camera movement in the form of pans, tilts, tracks and cranes is used to create some of the most elegant sequences in traditional cinema/film/video. With a tightly controlled rectangular frame the creator can selectively direct the viewer's attention and thereby create meaning. The aesthetic appeal of an immaculately crafted pan/tilt sequence such as the sequence from Fellini's 8 1/2 below is one of the charms of the viewer relinquishing control to the film-maker.
How can this masterful direction of the viewer's gaze be created in 360/VR/Immersive video. For me, the three aspects discussed in this reflection: movement within the frame, movement of the frame and mise-en-scène are the essential elements that must be crafted with precision when a narrative is being created in 360/VR/Immersive video. With greater viewer agency, the creator must cede some of the control previously enjoyed in traditional film-making. However, they must employ new techniques to influence the viewer's gaze. These new techniques will rely on visual and aural cues that will be explicit and implicit.
Holding a map vs. spinning a globe
For me, the enduring analogy between traditional cinema/film/video and 360/VR/Immersive video is that of the map and the globe. The map sacrifices accuracy for ease of use and customised representation. The globe represents the subjects more accurately, but can be harder to "read" and is definitely less portable. The map is selective and created with limited agency for the user. The globe allows the user greater agency, but sacrifices detail. 360/VR/Immersive video will not replace traditional cinema/film/video for the same reason that globes did not replace maps. Sometimes accuracy and agency are sacrificed for ease of use and detail.
Overall, the exercises, lessons and discussions in this course have been great at providing food for thought and motivation for experimenting with 360 video. From here on I plan to move onward in creating the techniques required for effective narration for 360/VR/Immersive video.