This document has been complied to help students participating on the Erasmus+ co-funded programme EMEX (Emerging Media Exploration). Grant number: 2018-DE01-KA203-004282.
This guide was created because by September 2020, it was looking increasingly likely that the EMEX Autumn workshop 2020 was only going to exist in an online space. There was to be no student mobility as planned. This would disadvantage students at institutions that did not have access to a studio or volume for the capturing and live compositing for virtual production because they would not have an opportunity to participate or learn from the requirements and processes of the studio.
Therefore, the guide (below) was created which might have allowed students to at least experiment with the process and live compositing either that their own institution or even in their accommodation if they were confined under government lockdown orders.
SETTING UP A BUDGET VIRTUAL PRODUCTION STUDIO
This guide details the various studio configurations and their associated budgets when trying to create a studio for hybrid-live green screen virtual productions. If you are unsure about the different types of Virtual Production, check out Epic’s Virtual production Field Guide: https://www.unrealengine.com/en-US/blog/virtual-production-field-guide-a-new-resource-for-filmmakers
The reason I have taken the trouble to create this guide (along with the accompanying video series) is, whilst there is information out specifically for Virtual Production out there, the advice is either coming from sources who own/have access to studio kit already or those coming from a complete budget setup. There does not seem to be anything that starts from the lowest of budget setups and then looks at the options in an attempt to create something that starts to resemble a professional studio.
Obviously, the more and higher quality equipment you have, the better quality the production values of your project have the potential to become. However, no amount of expensive equipment will improve your work if the concept is derivative, the composition is uncompelling and the creative vision/direction is sub-par.
There is never any substitute for talent and/or care and attention paid throughout your production. You are essentially communicating a message. You can create a greater impact with poor production values but powerful content than you can with the best studio and a vague, poorly communicated message.
This guide however, concentrates on just the studio itself, and what different types of setup might cost to establish*.
We’ll start with a setup that you could achieve for free**.
* All prices quoted are rounded-up to the nearest whole Great British Pound and were correct at the time of publication. These prices will fluctuate drastically and some products might have been withdrawn.
**Free, assuming you have the items already. A computer that is capable of running Unreal Engine 4.22+ is a given, but there’s a good chance you have got the other items too.
The prices were correct as of September 2020. It is doubtful that prices will have remained the same. They are included here for evidence and achieving purposes as they are pertinent to the content of this document.
One final disclaimer before we explore the options and configuration of different studio setups, and that is I am not a professional cinematographer, photographer, light technician, gaffer or filmmaker. This guide lists options I have either researched for their suitability for contributing to a virtual production studio or purchased myself (based on this research). I have tried to find the cheapest options except where it explicitly states why a more expensive option was selected. I will assume no responsibility if you choose to take any advice listed below. The information is meant for guidance only.
Setup 1 - The Bare Bones
Yes, even with just a computer, a webcam and the Unreal Engine you can create virtual productions. Add some green/blue paper to the mix and you can even practice your chroma key removal in real time. Simply acquiring a video feed and keying the green/blue screen out is at the heart of pretty much every virtual production. So you really can get started with very little investment.
Of course, your webcam is not capable of capturing a very crisp image from more than a metre or so away and it is unlikely to focus well without lots of light. But, while you are practicing, this can be all you need to acquire a live video veed and process the image within Composure (Unreal Engine’s compositing suite).
If you do not have any green/blue paper, that is ok. You could try with white paper (although getting an accurate key will be difficult) or just settle for bringing a live video feed in.
If you do not have a webcam then you can always practice by referencing video files on your computer. If you have video footage with talent/objects in front of a green/blue screen, you can use those and still practice keying in real-time!
Whilst you might not have a webcam, you might have access to a DSLR (or similar) camera - which is even better! This is discussed in the next setup...
Setup 2 - Home Studio (minimal entry level equipment)
So £440 is hardly a budget option right? Well, that assumes you haven’t got a DSLR (or similar) camera. If you do not have one, I would not recommend purchasing one just for this standard of setup (unless you really want to get into Virtual Production or Photography and will add to your studio over time). You can probably hire or borrow one while you are learning.
The same could be said for the Pop-up chroma green/blue screen, although that is a much more budget-friendly investment.
The only real investment you are likely to have to make is the Video Capture Device. And that can be very cheap. The model listed above seems to work as well as the most popular model on the market and is over ten times cheaper. Admittedly, the budget version does not support 4K output, but with this level of setup, you should not be reaching for 4K*.
*If your camera can capture and output a clean 4K video signal, then there are also budget options for the capture device. At time of writing, you are looking at around £30 for an equivalent, unbranded, device from Amazon.
Setup 3 - Home Studio (intermediate entry level equipment)
Again, £532 and budget might not sound compatible, but there is, again, the chance that you might have access to some of the above already so the total cost listed is based on if you had to buy everything from new.
This setup would give you a better opportunity to capture live footage and compositie it within Composure.
You could “make do” without a tripod for your camera (but I would not recommend this) and you could hang your fabric against a wall. These budget alternatives are great for maximum mobility but just require a little more patience when keying your content. Loose fabric will create wrinkles which are a pain to remove, which is exaggerated when you are keyying in real-time.
One thing missing from the list above, and discussion until the point, is light…
Available light is a very important part of any studio setup. So far, the assumption regarding the setups above is that you are making use of either natural light or lighting you have available in the space you are setting up your studio.
This video (from 8:03) explains how to use natural light when lighting a green screen for virtual productions:
He goes on to discuss his lighting setup.
Diffused natural daylight is the best option without any decent studio lights. This is because the light is at an appropriate colour temperature. See this Wikipedia entry for a discussion on colour temperature if you are new to this concept: https://en.wikipedia.org/wiki/Color_temperature
Avoid using direct, evening and morning sunlight. Direct sunlight will cast harsh shadows and evening/morning light shifts the colour temperature to something that will tint your green screen (and your talent). This will make it more difficult to achieve a suitable key (even for beginner standard).
You will also want to avoid your room lights as much as possible for similar reasons of colour and shadow casting. Ceiling mounted lights above the talent/subject aren’t too bad for shadow casting because they’ll be casting the shadow onto the ground. But there’s no escaping from the colour cast of artificial lights and total lack of control of natural light. The next setup introduces adding support for this, as well as improving the size and scope of the keyable area.
Setup 4 - Advanced Home Studio
As you see, the budget for this quality of setup is becoming quite expensive. However, I would still keep in mind that if you remove the camera from the equation and buy 2 packs of the LED lights (at full price), it is around £500 for a very decent home setup. Furthermore, you can save a lot of money with much more budget-friendly budget light options. Here’s a video that discusses just that:
Lighting is a big topic. Too big to discuss in its entirety here. However, it is worth mentioning the reason for including the types of lights in the example budget above as it at least starts the conversation on light selection and purchasing.
Up until a few years ago, most studios and serious amateur photographers/cinematographers had to acquire their kit from specialist AV vendors. Here in the UK, a popular example is http://cvp.com. I’m sure other countries have similar suppliers. Now however, the market has been expanded by waves of cheaper electronics. AV kit prices as a result have tumbled. I am making no comment on if this is a good, or a bad thing. I am simply pointing out that it has made AV kit in particular, affordable to the masses.
Additionally, I am making no judgment on quality other than that which I have observed for myself. Some products are cheap because they are poor. However, others are cheap and can match (and in some cases, exceed) performance of the industry standard or popular products.
You also need to factor in demand and perceived brand loyalty when selecting your kit. For example, I have already touched on how you can save money on devices that have been priced to match increased demand. Do plenty of research. If you are on a strict budget, consider the alternatives and possibly second-hand. I would avoid purchasing a camera second hand unless you can test it yourself first.
Lighting a Chroma Backdrop
There is going to be a lot more on this subject, but for the sake of contextualising the next discussion point, there are some basics you need to know. Below are two videos which attempt to explain them, and seem to be targeting home studio and budget setups:
How to Light a GREEN SCREEN in 4 Minutes:
How to get a flawless greenscreen shoot in a tiny room:
You need to consider how you will light your chroma surface (and why). This will determine what sort of lights you will need. However, solely imagining the lighting setup for a single shot is probably a poor decision upon which to make a purchase.
For example, you might have a shot in mind and make a great selection when it comes to acquiring your lights. But your next shot might require something totally different. Studios (and film schools) will have access to a range of kit which makes obtaining it fairly simple. As a student practitioner, investing in your own equipment, you simply won’t have the budget to purchase a range of lights for most perceivable situations.
This brings me nicely onto the explanation for selecting the lights I did in the last studio setup discussed.
I purchased the lights cited in the example for the last studio setup for two reasons: 1) they were cheap… really cheap (at the time) and 2) they provided control.
The first point is down to timing. They just so happened to be one of the products I was considering recommending and purchasing myself (at the lower end of the price spectrum). To be honest, even the full price is still a bit of a bargain when you consider what they do (and the build quality is solid). The price allowed me to acquire two packs (4 LED panels). This will allow me to light numerous citations because they provide control…
The second point was the deciding factor. There were cheaper lights but they either were not supplied with vital accessories (such as a stand or power supply), they were too small, or they lacked control.
With 4 lights, I can have 2 lighting the talent/subject and 2 exclusively lighting the chroma surface. Aside from being controllable, all panels are from the same manufacturer so I stand the best chance of obtaining a consistency of lighting when it comes to illuminating the chroma surface. It does not matter if the lights on your subject/talent are a little out of balance because you will usually be lighting from two (or more) directions. However, it is essential the lights illuminating the chroma surface are as similar as possible to help with establishing a good key.
The key takeaways around light selection boil down to:
- Amount of lights required
- Ability to light chroma surface with consistent colour of light
- Controllability of light colour and brightness
- How you will place and power the lights
Remember, you can still achieve a decent key with a chroma surface illuminated by an overcast day. But if you want full control, you are going to need a reliable source of available light.
Taking the tips from there (and other sources I encountered during my research), I was able to make some sort of an informed decision.
Moving to 4K
If you are investing considerable amounts of money in your studio, then you will want a noticeable improvement in quality. The setup above is still operating at 1080p30. Even if your virtual productions are output at HD, being able to key at 4K would make a noticeable difference over HD. And if you are wanting to output higher than HD then you will have to jump to 4K for your live video feed.
So, you will need to make two upgrades to your setup versus the last setup discussed:
*The example cost is in addition to the cost of the original piece of kit quoted in the previous setup.
If improving quality is what you are looking to do, the above is an indication of the extra you would have to pay if acquiring the kit from new. It is pretty much a 100% cost increase. So here you have to ask yourself, do I really need to capture at 4K?
All of the above studio setups, even the most advanced 4K example, still only cater for static cameras. You can add as many cameras with live feeds to the composition as you like but there is still no real support for capturing and translating camera movement into your virtual production.
The main issue is Composure only operates in 2D space. It is totally dependent on you aligning the CG world with your live video plate. Once aligned, nothing can move because your camera is not communicating with Unreal to tell it to move the CG world along with the live video feed.
Thankfully, there are some options, again, that target different budgets:
Keyframe & Sync (Free)
This first method does not require any additional kit, so essentially it is free. However, it does not create any live link between the camera and Unreal Engine either, so it is really inaccurate and time consuming.
The idea is that you could manually keyframe the camera inside Unreal Engine to match any motion included with the live shot. However, it’s going to be almost impossible to actually do this live. Therefore, you’ll film your live action against the greenscreen and record it. Then you can feed the recording into Compourse and attempt to match the motion with the virtual camera.
To be honest, the results are almost always terrible, but this method could be fine for a proof of concept.
Synced Virtual Camera (Budget)
The next option is a very clever solution to link your camera and the engine using your mobile device, but it is rather experimental and has its obvious limitations.
This is achieved by setting up a Virtual Camera inside Unreal that “listens” to movement data coming from your iOS device. This information is obtained via your phone’s gyroscope and accelerometer. Therefore, it can detect rotation and the speed at which that rotation is occurring, but it can not detect positional motion. This information will allow the camera to “look around” from its fixed location.
Think of this as Augmented Reality. You have probably all experienced AR in some form. This option will give you access to the sensors in the iOS device which provide you with data from a typical AR experience.
If your camera is mounted on a tripod, you can get some very nice sweeping motion or simple tracking shots. If you choose to hold your camera, you can take advantage of natural camera shake.
Whilst not the best solution for total camera control, using this will elevate your cinematography if used correctly (and subtly). The only downside is, it is currently only available to iOS devices which are capable of supporting ARKit. To be honest, that is pretty much every iPhone and iPad since the release of the iPhone 6S.
In order to mount the iOs device to your camera, you will need some form of “holder”. You can probably rule out larger iPads for this setup, but you can still use an iPad to control a Virtual Camera.
Here is an example of an inexpensive mount that could allow the fixing of an iPhone to a camera: https://www.amazon.co.uk/gp/product/B01MU73LLA/ref=ppx_yo_dt_b_asin_title_o06_s01?ie=UTF8&psc=1
You can actually use any phone to power this setup, but it involves a lot more software and configuring. Check out this video which explains (not particularly well, but hey) how you could achieve this with an non-iOS device:
VR Tracked (Advanced)
The best way to allow your camera and CG scene in Unreal Engine to communicate in real-time is to employ the services of a proper tracking solution...
The film and animation industries have been using performance capture for a long time now, and these systems are usually very expensive and complex to set up and control.
Fourtnetaly, thanks to developments in immersive VR, there are now “affordable” solutions which are designed to be set up at home or in a small studio.
One of the most popular choices for VR capture to work with Virtual Production is the HTC Vive range of products: https://www.vive.com/uk/product/
You don’t need a full pack with the headset (although this is still useful for previs and set design in the virtual space), it’s just the trackers and base stations that are required. The bridge between the tracking system and Unreal Engine is handled through Steam VR which is free.
Check out the following video which discusses how the systems work together:
Throughout this guide we have looked at different studio setups for the student and the enthusiast that have offered a range of options and improvements to match budget and intent.
My advice for students, and those completely new to photography/cinematography, is to explore the free setup as much as you can. When you feel as though you are held back by its very obvious limitations, consider adding to your setup.
Build your kit collection slowly. Focus on improving one aspect at a time. do not purchase any equipment on a whim or solely on the advice of this guide. Your own research is essential to take full ownership of your decision to invest.
If you do wish to invest, choose what you upgrade/acquire carefully. This is especially true when starting your kit collection from scratch. For example, it would make more sense to spend less on a camera (forgoing 4K output) if it meant you were also able to afford better lights and an improved chroma surface solution. Obviously, if you were purchasing the camera to make movies and virtual production was a side-venture, then you would be better off getting the best camera you can afford and try to use the budget options for your virtual production studio.
It is a matter of balancing, choices and compromises. I have undertaken some of the research for you here in this guide (please also see the disclaimer at the start of this guide), but you will still need to make your own judgement by researching each and every item you add to your studio.