Loading

VIRTUAL PRODUCTION Learning Getting Started & Resources, September 2020

PREFACE

This document has been complied to help students participating on the Erasmus+ co-funded programme EMEX (Emerging Media Exploration). Grant number: 2018-DE01-KA203-004282.

INTRODUCTION

In preparation for the EMEX Autumn 2020 workshop, the student participants were expected to create prototype virtual productions. However, the students all come from different disciplines meaning there could students who were completely new to the subject.

Tis guide was created to help students self-orientate towards online learning materials, based on their individual situation, to help bring them up to a base level of understanding prior to arriving at the workshop.

VIRTUAL PRODUCTION: GETTING STARTED & RESOURCES

Use this guide to help plot a course through the potentially overwhelming journey required to understand how one type* of Virtual Production works.

The guide has been broken up into “starting points” so you can best select the heading that matches your current state.

*There are multiple “types” of Virtual production. During this course, you’ll learn about the different types and hopefully get to practice at least one of these types for your response to the task. To learn more about the types right now, check out The Virtual Production Field Guide by Epic Games (link in one of the sections below).

If you’re new to Unreal Engine

Thankfully, Unreal Engine is probably the best supported tool in the industry. Just performing a quick Google search for a topic will usually yield a list of resources, from tutorials to forum posts to official documentation.

However, when you are completely new, the sheer volume of content and support content can be overwhelming. Additionally, not every YouTube tutorial is necessarily good or accurate. There’s also the issue that different versions of the engine have different features. As a beginner, the recommendation is to download the latest version of the engine - which as at time of writing is 4.25.

There might be reasons why you can’t get the latest version (your OS needs updating is the usual culprit) but as long as you can get a version above and including 4.22, you’ll be able to do most of the things we’ll be looking at during the supporting video series:

Back to learning now, and the best starting point is to use the official content. Then, when you are more confident, you can start assessing any third party tutorial for its usefulness.

Start here: https://www.unrealengine.com/en-US/onlinelearning-courses

Register for an Epic Games account (if you don’t already have one - you’ll need one anyway to download the engine) and access the official Unreal Online Learning platform: https://learn.unrealengine.com/home/dashboard

Once there, your first course should be “Introducing Unreal Engine”. Either search for the course title or find in the Content Library under ‘Getting Started’.

Other courses on the platform recommend for beginners:

  • Unreal Editor Fundamentals - Editor Introduction
  • Real-Time Rendering Fundamentals
  • Rendering Kickstart
  • Lighting Essential Concepts and Effects
  • Materials - Exploring Essential Concepts
  • Materials Kickstart

If you’re familiar with Unity but not with Unreal Engine

There is a specific course on the Unreal Online Learning platform called “Making the Switch from Unity to Unreal Engine”. Check that out first, then you can proceed to the next suggested collection of resources.

Everyone should probably… (even if just for the duration of the workshop)

Subscribe to the Unreal Engine YouTube Channel: https://www.youtube.com/channel/UCBobmJyzsJ6Ll7UbfhI4iwQ

Subscribed to the Quixel YouTube Channel: https://www.youtube.com/channel/UCNTZFGRsyqNtvI4T2syNkIw (more on this later)

Follow Unreal Engine on Facebook to be alerted to free Marketplace content each month (on top of the permanently free collections): https://www.facebook.com/UnrealEngine/

If you’re familiar with Unreal Engine but new to Virtual Production

Of course, you should start by making sure you are familiar with what Virtual Production is and how we plan to use it as part of the EMEX project. There is a video tutorial series dedicated to this: https://www.youtube.com/channel/UCVOGrcxJZEZtdEwTP-iXKOg

It is recommended that no matter how familiar you are with Unreal Engine, everyone should take some time to explore the official descriptions from Epic about the development, engagement and industry involvement with Virtual Production using Unreal Engine. This can be found in two places from their main website:

Broadcast & Live Events: https://www.unrealengine.com/en-US/industry/broadcast-live-events

Film & Television: https://www.unrealengine.com/en-US/industry/film-television

To consolidate Virtual Production into a single subject, Epic have also created their Virtual Production HUB, a central place for industry case-studies, interviews and advancements in the technology: https://www.unrealengine.com/en-US/virtual-production

These sources of information will hopefully demonstrate what is possible with the technology and a glimpse of how the industry is adopting game engines into their production pipelines.

Finally, it is highly recommend that you leaf through The Virtual Production Field Guide by Epic Games themselves: https://www.unrealengine.com/en-US/blog/virtual-production-field-guide-a-new-resource-for-filmmakers

This fantastic publication goes into great detail about the subject, how it’s shaping/disrupting industry and even features a breakdown of the most common types of Virtual Production. You should be referring back to this document frequently throughout your learning journey.

Obviously, be aware the information suggested above is mostly for Epic’s own marketing so it is far from objective, but there generally isn’t anything too sinister or misleading - mostly because their product is free. That said, you should still proceed cautiously using this source for academic referencing purposes, particularly industry uptake. Try to seek independent reports where possible.

Recently, Epic published a guide to In-Camera VFX. Whilst out of the scope and budget of all but the biggest studios, it does still provide a fascinating insight into the pinnacle of virtual production today. Read about it here: https://www.unrealengine.com/en-US/blog/new-in-camera-vfx-onboarding-resources-for-virtual-production

When it comes to learning resources, as before, it is advisable to start by reading the official documentation on Composure* - Unreal Engine’s main tool for compositing CG environments with live or recorded video plates: https://docs.unrealengine.com/en-US/Engine/Composure/index.html

By no means does this resource provide definitive tutorials on everything you’ll need, but it does provide a ‘Getting Started with…’ for a couple of the connected tools and workflows. Have a read through so you know what you can expect, try the examples, and then you’ll be in a better position to understand the process and tools.

*Just note, it is possible to create virtual productions without Composure (for instance using sequencer and camera projected video planes), but the Composure workflow is probably the closest analogy to the traditional VFX process you’re probably familiar with. Therefore, it is the obvious place to start.

Whilst you're looking at the official documentation, you should probably also check out the connected tools and workflows of:

Media Framework: https://docs.unrealengine.com/en-US/Engine/MediaFramework/index.html

Sequencer Editor: https://docs.unrealengine.com/en-US/Engine/Sequencer/index.html

Professional Video I/O: https://docs.unrealengine.com/en-US/Engine/ProVideoIO/index.html

Whilst not always required, you’ll certainly want to know about these tools and workflows to better complete your understanding of how the engine handles media and cinematics.

My advice is to quickly skim through the information above, get an idea of the role they play and some of the terminology, and then use these resources as reference further into your exploration.

Whilst there are no real courses on the Unreal Online Learning platform specifically for Virtual Production, you should still check out the: Creating Photoreal Cinematics with Quixel.

This course demonstrates cinematics within the engine (full CG, no VP), but it does introduce Quixel Megascans, an asset resource that everyone with an Epic Games account has access to. It’s quite incredible and could help you construct your CG environment for your virtual production. There is more on this topic later in the guide.

Now the really good stuff…

Your first port of call to witness the type of virtual production you are most likely to achieve is to watch Matt Goodman’s excellent videos on how he started his journey from a cinematographer, to an authority on Virtual Production using Unreal Engine.

Subscribe or access his YouTube channel (Cinematography Database) here: https://www.youtube.com/channel/UCnw2-4hXY26-W2w9Ja9GBvw

Specific videos to get you started are as follows:

Getting Pro Video into Unreal Engine

Indie Virtual Production is here!!!

Indie Virtual Production | Real Time Key in UE4

These should give you an idea how to create your first hybrid live green-screen production in real-time. Hybrid-live green screen is the Virtual Production type that this guide will be focussing on. One thing to stress however, is that his set up and available equipment is approaching a professional standard (despite him being new to this field at the time these videos were released).

In one of the videos supporting this workshop, we discuss how you can replicate the results obtained by Matt in his videos with lower cost technology. In fact, if you have a USB webcam and a few sheets of green paper, you could technically do this for free. Obviously there’s pros and cons to different types of setup, and the better quality kit you have, the better the results will be.

For this course, we’re not necessarily looking for the best production values. We are more interested in how you choose to use these virtual production workflows to communicare your concepts.

In one of his videos (possibly in one of those linked above), Matt makes a reference to a tutorial from Andy Blondin. Andy worked with the development team at Epic to develop some of these tools, and in these videos, he goes into the Composure workflow in a little more detail as well as describing in detail, how the keying system works:

Unreal Engine Composure Intro

Unreal Engine 4.25 Green screen Keying and Composure

These are essential concepts to grasp, even if you don’t fully understand exactly how they work. The key takeaway to try and understand is that the various “Transform/Compositing Passes” are made up of Unreal Engine Materials or Material Functions. Think of these like adding “effects” to an AfterEffects composition.

The reason they took this approach as opposed to simply providing a collection of “effects” is to allow developers to write their own effects and passes. It might seem like an unintuitive solution at first, but it really is the smart way to approach this. Thankfully, a collection of the most useful compositing “effects” are already provided and populated by default when you add a media plate to Composure.

Further Exploration

If you’ve engaged with most of what we have looked at so far, you should have a pretty good understanding of what is involved with learning how one of the Virtual Production types is approached.

It is also worth pointing that the emphasis of the content has been around exploring the Hybrid-Live Green Screen workflow specifically. This is because it is probably the workflow you are most familiar with. It is also the workflow you are most likely to be able to engage with if you have limited access to studios and equipment.

The topic of Virtual Production is very hot currently, especially in the indie scene. There will be a YouTube video to support you with whatever you wish to try.