1969: A VR Music Video

— and a production blog!

Imagine a pop-up book come to life—in virtual reality

My capstone project at USC film school was 1969 VR, made possible by the Jaunt Cinematic VR Lab and cross-disciplinary coursework in the departments of Film Production and Animation. The project presented many unique technical challenges from the new medium of VR at the time, and for being my first foray into a 3D/VFX production of my own direction. We made tons of mistakes — here’s just a bit of what we learned along the way!

Exhibitions: 1969 VR has screened at USC First Look 2018, Sydney Lift-Off Online Film Festival 2017.  Proof-of-concept media have been showcased at Dubai International Film Festival 2016 (unofficial) and the VRSC Student Showcase 2017.

Awards:  Jaunt Cinematic VR Lab - Experimental Project Grants recipient 2016-2017. USC Discovery Scholars Award (Cinematic Arts). USC Symposium for Scholarly & Creative Work: Second Place Winner for Digital Media.

VR TEST SHOOTS

Testing proximity and the minimum "screen-safe" height that our green screen will need to be.

If you take a closer look at the image on the far-right (the equi-rectangular preview frame captured by the Jaunt cam), you'll notice horizontal blue tape marks on the building behind him.  These marked out potential screen heights (increments between approx 11' and 17') that we could measure against our VR lab specialist Shaoyu (6'2 tall, approximately the same height as our lead actor) from his spot at the minimum "safe" distance (3 to 4') from camera.

PRODUCTION & VFX

Click to see the construction plan, as well as the supply list (courtesy of our Key Grip Philey Sanneh).

In the gallery below, click on an image and hover
to reveal descriptions of each step of the process.

THE 270-DEGREE GREEN SCREEN

Top right: diagram of green screen stage assembly.  (1) Speed rail frame is suspended from the lighting grid, lowered to reachable distance. (2) Top frame is raised as lower frame is placed beneath.  (3) Verticals are inserted into joints that bring the top and bottom frames together.  (4) The green screen is attached to the top frame as the unit is raised, then stretched/attached to the bottom frame after the unit is raised to its maximum height.  (5) The green floor is inserted.

Throughout the building and lighting process, our VFX sup Jonathan Ho diligently monitored the evenness of the screen to ensure a useable key in post-production.

MOTION CAPTURE

To sync the Jaunt footage with the motion capture data, we would do the following for every take: (1) Shaoyu and Ari hit REC at the same time; (2) after the slate is called, the actors would "slate" by getting into T-pose and dropping their arms to the side while on camera, before we called action (center-right and lower-right).

Of course, hiding the motion capture suits underneath the costumes were one of the other challenges that our production designer/costumer Monah Sutlapa had to tackle for this shoot!

POST-PRODUCTION

After our post-production moved out of its experimental phase, it was time to get focused.  Knowing what processes did and did not work, I needed to start streamlining the rest of post and ensure the consistency of our weekly asset deliveries.  So, to start the new year off on the right foot, I created a new post-production workflow guide (again, for the morbidly curious and detail-oriented readers).  This document standardized the delivery formats and schedule for the several animators on our team, all of whom had to juggle a full course load and other activities along with this project!

2D ANIMATION

LOOPS & CYCLES

As indicated in the full post-production workflow guide, 2D animations were created in a variety of different programs depending on the preference of the individual animator, ranging from Photoshop to After Effects to TV Paint.  But there's a sizable amount of animation work that needs to be done: 13+ unique characters and multiple hand-drawn graphics for a 4-minute piece!  

So, the team was encouraged to create looping animations in segments whenever possible.  

Here are some of our favorites!
(Courtesy of Megan Maniago, Deena Baum, Kaley Cho, Yimin Zhang)

ANIMATION TEMPLATES

To speed up turnarounds for some of our animated assets, I created animation templates to apply to our batches of images, instead of animating some 50+ elements' effects by hand. These templates combined (1) an animated track matte with (2) an animated paper texture to create the effects of paper crumpling, tearing, and folding itself to life.

Below: Templates and composites for batched animations/fx.

MAYA INTEGRATION

CREATING A VIRTUAL "JAUNT CAMERA"

Before the Adobe Creative Suite and Dashwood made 2D VR graphics absolutely seamless, our best practices for bringing 2D art into a stereoscopic 360 space was by rendering stereo-360 cameras in Maya. Setting up the project, we mapped the left-eye Jaunt footage onto a sphere with two-sided texturing and had to match the Maya camera’s position to a T at the center of the sphere. From there, we could use the Jaunt footage as a reference layer to determine our 2D sprite layout. While simple x-offset workarounds will fly with re-positioning elements in monoscopic 360 video, the math doesn’t translate the same way with stereoscopic 360. Thus, we had to ensure that our layout and comp elements were perfect (or close enough!) in the stereoscopic 360 renders we were kicking out from Maya.

INTEGRATING 2D ANIMATIONS

We mapped image sequences or sprites onto 2D planes as textures, in order to achieve the look of a pop-up book in 3D space. This means we had to manually adjust the z-depth and positioning in Maya before staging the final layout and renders — all a guess-and-check process.

CREDITS

POST-PRODUCTION

Editing, compositing by
Kathy J. Liu

VFX Artists
Jonathan Ho, Yimin Zhang

Digital Production Designer
Kathy J. Liu

Digital Art Directors:
Deena Baum, Cheryl Chan, 
Brenda Chen, Kaley Cho, 
Megan Maniago, Yimin Zhang, 
Dave Zhu

Faculty Advisor
Candace Reckinger

Special Thanks
Jaunt Studios & the Jaunt Cinematic VR Lab,
Mike Fink, Mike Patterson, Eric Hanson,
Christine Berg, Carlos Jauregui,
Composite Components