Mixed Reality Filmmaking F2021
Documentation
Week #1: Intro
Hello! My name is Andrew Win.
I am a first year IDM grad student. I love playing games and watching movies just as much as I enjoy learning about what makes them enjoyable.
I am a self taught artist, and I've made a few small games with a game developers and designers. I'm a bit familiar with Unity, Maya, Photoshop and Hexels 3. I was mostly a pixel/2D artist for previous projects, but always wanted to dive into 3D. I'm excited to be in Mixed Reality Filmmaking, because I want to learn how I can incorporate filmmaking techniques and camera movement into video games to make them feel more cinematic and immersive.
For my virtual world, I wanted to create a South East Asian themed fantasy world. I took inspirations from countries like Thailand, Myanmar, and Singapore. After watching the basic tutorials about UE4, Project File Naming and Materials, I created a landscape in an empty scene with large mountain ranges. I used the assets: Infinity Blade: Temple, and Sun Temple. I combined a few of the meshes from these assets and also recolored most of the materials.
Unreal crashed so many times in the span of a week, cause of D3D device being lost. I still have yet to figure out why this happens or how to fix it. I updated the drivers, Windows, and uninstalled Norton which seemed to have brought down the frequency of it crashing.
Since last week, I learned about Unreal's water plugin. It took some playing around with to get a hang of. With this new tool, I wanted to add a lake and rivers to my world. To do this though, I would have to create the landscape from scratch. (Creating one on an existing landscape created some weird crater around the existing landscape.) So I did. I also painted some textures onto the landscape, which worked out fine in the beginning, but after Unreal crashed (D3D device being lost), the entire thing got a bit wonky. Textures went missing from the paint window, and the rendered textures were all blotchy. I wanted to add spectral spirits to my world, but due to not finding any, and lack of time to learn how to 3D model one, I used Mixamo to find something that would fit into the theme of my world. I found one and imported it into Unreal. For some reason, it came out looking transparent, but I think this actually looks better than if it were completely solid as it was what I was aiming for anyways.
From here, the next steps would be to fix and improve the landscape textures, polish and add more lights, and add more objects to the world. I wanna add different types of creatures to serve as "guardians". Maybe also some music?
Week #4: Sequencer and Camera Movement
Sequencer was both fun and challenging. I experimented with multiple camera cuts and blending them but it was still a bit overwhelming for me. I stuck with a single linear camera and tried to show the world from the perspective of someone who is visiting and has never seen this world before.
One interesting warning I saw was "TEXTURE STREAMING POOL OVER BUDGET". I'm curious what it is and how I could fix it. My assumption would probably be to be more wise with material usage, using material instances rather than the actual material.
Week #5: Keying with After Effects
This week's assignment revolved around keying green screen footage in After Effects. I'm excited to finally be able to play around with green screens now. I feel like the possibilities have broadened so much now.
Because there was a specific narrative I wanted to tell with my initial world, I couldn't find any green screen footage that fit. So instead, I decided to have some fun and used green screen footage of Jean-Claude Van Damme. Rather than creating a new world, I used the SunTemple project to serve as the location. I think the fact that the location and Van Damme clearly don't synergize well makes it funnier. To further this intent, I added a Luchador character from Mixamo.
Week #6: Match Motion I: After Effects
○ Use the 3D camera tracker to track an outdoor scene and one on greenscreen.
○ Add 2D elements to the tracked scene in After Effects and render it out.
Added 2D portal gifs to the camera tracking for an outdoor recording as well as a greenscreen recording.
Week #7: Match Motion II: Maya
○ Transfer your camera data from AE to Maya using Live Link.
○ Place 3D objects into the scene and view through the tracked camera.
Tracking a 3D Donut on a plate in a video recording of an empty table.
Demonstrate your understanding of the pipeline by using composure to composite tracked video footage with a virtual environment.
Chair and table added to a video in UE4.
The camera tracking seemed off when imported to UE4. The tracking seemed fine in AE, I may have did something wrong in Maya and/or UE4.
Week #9: Final Project Proposal
For the final our video idea was a story of a young man(teen/ early 20’s) on the search for the city of gold, rumored to be found atop the highest mountain in the world. He sets off from his home town and finds the mountain. As he ascends the mountain, he ages rapidly. By the time he reaches the summit, he is an old man (80’s, 90’s) and is too weak to carry back any of the gold he finds.