In Plain Sight - Sprint 1

 

Sprint 1 Contributions

1. Research and Development for Yarn Character Rig

Our cinematic, "In Plain Sight", has a character that is a ball of yarn with a loose string. Since it has no facial features, arms, or legs, it needs to be able to move and emote the way we want it to. As such, figuring out how to rig the character has proved challenging, as there is little to no reference for this type of character. 

The yarn character will move by rolling itself, simple enough. However, due to the loose string it has, I need to figure out how the yarn needs to be modeled and consequently rigged so that the loose string, which will act as its limbs and means of expressing itself, needs to move with the rest of the yarn "body" (the ball part). 

In this sprint, I created an iteration of the yarn character 3D model and identified the requirements for the yarn rig. The yarn character is scaled to its proper size according to our reference sheet and in the engine. 

The plan moving forward is to start testing different kinds of control rigs using this proxy model; however, I expect to use a ribbon-based control rig for the yarn character, so I want to write a script that makes ribbon-based controls faster to iterate.

2. Shader Research

I will also be doing the shader for this cinematic. Our team settled on an outlined, cell-shader, inspired by the game "Dispatch."

Here is what I learned so far from my research:

Shader Initial Observation:

1. Cell/Toon Shading

1. Banded shadows

2. Outlines

3. Western mixed with anime look

Requirements

3D models:

1. Reduce polygon noise. Remove minute details and surface complexity that would fight the stylized shader

2. Flat UVs. The texture should be large, flat color zones. Preferably non-photorealistic materials, using hand-painted textures,

3. Normal map. Either bake very minimal normals (subtle, stylized) or skip normal maps entirely on characters (probably not the best idea though). No complex normals.

Shader:

1. Toon Shading. 

1. Using the banded diffuse lighting method:

1. Get the dot product of world normal vs. the main light vector 

2. 3 main bands: Shadows, Midtones, Highlights

1. Using the “posterize” node

3. Should be able to customize band amount, light amount,

2. Outline post-processing material. 

1.  Using inverted backface culling 

2. Samples the scene's depth buffer and normal buffer, detects edges, and draws black lines 

3. (depth edges = silhouettes, normal edges = folds and details in the character) 

3. (Fake) Rim Light. Dispatch doesn’t necessarily have a fake rim light but we might as well explore this just in case we need to separate the characters from the background more

1. Controlled, bright (highlight) outlining the character based on light direction

Compositing:

1. Handrawn Background. Dispatch’s backgrounds are all entirely hand-drawn then composited with the 3D characters. If we want to get closer to their style as much as we can we need to:

1. Render out the 3D model with the cell shader via the Movie Render Queue in UE

2. Composite the hand-drawn background and 3D movie render in After Effects 

3. Add a texture overlay

4. Color Grade both of them together 

2. Full UE Pipeline.

1. Banded lighting (per material)

2. Edge detection outline pass

3. Screen space noise/grain texture

4. Color grade to flatten and desaturate (low contrast, muted mid-tones)

Use Unreal's Render Targets to composite hand-painted background images as flat plane behind the scene instead 

Apply chromatic aberration and film grain in the post-process volume.

Conclusion:

1. 3D models will need a cell-shaded material on them. The material should be affected by the cell shader based on the scene depth node, replacing the PBR lighting to create a more flat-shaded look.

2. A post-processing material will need to be layered on top of the scene that extracts the scene’s depth and normals to create the outline effect

3. We can add another post-processing volume to create a more film-grain effect or go crazier with the color grading to avoid needing to composite outside of UE.

More Resources:

1. 3 banded Cell shader: https://www.youtube.com/watch?v=lOOs74MLxec 

2. Cell shader with outlines: https://www.youtube.com/watch?v=YwZH4jCO4ZM&pp=ygUZY2VsbCBzaGFkZXIgdW5yZWFsIGVuZ2luZQ%3D%3D

3. Advanced Material cell shader: https://www.youtube.com/watch?v=HDyswSWIdY0&t=1768s&pp=ygUZY2VsbCBzaGFkZXIgdW5yZWFsIGVuZ2luZQ%3D%3D

4. HLSL Custom Nodes: https://www.youtube.com/watch?v=TqZKf8tMphU&t=11s

5. Anime Shader: https://www.youtube.com/watch?v=xf21CBx8rYs&t=1192s 

SDF Toon Mapping for the face (It’s in Korean): https://www.youtube.com/watch?v=T-bfBUhjCVc