In Plain Sight - Sprint 1

 

Sprint 1 Contributions

1. Research and Development for Yarn Character Rig

Our cinematic, "In Plain Sight", has a character that is a ball of yarn with a loose string. Since it has no facial features, arms, or legs, it needs to be able to move and emote the way we want it to. As such, figuring out how to rig the character has proved challenging, as there is little to no reference for this type of character. 

The yarn character will move by rolling itself, simple enough. However, due to the loose string it has, I need to figure out how the yarn needs to be modeled and consequently rigged so that the loose string, which will act as its limbs and means of expressing itself, needs to move with the rest of the yarn "body" (the ball part). 

In this sprint, I created an iteration of the yarn character 3D model and identified the requirements for the yarn rig. The yarn character is scaled to its proper size according to our reference sheet and in the engine. 

The plan moving forward is to start testing different kinds of control rigs using this proxy model; however, I expect to use a ribbon-based control rig for the yarn character, so I want to write a script that makes ribbon-based controls faster to iterate.

2. Shader Research

I will also be doing the shader for this cinematic. Our team settled on an outlined, cell-shader, inspired by the game "Dispatch."

Here is what I learned so far from my research:

Shader Initial Observation:

1. Cell/Toon Shading

1. Banded shadows

2. Outlines

3. Western mixed with anime look

Requirements

3D models:

1. Reduce polygon noise. Remove minute details and surface complexity that would fight the stylized shader

2. Flat UVs. The texture should be large, flat color zones. Preferably non-photorealistic materials, using hand-painted textures,

3. Normal map. Either bake very minimal normals (subtle, stylized) or skip normal maps entirely on characters (probably not the best idea though). No complex normals.

Shader:

1. Toon Shading. 

1. Using the banded diffuse lighting method:

1. Get the dot product of world normal vs. the main light vector 

2. 3 main bands: Shadows, Midtones, Highlights

1. Using the “posterize” node

3. Should be able to customize band amount, light amount,

2. Outline post-processing material. 

1.  Using inverted backface culling 

2. Samples the scene's depth buffer and normal buffer, detects edges, and draws black lines 

3. (depth edges = silhouettes, normal edges = folds and details in the character) 

3. (Fake) Rim Light. Dispatch doesn’t necessarily have a fake rim light but we might as well explore this just in case we need to separate the characters from the background more

1. Controlled, bright (highlight) outlining the character based on light direction

Compositing:

1. Handrawn Background. Dispatch’s backgrounds are all entirely hand-drawn then composited with the 3D characters. If we want to get closer to their style as much as we can we need to:

1. Render out the 3D model with the cell shader via the Movie Render Queue in UE

2. Composite the hand-drawn background and 3D movie render in After Effects 

3. Add a texture overlay

4. Color Grade both of them together 

2. Full UE Pipeline.

1. Banded lighting (per material)

2. Edge detection outline pass

3. Screen space noise/grain texture

4. Color grade to flatten and desaturate (low contrast, muted mid-tones)

Use Unreal's Render Targets to composite hand-painted background images as flat plane behind the scene instead 

Apply chromatic aberration and film grain in the post-process volume.

Conclusion:

1. 3D models will need a cell-shaded material on them. The material should be affected by the cell shader based on the scene depth node, replacing the PBR lighting to create a more flat-shaded look.

2. A post-processing material will need to be layered on top of the scene that extracts the scene’s depth and normals to create the outline effect

3. We can add another post-processing volume to create a more film-grain effect or go crazier with the color grading to avoid needing to composite outside of UE.

More Resources:

1. 3 banded Cell shader: https://www.youtube.com/watch?v=lOOs74MLxec 

2. Cell shader with outlines: https://www.youtube.com/watch?v=YwZH4jCO4ZM&pp=ygUZY2VsbCBzaGFkZXIgdW5yZWFsIGVuZ2luZQ%3D%3D

3. Advanced Material cell shader: https://www.youtube.com/watch?v=HDyswSWIdY0&t=1768s&pp=ygUZY2VsbCBzaGFkZXIgdW5yZWFsIGVuZ2luZQ%3D%3D

4. HLSL Custom Nodes: https://www.youtube.com/watch?v=TqZKf8tMphU&t=11s

5. Anime Shader: https://www.youtube.com/watch?v=xf21CBx8rYs&t=1192s 

SDF Toon Mapping for the face (It’s in Korean): https://www.youtube.com/watch?v=T-bfBUhjCVc

Artist Interview Experience

 Artist Interview with Joshua Axner and Brad Clark

Presentation

Interactive Experience - Stage A

1. Implemented anim montage for interactions

2. Made sure that the flask and interaction animation won't play when character is moving or jumping / finishes the montage first

<Showcases better in game playthrough>

Interactive Experience - Stage B

 1. Updated the weights on the new mesh!

2. Replaced the new skeleton mesh in the engine and made sure movement works with the controller. Also added gamepad controller input!:

3. Cell Shade Update: Testing with a non-realistic textured model as suggested!

With Cell Shade:

Without Cell Shade:

    Player character with cell-shading:

Inheritance

Script Objective:

1. I would run the import script first to get the geometry, csg, box, and tube module I created, then my code to get the desired output I wanted:

2. Final Shape:

Perforce Proof:

Interactive Experience - Stage C

 

Shaders and Post-Processing Effects

1. Updated Post-Process Shader Material

• 4 bands
• Bands are each affected by the lights
• Shadow contrast
• Ability to tint shadow and highlight

2. Updated Wine shader

•     Surface distortion when moved

3. Material Research:

Using the Substance Painter Stylization filter to create a hand-painted brush stroke effect:

• Affects all maps, including normal maps
• Added on top after the mesh is painted
• Easily integrate the maps into the engine's material instances.

4. Fixed foot sliding!

Python Classes

 

CSG Object Calculator

Script Objective:

This script creates and manages geometry in Houdini with Python:

• Auto-placed Objects: Each new object appears in the scene with a name label.

• Merged Layout: All objects are in a single container.

• CSG Operations via Python Operators:

• ~ Invert

• + Union

• - Difference

• & Intersection

1. Objects laid out with labels. Using the code stated in Canvas to test out the calculator:

box = CSG("box", "box")

sphere = CSG("sphere", "sphere")

thing = CSG("thing", "testgeometry_rubbertoy")

#Operations

union_result = box + thing

diff_result = thing - sphere

intersect_result = box & thing

invert_result = ~thing

2. Other operations:

#Operation Difference from first result:

union_result = sphere+ thing

diff_result = thing - box

intersect_result = sphere& thing

Troubleshooting:

When I was writing the script, it kept on giving me an invalid node name error, so I just looked up the error on Google and scoured several forums, which led me to the Python documentation for isalnum, allowing me to hardcode and remove any special characters Houdini does not like:

• only allow letters, numbers, and _
• for c in name: iterate over every character in the string
• c.isalnum(): true if character is a letter or number
• or c == "_": allow _
• else "_": any invalid name characters are replaced with _
• "".join(...): stitch back the string together

c meaning character

Perforce Submission:

//CLASSES/Cohort22/Students/Jocelyn.Fayola/Tech Art/Python/HW10.py