MFA Thesis:
Comparing FX for Real-time and Off-line
Title:
​
Programs:
Houdini, Unreal, Substance Painter/Designer, Photoshop
​
Modeled:
Houdini and Purchased Models
​
SIM:
Houdini and Unreal
​
Lighting:
Mantra and Unreal
​
Texture:
Mantra, Substance Painter/Designer, Photoshop
​
Rendering:
Mantra and Unreal
​
Composite:
NukeX
Description:
A Comparative Study of Differing Design Challenges when Creating FX for Realtime, or Off-Line Industries.
​
Thesis Committee:
Professor Deborah Fowler - Chair
Professor Stuart Robertson
Jennifer McSpadden (Goodbye Kansas)
​
Thesis Statement:
Real-time production capabilities have given rise to new design challenges when producing Visual Effects. Using current simulation FX techniques and exploring multiple FX, a comparative examination will illustrate the new real-time production workflow’s advantages and disadvantages compared to the traditional pipeline, referred to as off-line from this point. The examination will demonstrate that real-time offers substantial benefits to projects using virtual production. However, this comes at the cost of FX still dependent on the off-line workflow.
​
The FX case study will focus on an impact/explosion on the lunar (moon) surface dealing with particles, volumes, destruction in the absence of an atmosphere, and low gravity to document the technical and creative solution/challenges by comparing the two workflows of real-time and off-line FX. The FX will be designed for use within an LED Volume virtual production project, thus not requiring an entirely open-world experience.
Thesis Paper
​
Visual Component
Final Comparison
Kevin Cushing Interview
MFA Candidacy Reel
Expected Timeline
Thesis Committee Meetings
​
March 8, 2021 (Stuart Robertson)
Comments:
-The layout is okay, but the spacing between the array's solar panels is close together.
Solution:
-Adjust phyllotactic "cval" value from 2.0 to 3.0. Agreed with the committee member, the solar panel array layout was improved.
​
March 9, 2021 (Deborah Fowler)
Comments:
-The layout is coming along. Needs improvement to terrain for "reactor" building. Also, the outer hills would be the best impact site for interesting debris and a solar panel array in place.
Solution:
-Improved procedural terrain update on "reactor" building. View this Solution
-Updated solar panel placement to better source in the desired region. View this Solution
​
March 14, 2021 (Jennifer McSpadden)
Comments:
-Use a wide-angle shot to set the context and then punch back into close up for the FX in both offline and realtime. Aim the close-up shot on the hill terrain facing toward the base to have context in the background. Agrees best point of impact would be in the outer hills with the solar arrays. When capturing out realtime FX, use a moving camera and play the level using a screen capture program and any debugging information.
Solution:
-Continue making updates with comments in mind.
​
March 23, 2021 (Deborah Fowler)
Comments:
-Layout camera/camera moves before beginning FX. Create two different camera rigs, one planning on using a cut edit and the other using a single camera move.
-Create spheres to represent the objects that impact to have an idea of framing.
-Use space rocks as impact objects.
​
Solution:
-Begin testing camera angles/movements. View this Solution
-Design impact system and area to aim at impact. View this Solution
-Test bringing heightfield into Unreal. View this Solution
​
March 24, 2021 (Stuart Robertson)
Comments:
-Consider showing outer space as a bit of background (possibly with Earth in view) to help better set the scene context.
​
Solution:
-Create a new camera in the suggested position. View this Solution
​
March 30, 2021 (Deborah Fowler)
​
Comments:
-Good progress on testing HF into Unreal and beginning design of rock (impact) objects. The second camera movement is working well, but the first camera begins its motion a little too early, consider adding a little more transition time. Begin to do simple shader tests for offline render to see the impact of motion blur and detail. Finally, test HF cutout to see if can be imported into Unreal and have seamless transition to FX geometry.
​
Solution:
-Begin testing HF cutout into Unreal with simulation RBD geometry. View this Solution
-Create UV's on impact rocks and begin shader testing. View this Solution
-Adjust the timing of Cam1 animation to allow more pre-roll/transition time. View this Solution
​
April 6, 2021 (Deborah Fowler)
​
Comments:
- Check out this tutorial on UE4 and Houdini destruction and see if it is of any use.
Simon Verstraete: Houdini Real-Time Destruction
https://www.sidefx.com/tutorials/real-time-destruction-1/
​
April 6, 2021 (Jennifer McSpadden)
​
Comments:
-Comes up often on set, creating a volume for the camera operators to work within. "This is the part that works for you (camera operator). While you are in a 3D environment, like a game engine, where you have elements in all directions, it doesn't mean you have full freedom to move anywhere you want. We have designed for you (camera operator) a space to work within."
-Does the scale and position update on the Houdini Engine HDA live update within Unreal?
*No, it requires recooking to update the HDA; however, translations applied to the geometry created within unreal can be live updated. Upside recooking only takes about a min or two*
-Consider turning on and testing raytracing within the project, seeing the impact on performance.
-How much of the simulation controls and rock generation will be available in Unreal.
*Unfortunately, at the moment, I wouldn't trust the Houdini Engine to run a complex RBD simulation within Unreal in realtime.*
-The simulation is planned and timed out, and the result comes into Unreal without interactive change.
-Begin mapping out what you want the Unreal blueprint to do and what inputs will be required to run. From there, we can begin designing and testing the system. Also, consider creating an "empty actor" to hold the blueprint to be quickly loaded into any scene/project.
-As a stretch goal, consider trying to test the Unreal system with live camera capture. Consider downloading the Unreal Remote system onto a phone or iPad and use that as a live camera track. You could then bring the keys from the live track into any other software.
​
IOS Unreal Remote
https://docs.unrealengine.com/en-US/SharingAndReleasing/Mobile/iOS/UsingSlateRemotePlugin/index.html
​
Unreal Live Link
https://docs.unrealengine.com/en-US/AnimatingObjects/SkeletalMeshAnimation/LiveLinkPlugin/index.html
​
Solution:
-Set up full Unreal project structure and test raytracing within the scene.
-Map out the desired result of the Unreal blueprint for volumetric display, along with required inputs.
-Check out Unreal remote and Live Link.
​
April 13, 2021 (Deborah Fowler)
​
Comments:
- Good progress on fracture. Keep working on sim piece activation and reach out if still having issues.
​
April 20, 2021 (Deborah Fowler)
​
Comments:
- Good that the simulation piece activation is working.
- Unreal match-up appears to be working well, work on optimizing later. Begin developing the pipeline for particle systems.
​
Solution:
- Began setting up Houdini particle simulation sources and simulation.
- Researched Houdini particle to UE4 export methods.
​
April 25, 2021 (Jennifer McSpadden)
​
Comments:
- Working with UE can be frustrating at times since it doesn't give you clear error message so you have to keep good track of what you are doing and sometimes blindly guess what is wrong.
- Look into how UE is importing the solar panels, are they treated as a unique assets two-hundred times, or as one asset instanced two-hundred times.
​
Solution:
- Keep close track of steps taken for exporting Unreal assets.
- Unable to use instanced solar panels due to needing individual fractures inside Houdini.
​
April 27, 2021 (Deborah Fowler)
​
Comments:
- Good to see that you can import Houdini point caches into Unreal.
- Now need to work on improving the visual look. At the moment the particle sprites look bad. Find a way to improve the look of the particles without draining resources.
​
Solution:
- Investigate different methods for displaying particles in UE4.
- Meet with ITGM Professor Charles Shami on April 29, 2021.
​
April 30, 2021 (Debroah Fowler)
​
Notes:
- Being shown at this meeting was a changed methedology of rendering the Houdini particle simulation split into three layers and rendered then use as sprite flipbooks on three different planes in Unreal.
​
Comments:
- Three plane setup does not work. You regain the point count but it looks flat and doesn't line up well with the rest of the FX. Try a different method.
- Look into UE4 Niagara particle limits and how to possibly adjust this limit.
- Reach out to Professor Ari Cookson for more help with Niagara.
​
Solution:
- Reached out to Professor Ari Cookson for a meeting.
- Researched UE4 particle limits and the best way to represent Houdini simulation particle count while maintaining scene depth.
​
May 4, 2021 (Deborah Fowler)
​
Notes:
- Update particle system to use Houdini point cache in Niagara but with limited count. Each particle then uses sprite render to display a SUBUV texture to pick a rendered image with a large quantity of particles.
​
Comments:
- This method looks promising. Use this technique as the basis for all further testing.
- Attempt to get the sprite to now properly align and scale based on the Houdini velocity.
- Begin working on the visual look and control of the particles themselves.
​
Solution:
- Investiage how Niagara aligns the sprites to its own velocity, and determine if a custom vector can be inputed.
- Develop a way to control the visual look of the lighting on the particles / material.
​
May 11, 2021 (Deborah Fowler)
​
Note:
- Unreal tries to hide many things to allow quick plug and play workflow, but it doing so makes itvery difficult to wrangle back control and build custom setups.
​
Comments:
- Try dropping the individual POP sprite render to 512 to better optimize.
- Begin researching how to do volumes and spend some time learning Unreal in general.
- Reference Ian Harvey's blog about volumes into Unreal.
- Focus on working with getting detail into the materials.
​
Solution:
- Particle sprites rendered at 512x512 and converted into atlas sheet. Works well for optimizing frame rate in Unreal.
- Reference Ian Harvey's blog about Unreal and decided to use a similar system.
- Began learning more about Unreal lightmap and LOD systems.
​
May 16, 2021 (Jennifer McSpadden)
​
Comments:
- Look into enabling motion blur within the camera itself and in post processing volume.
- The solution used for the particle opacity is good. Now try to improve the overall look.
- Consider looking into ways to save and export this as a sharable project, but only after everything else is complete.
- Begin looking into the camera setup for final capture.
​
- When writing the paper lay it out in a manner that you explain what you did and then use the source to justify why you did it that way.
​
Solution:
- Particle look was improved by changing the Atlas from Houdini used in the SubUV setup.
- Other aspects are being worked on.
​
June 24, 2021 (Deborah Fowler)
​
Comments:
- Begin writing out the paper while the pipeline is fresh in your mind.
- Volume texture is working well.
- The dust (POP) feels too dominant/big.
- Tone down dust color.
- Leave more particles for pyro instead of dust.
​
Solution:
- Outlined paper and key points to hit for each simulation type.
- Reworking particle sprites and overall look.
- Began adjusting particle numbers per system.
​
July 11, 2021 (Jennifer McSpadden)
​
Comments:
- Fix dust particles camera facing orientation.
- Try changing the particle material to a translucent material for an improved look.
- Look into texture baking out of Houdini to help UE's performance as much as possible. Maybe reach out to professor Shami.
- Explore culling volumes.
​
- In the paper make sure to mention in each section what you learned along with what decisions you made.
​
Solution:
- Particle orientation was adjust to be based on the camera position instead of the Houdini velocity.
- The particle material was left as masked since changing to translucent left all the particles as semi-opaque.
- Houdini has a BakeTexture ROP that will be used in the pipeline.
- Still exploring cull volumes.
​
July 15, 2021 (Stuart Robertson)
​
Comments:
- Pay attention to unclear sentences in thesis paper.
- Make sure to clearly state the differences and challenges in each section of the breakdown.
- Pick one citation style and ensure to adhere to formatting. MLA, Chicago, etc.
​
July 21, 2021 (Deborah Fowler)
​
Comments:
- Adjust thesis proposal timeline to allow more time for committee to read final thesis paper.
- Unreal star map( Sky_Sphere) looks repetitive.
- Look at NASA for star maps.
- Keep an eye on frame rate.
- Chekc UV type for lightmap issues. UV's have to be on vertex not point.
- Remove some of the background base elements to improve FPS.
​
Solution:
- Adjusted and submitted thesis proposal to other committee members.
- Brought in an HDRI star map from NASA for background.
- Worked on optimizing FX and scene as a whole.
- Lightmap resolution improved and created new lightmaps inside Houdini.
- Removed five base elements to improve FPS.
​
August 8, 2021 (Jennifer McSpadden)
​
Comments:
- Real-time render is looking good. The post process volume is doing a lot of the heavy lifting.
- Begin setting up the sequencer video.
- Capture final result from viewport with analytics.
- Consider turning off one monitor to see if that will improve performance.
- Disconnect from internet for final capture.
- Kill any unwanted tasks.
- Intersting point to talk about how UE categorizes its functionality for what it considers to be part of rendering and part of the particle system.
- Remember to emphasis that you are designing this for virtual production and a camera, not an entirely open world experience.
​
August 11, 2021 (Deborah Fowler and Charles Shami)
​
Comments:
- Make sure to be consistent with double vs. single quotes.
- Texture have greatly improved.
- Do not leave the Houdini Engine HDA in the finalized build.
- Begin working on off-line elements.
- Inside thesis paper put Unreal render frames.
- Adjust Earth outline/halo effect.
- Unreal Scene Approved, begin capturing elements!
​
Personal Log/Notes - Click Date to View if Available
​
February 21, 2021 Niagara Rop
​
-Program: Houdini 18.5.408 / .462
-Niagara ROP was writing out an empty file cache 1kb in size.
-Error reported in HBJson.py file
-Located in C:/Program Files/Side Effects Software/sidefx_packages/SideFXLabs18.5/scripts/python/
​
Error running Python menu script in parameter,
Type (/obj/Niagara_rop_export_failure/niagara1/ropnet1/niagara_rop1/attr_type1) :
Traceback (most recent call last):
File "<stdin>", line 3, in expression
File "C:/Program Files/Side Effects Software/sidefx_packages/SideFXLabs18.5/scripts/python\hbjson.py", line 57, in get_type_menu
for name, value in sorted(cls.__dict__.iteritems(), key=lambda i: i[0]):
AttributeError: 'mappingproxy' object has no attribute 'iteritems'
Error running Python menu script in parameter,
Type (/obj/Niagara_rop_export_failure/niagara1/ropnet1/niagara_rop1/attr_type2) :
Traceback (most recent call last):
File "<stdin>", line 3, in expression
File "C:/Program Files/Side Effects Software/sidefx_packages/SideFXLabs18.5/scripts/python\hbjson.py", line 57, in get_type_menu
for name, value in sorted(cls.__dict__.iteritems(), key=lambda i: i[0]):
AttributeError: 'mappingproxy' object has no attribute 'iteritems'
​
-Resolution: Update to python 2 release of 18.5.495
-Conclusion: Bug in python 3 releases of Houdini 18.5
​
February 24, 2021 SideFX Labs Tools
​
-Texture Sheet Node
-Auto generates $F4.exr namespace regardless of user input.
​
-Motion Vector Node (Based on Rebelway Tutorials)
-Allow editing, and turn off override camera resolution on "motion_vector_render" node.
-Add color correct in "dilate" comp net.
-Reference camera resolution in the final color node in "dilate" network.
​
-If the light direction is known can optimize for realtime by using split sheets, otherwise have to use RGB split channels (multiple).
​
February 26, 2021 Thesis Review
​
-Compare compositing in Nuke vs. Unreal Composure tool.
​
March 5, 2021 Lunar Base Layout Variant
​
-Issue: Want to use a variant attribute with multiple groups to choose from.
-Solution: Decided to make "variant" attribute a string instead of int to allow concatenation.
​
March 7, 2021 Flatten Heightfield
​
-Issue: When base models are copied to points they often clip inside the terrain.
-Solution: Query the P.y position of the model point and set HF "height" as this value.
​
-Issue #2: HF will only query point 0's height.
-Solution #2: Use foreach block set to By Pieces or Points gather method as Feedback Each Iteration and create a second meta import node and set to Fetch Feedback with the initial HF plugged in as input.
​
March 8, 2021 Solar Panel Orientation
​
-Issue: Solar panels need to be oriented to the lunar surface and facing toward the desired direction when copied to points.
-Solution: Use code created inside the "Heightfield Scatter" node to generate new point normals. Use QRotate with original direction normals and define heightfield normal as the "up" vector.
​
March 14, 2021 Volume Sample Heightfield
​
-Issue: Attempting to sample "mask" volume from heightfield onto scattered points. When doing so, the returned value does not reflect the painted mask.
-Solution: Attach a primitive node to the heightfield turn on Adjust Border and set Border Type to Streak under the volume tab.
​
March 25, 2021 Heightfield to Unreal
​
-Issue: Want to bring HF into Unreal with as little resource strain as possible.
-Solution: Attempted method suggested by Deborah Fowler using Unreal terrain tools and HF map.
​
-Issue #2: Import works but has stepping issues in the resulting field in Unreal.
-Solution: Research ongoing.
​
March 29, 2021 Heightfield to Unreal Continued
​
-Issue: See March 25, 2021
-Resolution: Upres Houdini HF using "Heightfield Resample" to have a similar voxel scale to desired pixel aspect ratio. The SideFX Houdini node "Heightfield Output" then writes out a correct 16bit grayscale png file needed.
​
*April 2, 2021 Unreal and Houdini Engine*
​
-Lining up the Unreal HF and Houdini geometry proved difficult due to the difference in unit scales. Instead installed the Houdini Engine plug-in for Unreal 4.26.1 and began creating an HDA to load within Unreal.
​
-Houdini heightfield "mask" volume can be used as an opacity mask on Unreal heightfield when renamed "visibility" and read in a material.
​
-After creating the HDA and loading it into Unreal, the generated geo was not aligning correctly with the Unreal heightfield. The HDA spawn location must be at the origin {0,0,0}; otherwise, it cooks incorrectly.
​
​
-When using the HDA, the seams between the HF and geometry need to be put in lower slope regions or in inconspicuous locations to hide transition.
​
-Note: There is no seam issue when working inside of Houdini itself.
​
April 6, 2021 Houdini Engine Note
​
-Each time the HDA is recooked within Unreal the inputed geometry is recreated in the HE temp folder. Regular cleanup of old geo version will be necessary to prevent overload and system drain.
​
April 10, 2021 Unreal and UV's
​
- Unreal is very picky about UV's. They have to be completely clean to be viable.
- UV Flatten in Houdini is extremely useful to create and adjust / layout UV.
- UV Layout accepts a parameter called "iscale" for variable scaling of UV. Allows adjustement of island importance.
​
April 12, 2021 Fracture Updates and Optimize
​
- For best results in fracture bottom face of geo needs to be poly reduced but not turned into a single face.
- Clean node needed to remove unwanted geometry and edges.
- Divide node used after fracture and pos restore fixes non-manifold geometry.
​
April 13, 2021 RBD Activation and UE4 Alembic Import Error
​
- UV Layout can use second input to place new islands over previous UV.
- When preping geometry for RBD simulation, I wanted to deactivate the outer edge pieces. Promoted "Extrude Side" group from extrude volume node to point group. Then packed (asseble) to reference group.
- Didn't work.
- Importing alembic into UE4 caused error message and failure to load.
"Expecting triangles (3) or quads (4)"
​
April 14, 2021 RBD Activation and UE4 Alembic Error Continued
​
- To deactivate sim edge the geometry is unpacked and all except the "extrude side" group is blasted. Then the "findattributeval" vex code is used to test if a point on the packed sim geo has a matching point value on the blasted edge and is then deactivated.
-To solve the Unreal alembic import error, the finalized sim Hi-Res geo is unpacked and a divide node is used to triangulate all faces.
- Imprting geo with large poly count / bizarre setup crash HE in Unreal.
- Pay attention to poly reduction methods.
- Bricker Polygon (Interior Detail) lags Unreal scene. REMOVE.
​
April 19, 2021 UE4 Lag and Destruction Optimization
​
- Destruction elements lag Unreal to 23 FPS.
- Optimize fractures based on distance from impact.
​
April 21, 2021 UE4 Niagara and Houdini Particel Sim
​
- Houdini Niagara reads Houdini point cache, but does not despawn previous frame points leading to large point accumulation.
​
April 22, 2021 Niagara Particle Cache Fix
​
- Must write out a float attribute that UE4 can read into Niagara and use to kill particles.
- Declared "dead" prior to POP simulation or use a different name to stop simulation conflict.
- Niagara ROP only works when Houdini is in Auto-Update mode.
- Point count runs extremly slow adjust to pre-render texture placed on sprite planes driven by point from sim.
​
April 27, 2021 UE4 Sprite Atlas Creation and Different Methedologies
​
- When rendering texture atlas DO NOT render pre-roll, instead render from start of action and adjust timing in Unreal.
- Try large sprites with particles on top?
- Unreal max supported import textures = 16,384 x 16,384.
- IDEA: Create 3 progressively larger planes, create uv from Camera so that they overlap, curve the planes and use as sprite geo.
​
April 28, 2021 Alternative to Atlas Flipbooks
​
- Found you can read img sequence in unreal but get an import error after saving the .uasset.
​
April 29, 2021 UE4 Movie Sequence
​
- PNG sequence not saving proerly in IMG Media Seq .uasset.
- The Mov / MP4 file needs one for color and another for alpha.
​
May 3, 2021 Professor Cookson Meeting
​
- Look into Volumetric Cloud (Night Sky Dome)
- Flickering particles -> Adjust emitter bounds in Niagara and in UE4 scene.
​
May 4, 2021 Niagara Particles SubUV Textures
​
- Use SubUV set Niagara parameter on spawn "subUVImageIndex".
​
May 7, 2021 Particle Needed Fixes
​
- Sprite system causing lag need to optimize.
- Optimize by reducing particles in high probability areas in Houdini.
- Reduce sprite texture scale to 1024 x 1024.
-Need way to adjust sprite scale.
- Scaled opacity by distance from camera.
​
May 9, 2021 Niagara Dynamic Scripts
​
- Tried to create dynamic input script to update sprite size by distance from camera.
- UE4 Camera query position (Experimental) only updates once per compile instead of over time.
​
May 14, 2021 Unreal Optimizations
​
- Turn off SRGB on textures using RGB packing to prevent gama correction.
- Consider writing out alpha as separate image.
- Watch model material ID number.
- Look at scene iport for environment elements.
- Use merge actor tool and HLOD.
​
June 14, 2021 Pyro Simulation Setup
​
- For pyro a hi-rez sim is not needed. Instead use a low rez simulation to drive point movement.
​
June 15, 2021 Pyro Texture
​
- Consider finding a way to switch pyro textures on sim points.
- Impact data or relationships to meteor/ground.
​
June 18, 2021 Pyro Texture Continued
​
- Used Rebelway Creating Looping FX Using Unreal.
​
​
- Need way to improve pyro sprite look, normals?
- Slow down sprite 5+ seconds by reducing pryo velocity.
​
June 22, 2021 Volume Sprites Normals
​
- Create looping texture render.
- Create sprite sheet.
- Import into photoshop and increase contrast from 6D lighting.
- Convert to black and white.
- Filters -> 3D -> Normal map.
​
July 1, 2021 Research Texture Baking
​
- Research Houdini texture baking for Hi-Rez surface model to low-rez simulation output.
- Houdini ROP BakeTexture node.
​
July 17, 2021 Dust Particle Change
​
- POP sprites, rerender points and small rock fragments
​
July 19, 2021 Unreal Materials
​
- Unreal-material node name path must be "relative/path/to/shader/inside/Unreal/project" to work
​
July 22, 2021 Niagara and Texture Baking
​
- Niagara module script "Houdini" namespace can not be spelled "HOUDINI"
- BakeTexture ROP, EXR and RAT only file format that writes correctly. Must turn off 'Create Intermediate Directories'
- To bake out textures must have a principle shader assigned.
​
July 23, 2021 Alembic Cache
​
- Alembic must be imported as geometry cache for light build.
​
July 26, 2021 Normal Issues
​
- The active and inactive RBD elements have mismatched normals.
​
July 27, 2021 Normal Issue Continued
​
- Houdini Engine makes change to normals that do not match FBX and ABC export. Use only for static elements.
​
July 28, 2021 Landscape Materials
​
- Unreal landscape can query Houdini heightfield data and use in the material network, but it must be remapped to correct range.
​
July 30, 2021 Unreal Optimization
​
- Alembic cache needs unreal Stationary light to cast dynamic shadows.
- Sprite texture must be two-sided for proper shadowing.
- Unreal SWARM core number needs adjusting to control heat output during lighting build.
​
July 31, 2021 Unreal Optimization Continued
​
- Bass Pass in GPU Profile identified issues with the amount of geometry and material complexity.
​
August 1, 2021 Unreal Optimization Continued
​
- Turn off dynamic shadow on landscape unless absolutely necessary.
- Landscape UV scale should be one component (grid) x 3. EX 1(64) x 3 = 192.
- UV layout scae in Houdini affects lightmap scale in Unreal.
​
Auugust 3, 2021 Unreal Post Processing
​
- White Balance Temperature : 7100
- Enable Depth of Field
- Exposure Metering Mode : Auto
- Global Saturation and Contrast adjustment
- Enable Look-Up Table
​
August 6, 2021 Unreal Foliage Rocks
​
- Use Unreal's foliage brush to paint rock models onto the lunar surface.
​
August 11, 2021 Unreal Scene Approved!
​
- The Unrela FX scene has been approved for final capture and comparison.
Case Study: Lunar Impact
Terrain Heightfields
Scan data was taken from Terrain Party to be used as the base for the lunar surface.
​
Location: Atacama Desert, Chile
Terrain Heightfields
Scan data was taken from Terrain Party to be used as the base for the lunar surface.
​
Location: Meteor Crater, Arizona
Combine Scan Data
The two scans were combined using heightfield layer to create a crater like moon surface.
Flatten Heightfield
The bottom of the crater had to much of a rough environment. A new flat heightfield was generated to flatten these areas. The original HF was masked by features to flatten specific regions.
Add Noise
The bottom of the crater had to much of a rough environment. A new flat heightfield was generated to flatten these areas. The original HF was masked by features to flatten specific regions.
Base Layout: Paint Scatter Mask
The area for buildings to be placed is a HF painted mask combined with a mask by feature node to avoid any steep angles.
Base Layout: Scatter Points
HF Scatter set to "Total Points using Mask". Unwanted attributes are deleted along with the HF itself. A random orient is generated to have uneven directions of the buildings.
Base Layout: Create Variants
Each asset is packed and given a string variant attribute. The models are then merged and given a string group. When concatenated together each model has a unique identifier.
Example: 1_3 = Group 1, Asset 3
​
Models available from KitBash3D Lunar Base
Base Layout: Copy to Point
The packed geo is now copied to points. Any unwanted attributes are deleted and a few minor manual adjustments are made to the models rotation.
Base Layout: Heightfield Update
Inside a foreach loop each incoming model is check whether it is the reactor model or not. Based on this attribute the model is deleted and passed through one of two channels
Base Layout: Heightfield Update
The HF is masked by the current object. The mask is expanded, and then blurred. This mask is used to vex LERP between the HF's current height value and the P.y value of the models spawn point.
​
If the current model is the reactor the update occurs a second time to account for a variance in the depth of the model.
Base Layout: Heightfield Update Result
The resulting change in the HF after the height update.
Base Layout: Heightfield Noise
A noise is generated and layered back on top to prevent flat ground where the models sit.
Base Layout: Solar Panel Mask
A new mask is painted on the HF and is multiplied by the mask generated from the buildings to prevent the solar arrays spawning inside.
Base Layout: Solar Scatter Points
The identical steps taken as for the buildings.
Base Layout: Create Solar Arrays
The solar panel model from KitBash3D is taken and the bounds calculated. Points are created in the phyllotactic pattern using code based on notes taken from Professor Deborah Fowler. The points then create a starting normal based on their position from the centroid.
​
Deborah Fowler Phyllotactic Pattern:
Base Layout: Update Scatter Pos
To make the solar arrays more self aware they use the same code implemented for the building scatter and adjust their position from one another. Each point then reference the "mask" volume of the HF. If the value is below an inputed threshold the point is removed.
Base Layout: Solar Array Cleanup
Code was taken from within the HF scatter node to create new orientation normals along the surface. These normals are then rotate using the original inward facing normals, and finally the HF normals are set as the UP vector.
The new normals are compared using a dot product and an inputed up vector {0,1,0} and then removed if beyound a specific threshold. Points also self check to see if they are within the bounding area of an individual solar panel and removed to prevent collision.
​
Each building creates a bounding box which is used to group any solar panels that got placed inside and deleted.
Base Layout: Copy to Points
The solar panels are copied to the new point and are grouped based on their spawn point.
Base Layout: Combined Elements
HF to Unreal Test
The first test of importing a HF into unreal.
Method reference from Deborah Fowler.
​
Note: Method works but need to investigate getting smoother values on HF. Currently has stepping issues.
​
http://deborahfowler.com/HoudiniResources/HoudiniHeightFieldsToUnreal.html
HF Map to Unreal Test
Upon further investigation, it is discovered that SideFX has created a node named "Heightfield Output" for the exact purpose of exporting heightmaps to other programs.
​
Check format = "Single Channel"
Type = 16b Fixed
Image Channel = "Height"
Turn on auto remap and set to 0 - 1
Houdini Engine
The Houdini Engine allows the use of Houdini's powerful procedural processing within the Unreal engine.
Houdini Engine Unreal Plug-In:
https://www.sidefx.com/products/houdini-engine/plug-ins/unreal-plug-in/
HF Export
The HF is resampled to have closer to 4k x 4k voxels and cached out.
​
Mask Object
The region requiring impact geometry is painted and converted to polygons, normals are computed, and then any unnecessary geometry is removed.
Mask Object: Export
The top of the mask object is flattened to prevent intersecting when creating the mask. Then it is extruded to create a masking volume.
Import Geo
The initial HF is then masked by the new mask object.
Import Geo Prep: Mask Adherence
The points from the converted HF are compared to the mask of the newly generated mask object and reaped accordingly.
Next the points are rayed to the HF, extruded to give volume and normals created on the new sides.
Import Geo Prep: UV
Edges are selected as UV seams. UV Flatten then uses these seam groups to flatten the UV's. Finally UV layout cleans up and fits the UV's in a single UDIM.
Houdini Engine: HDA
The Houdini Engine uses a Houdini Digital Asset (HDA) to run any Houdini processes.
​
The HDA created "realtime_env" allows users in Unreal to input Houdini read parameters for the desired HF, Mask Object, and Houdini Geo to be used. Along with object scale, and adjustments to the HF visibility radius.
HDA Design: Move to Center
The user inputted HF is masked by the desired mask object. The HF is then moved so that the center of the masked region is moved to the world origin.
HDA Design: HF Visibility
The "mask" layer is expanded or shrunk depending on the user inputted value. The mask is then renamed to "visibility" for use in an Unreal material.
HDA Design: Houdini Geo
The inputted Houdini geo calculates is centroid and then moves itself to the world origin. The HF Transform also references this value.
​
The normals are promoted from vertex to points. The points then grab the normals and determine if they are the outermost points and set themselves in the "ray" point group, based on the dot product.
HDA Design: Edge Ray
The "ray" point group is passed through a ray node using the a "rayIncident" vector as the direction and colliding with the HF mask region.
HDA Design: Prep for Export
Any attributes that will not be needed in Unreal is deleted and an "Unreal_Material" node is created. Finally the HF and geo are merged together and the display flag is placed on the output Null.
Houdini Engine: Import HDA
Import the HDA into Unreal's content browser and drag into the scene.
Right-click and copy reference on the HF materail and paste into the material path of the HDA.
Ensure that the HDA's translations are zeroed out and click rebuild.
Houdini Engine: HF Visibility
Import the HDA into Unreal's content browser and drag into the scene. Ensure that the HDA's translations are zeroed out and click rebuild.
​
Unreal HDA Test
Testing the HDA within Unreal 4.26.1
​
When using the HDA, the seams between the HF and geometry need to be put in lower slope regions or in inconspicuous locations to hide transition.
Offline Camera Test 001
Offline Camera Test 002
Lunar Rocks: Base Rock
A poly sphere is UV unwrapped and laid out.
Lunar Rocks: Noise and Bounds
A mountain node adds irregularity to the model and the bounds are calculated.
Lunar Rocks: Spawn Points
A sphere creates a bounding region where points are scattered inside. Afterward "spawnPT" is set to the point number.
Lunar Rocks: Adjust Position
The same code used on the Base portion is repeated to move the rock spawn points to prevent intersection.
Lunar Rocks: Pscale and Fractures
A new pscale is generated based on a random value and the distance from any neighboring points. Afterward, the number of fractures to occur on each rock is created.
Lunar Rocks: Fracture Prep
A name attribute is generate on the primitives from an inputted name. Next the volume of each rock is measure and the total fracture points are updated based on this volume.
Lunar Rocks: Fracture
Inside a foreach loop.
​
A volume is generated with added noise in the density. Points are scattered within the volume referencing the fracture points of the current model.
​
A rest attribute is created and noise is added to the points. The geometry is fractured and passed on.
Lunar Rocks: Reduce and UV
The rock's "rest" pos is restored and then polyreduced as needed. Interior detail is added and then the newly generated "inside" face group is uv'd.
Lunar Rocks: Pack
The new fractured pieces are packed for RBD efficiency. Afterwards on each point a separation vector is created for later use.
Lunar Rocks: Clusters
After being unpacked an RBDCluster node generates clusters on the fractured geometry. The cluster is then promoted from primitive to point and then packed. A detail attribute is created to hold a list of all the cluster values.
Lunar Rocks: Seperation
The points then fetch their cluster value and use the first instance within the list to grab that specific points separation vector.
​
The points then move themselves according to their separation vector if they are part of the "sep" group. Followed by a random deletion.
Lunar Rocks: Surface Targets
A target region is painted onto the HF lunar surface. An equal number of points as original rocks is scattered within the mask and the PTS number is set.
Lunar Rocks: Target Velocity
Each of the fracture pieces calculates its target velocity by subtracting its position from the lunar surface points that coorelates to its original "spawnPT". The length is calculated and the velocity is divided by this length to give the direction.
​
A user inputed speed is used to multiply the direction vector and replace the removed length.
Lunar Rocks: Activate
The points part of the "sep" group have noise added to their velocity to prevent uniformity.
​
All points are turned inactive. Followed by the creation of a random start frame set as a detail attribute list. Finally each point grabs its start time from the list and sets itself as active when the frame is greater than or equal to that number.
Lunar Rocks: Visual Impact
This setup quickly gives a rough visualization of where each piece will impact on the lunar surface and when it turns active.
RBD Surface: Prep
The lunar surface mesh is moved to the world center. Afterwhich the vertex normals are promoted to a point attribute, and used to create a new Ray point group. The Ray group is then expanded to have a large number of points. Any points that exist on the bottom of the model are removed and finally a noReduce group is created.
RBD Surface: Prep
The new noReduce group is used as hard points in a polyreduce node. A point group named noise is created excluding the ray points. A rest attribute is created to restore the position after fracture. A attribute vop adds noise to the points in the noise group.
RBD Surface: Scatter Volume
The rock impact points are used to scatter spheres to points along the lunar surface. The spheres are then converted into a volume and their intersection with the surface is used to create a scatter volume.
RBD Surface: Scatter
The SDF is converted to a Fog volume and renamed to density. A volume vop adds noise to the density field. Points are then scattered inside this density field and given an ID attribute.
​
This process is repeated multiple times to give the desired fractures in specific locations.
RBD Surface: Primary Fracture
The fracture points and reworked mesh are fed into a vornoi fracture node. Afterward, the mesh's position is restored using the rest attribute. A new noReduce is created for later on in the pipeline.
​
The top surface of the model is then unwrapped and laid out. A divide node restricts the max edges to four to ensure Unreal doesn't complain later when imported. Once complete, any unnecessary attributes/groups are removed, and the geometry is packed.
​
The packed pieces are then grouped based upon the previously used impact points for secondary fracture.
RBD Surface: Secondary Prep
The pieces needing secondary fracture are seperated from the pieces that do not and both are processed separately.
​
Each piece gets a packed number and packed position to track the secondary fracture's original. The pieces are then unpacked and given a total number of fracture points based on their distance from the impacting rocks.
RBD Surface: Secondary Fracture
The pieces are then fed into a ForEach block and the previous fracture process is repeated with the scatter node referencing the total fracture points calculated earlier.
RBD Surface: Secondary Cleanup
The previous cleanup process is repeated again. The model Is unpacked, the position restored, each piece divided to three edges for Unreal, and any unneeded attributes/groups removed.
The pieces are then repacked and merged with the primary fracture pieces.
RBD Surface: Final Cleanup
The completed fractures are then unpacked again. A new uvscale is created on the inside faces to guide the UVLayout of the inside faces around the main surface UV. The model is then divided one more time to ensure all faces are triangulated. Finally any and all texturing attributes are created along with the Unreal_Material node for the inside and outside faces.
RBD Solar Panels: Prep
The solar panel array closet to the impact region is selected and the panels are grouped to be fractured based on the rock expected impact points.
RBD Solar Panels: Prep
Each solar panel then gets a totla number of fracture points based on their distance from the impact sites.
RBD Solar Panels: Fracture
The componenets of the solar panel are broken apart and fractured individually and then merged back together.
​
Note: The method is identical to the secondary fracture method of the lunar surface.
RBD Solar Panels: Fracture
After being merged together the new inside faces are UVLaid out, divided for Unreal import, normals calculated, and cleaned of any unwanted geometry.
RBD Solar Panels: Fracture Result
The final result of the fractures.
RBD: SIM Activate
The hi-rez geometry is removed for later. A new packed point num is created and all pieces are turned inactive. Using the same technique from earlier the pieces are grouped based on bounding region of the impact points to be turned active. Any pieces that are on the bottom are removed from the active group. The final check is to see if any of the piece are part of the outer edge and turned inactive.
​
RBD: SIM Constraints
Note, all RBD elements, lunar surface, solar panels, and impact rocks use the same constraint creation system.
​
The geometry is unpacked and a connect adjacent pieces create the primitives needed for constraints. Glue holds the pieces together, while pin allows a minor amount of bending.
RBD: SIM Network
The RBD simulation network setup.
RBD: SIM Settings
RBD Simulation settings
RBD: SIM Export
The RBD simulation is cached out as points which are then used to animate the high-rez geometry using a transformpieces node.
​
The active and inactive pieces are then separated for final processing. The active geometry removes any unwanted attributes/groups, and is unpacked.
The inactive is converted to a single piece of geometry using the labs destruction_cleanup node. The new single mesh is then poly reduced to optimize for real-time.
​
The two parts are then merged back together, transformed by a uniform scale of 100 and exported as an alembic.
​
These steps are repeated for each component of the RBD sim, lunar surface, solar panels, and impact rocks.
RBD: Unreal Import
The RBD import into Unreal.
POP: Source
The sim geo is remeshed and points are scattered onto the active pieces based on perimeter.
POP: Source
The remeshed pieces are animated through transformpieces and run through the debris source solver. The scattered points are also transformed and adjusted using a pointjitter.
​
The two sources are merged together, and run through a dot product comparison, velocity culling, and positive y position.
POP: Source
The points sources are seperated based on their spawn group. Points are replicated to give a larger amount of particles. A new velocity noise is created and then added into the original velocity.
​
Finally a dead attribute is generated to be used in Unreal later.
POP: SIM Network
POP: SIM Settings
POP: Export
The points are cached out and saved. Afterward, the sim is timeshifted to the correct timeframe. A UV texture givs a UV attribute if needed. A solver node is used to calculate the Unreal dead attribute, and then reap the particles if they were dead the previous frame.
​
The Labs Niagara Rop then writes out an HBJSON file with the dead attribute.
POP: Unreal Import
The Unreal POP Niagara System
POP: Sprite Atlas
The Unreal POP Atlas
POP: Niagara Network
The points sources are seperated based on their spawn group. Points are replicated to give a larger amount of particles. A new velocity noise is created and then added into the original velocity.
​
Finally a dead attribute is generated to be used in Unreal later.
POP: Niagara Module Script
The points sources are seperated based on their spawn group. Points are replicated to give a larger amount of particles. A new velocity noise is created and then added into the original velocity.
​
Finally a dead attribute is generated to be used in Unreal later.
Pyro: Source
The particle sim points are brought in, culled by their age and a new pscale generated. A ForEach block is used to create a custom trail method. An add node creates a curve.
Pyro: Source
The original POP source is brought in and trailed. The points are then merged with the POP Sim curves.
Pyro: Source
An SDF is created from the particles and used in a pyrosource node. The velocity, ramp, and ID attributes are retransfered back to the points. Finally the points are volume rasterized.
Pyro: Source
After caching out the pyro source a volume vop node is used to allow editing of the velocity field.
Pyro: SIM Network
Pyro: SIM Settings
Pyro: Output
The resulting pyro SIM that will be used to guide the particle SIM.
Pyro POP: SIM Network
Pyro POP: SIM Settings
Pyro POP: SIM Export
The particle simulation uses the same export method as the actual dust simulation.
Pyro Sprite: Source
A new pyro source is create to simulate the Unreal pyro sprite.
Pyro Sprite: Loop
The cached simulation is then converted into a looping FX using the Rebelway technique.
​
Rebelway Tutorial:
Pyro Sprite: Atlas
The sprite atlas made in Houdini for Unreal.
Pyro: Unreal Import
Pyro: Niagara Network
The points sources are seperated based on their spawn group. Points are replicated to give a larger amount of particles. A new velocity noise is created and then added into the original velocity.
​
Finally a dead attribute is generated to be used in Unreal later.
Pyro: Unreal Material
The volume material inside of Unreal is critical for the overall loop of the FX. Inside the system is controls allowing the adjustment of light intensity, looping time controls to adjust the animated flipbook texture, and the ability for the material to use the cameras current facing position to switch between two different lighting options.
Unreal: Base Export
The base layout needs to be exported as an FBX file to allow the grouping of the faces using a namespace.
​
The namespace was created based on three components, the building name, the material applied to the face, and the point that spawned the building.
​
When exported, the heirarchy was built based on the name for Unreal.
Unreal: Solar Panel Export
The points sources are seperated based on their spawn group. Points are replicated to give a larger amount of particles. A new velocity noise is created and then added into the original velocity.
​
Finally a dead attribute is generated to be used in Unreal later.
Unreal/Houdini: Texture Baking
Unreal is unable to handle a large amount of geometry needed to create a high detail RBD sim. To address the polycount problem texture baking from Houdini to Unreal can bring back some of the detail.
​
The low poly mesh contains the appropriate UV's while another model is created to have a large polycount and detail baked into the model.
Unreal/Houdini: Texture Baking
A Houdini material is applied to the high and low poly meshes. The material defines the base color, normals, emission, specular, and a noise pattern to be used inside Unreal.
Unreal HE: Normals Issue
The Houdini Engine performed some kind of adjustment on the imported vertex normals.
​
Unreal/Houdini: Texture Baking
The Houdini BakeTexture ROP settings to create texture maps for the low poly mesh based on the high poly model.
Unreal: Base Import
The FBX is imported into Unreal and placed within the level. Afterward, the material is assigned.
Unreal: Merged Actor LOD
All the pieces that have the same material are converted into a merged actor. Then LODs are generated and manually adjusted to use the lowest level needed. Finally, the lightmap resolution is updated to better reflect the necessity of the pieces.
​
Note: This is applied to all non-alembic file assets.
Unreal: Reflection Caputre
Reflection caputres were placed around the main FX solar panels to capture the metallic nature of the panels and their reflection of the surrounding geometry.
Unreal: HDRI Background
An HDRIBackdrop is placed inside the Unreal scene using the star maps available from NASA to create the outer space background.
​
Unreal: Earth Model
Instead of using a flat texture included in the star map, it was decided to create a simple Earth model and texture to better respect parallax and enhance the overall look of the scene.
Unreal: Post Process Volume
A post-processing volume set to infinite bounds adjusts the entire Unreal scene. The volume auto adjusts the gama of the camera, adds depth of field, and motion blur.
Unreal: Look-Up Table
Also included in the post-process volume is the use of a Look-Up Table (LUT) to give a more uniform color correction of the scene.
​
https://docs.unrealengine.com/4.26/en-US/RenderingAndGraphics/PostProcessEffects/UsingLUTs/