Isotropix Forums

Turning on materials causes render to crash

General Discussion about Isotropix and CG related topics

Turning on materials causes render to crash

Unread postby Ken_M » Wed Feb 13, 2019 5:42 am

Hi,
Sorry for the newb question.
My first clarisse tests were creating complex scenes with large amounts of geometry, but being relatively simplistic in the material/texture side of things.

I've gotten more into the material nodes and bringing out lots of 4k maps from my creations in substance painter.

I have a scene made out of an extreme amount of geometry that I can render successfully if I disable the materials and/or use the material override on the 3D layer to reduce the scene to one basic material. This makes me think it's the materials or maps that is the problem.

If I turn on materials in the complex scene clarisse sometimes behaves a little oddly. Previously doing renders in Clarisse resulted in the renderer using 100% of my cores until the render is complete.
With the maps/materials turned on the graph displaying my processor goes up and down somewhat rapidly.. sometimes staying at 75% for a while, sometimes dropping to 30%. The likelihood of getting a crash during the first few percent of the render increases.

Often Clarisse would crash when trying to view the 'image view' with my maps turned on. The scene displays relatively fine in the 3D view with 'simple shading'

In preferences I can change the cache size of 'streaming maps' to a different amount. Increasing it seems to increase the likelihood of a crash. Decreasing it from the default 4gb cache to 2gb tends to give me more stability.
Originally I increased it because I recalled someone else on the forum having difficulty rendering anything until they were asked to increase their cache size.

I just don't exactly know what I'm doing.

If I do a really simple render with only one OBJ and its textures the scene renders fine and uses 100% of my processor until complete.
Is there a superior way to set up materials in large complex scenes?

I have 32GB of Ram and i7 intel quad core.

Thank you.
Ken_M
 
Posts: 59
Joined: Sun Jun 03, 2018 12:57 am

Re: Turning on materials causes render to crash

Unread postby mdkai » Wed Feb 13, 2019 8:39 am

Hey Ken_M

Best way is always to provide a sample scene for isotropix to check for themselves. Any chance you could provide a reduced version that shows similar behaviour?
Are you using streamtexture nodes and converted TX files ?
Are you using shader variables?

Also you could send the log file with a high verbose mode (preference settings)

If the processor goes low it usually is because of IO data. So as long as textures or files have to get loaded first not much else will happen.
Once enough is in cache the cpu can process the data further.
Clarisse can handle large amounts of data with ease so it should not be a problem.

Cheers
Kai
Join us at the Discord https://discord.gg/G5cJj9A
mdkai
 
Posts: 309
Joined: Tue Oct 07, 2014 7:24 pm

Re: Turning on materials causes render to crash

Unread postby Ken_M » Thu Feb 14, 2019 3:07 am

Thank you very much, Kai.

"Are you using streamtexture nodes and converted TX files?"

Yes, I think so... and no... I wasn't.

After seeing your message I found this script to convert my texture maps into TX format and the file seems to render without issue now.

viewtopic.php?p=16892

The TX thing appears to have been the issue.

I had lots of assets exported as OBJ out of various software (such as Daz3d, megascans, spreedtree, substance painter etc) which came with their own texture maps.
When I imported them into clarisse I said 'yes' when it asks if I want it to figure out the materials for me. Usually it figures out the diffuse map at least and sometimes where the bump goes. I have to manually plug in the other maps.

I didn't realize not having all those as .tx format would create such issues.

Something I didn't expect was now some of my assets look more 'realistic' after running the script? Not sure how to describe it...

When Clarisse imports texture maps while importing OBJ files is it assigning those textures as 'maps' or 'streaming maps'?
Does me importing the tiffs and PNGs out of substance painter as 'streaming maps files' also creating issues?

That's a good idea about uploading a project for isotropix somehow. I checked the folder for my project and after running the script I'm a little over 100GB in referenced assets and image maps/files now (including geometry.). I'm not sure how to transfer something like that.

I spent last night/this morning trying to create a smaller project to resemble the same issues, but it only seems to appear in large-scale projects that have been worked on for a long time and have an accumulation of lots of assets/materials/folders.

It does appear that the .tx thing was the problem.

Thanks for diagnosing that.

Thanks very much for the help!
Ken_M
 
Posts: 59
Joined: Sun Jun 03, 2018 12:57 am

Re: Turning on materials causes render to crash

Unread postby mdkai » Thu Feb 14, 2019 12:36 pm

Glad it helped !

Stream format wants to have a stream"able" format which .tx is all about. The conversion is basically doing multiple levels of automated mipmapping to be able to pick the best based on camera distance, which reduces file IO drastically.

Something I didn't expect was now some of my assets look more 'realistic' after running the script? Not sure how to describe it...

oh no how did that happend :)

The import is very straight, not much assignment or conversion happening.
The forum got a script which allows to mass convert textures into TX files .. keep in mind that the tx conversion should be adjusted for the texture type you are using.
Basically aALL tx files are assumed to be linear, to keep things simple.

So take care of which files you convert:
Colormaps like diffuse or specular color are usually sRGB and will convert to linear
however technical maps like normalmaps, bump, roughness, occlusion or any mask type is usually already linear and requires no color conversion.

Once done all tx maps are ready to be used with the streamTexture node.
There is more detail about the TX tool, like checking for NaN values or to convert equirectangular maps . etc
Check some of the official site or Arnolds take about how to use it . it applies also for clarisse
https://docs.arnoldrenderer.com/display/A5AFMUG/Maketx

Cheers
Kai
Join us at the Discord https://discord.gg/G5cJj9A
mdkai
 
Posts: 309
Joined: Tue Oct 07, 2014 7:24 pm

Re: Turning on materials causes render to crash

Unread postby Ken_M » Sat Feb 16, 2019 12:18 am

Thank you, Kai.
That's some great information.

When you convert to TX to do you also keep the original tiff/jpeg/png maps in case you have to edit them later? - and then run the tx script again?

Or do you only keep the .tx files have some program or plugin that opens them for edits outside of clarisse? I tried to open the .tx in photoshop, but it doesn't open.

Thank you,

Ken
Ken_M
 
Posts: 59
Joined: Sun Jun 03, 2018 12:57 am

Re: Turning on materials causes render to crash

Unread postby mdkai » Sat Feb 16, 2019 12:30 am

the tx file format is often to some extend a tiff with mipmap levels (not with exr and other float based format).

The format is made for the endresult , usually that means you keep your source files for editing. In Arnold that feature is even automated (unless you deactivate it).

For personal projects i adopted a simple folder structure for my workflow which is
00_input - usually references and things to start with
01_scource - actual workfiles i use to create content , like a PSD file
02_wip - hold all states i did work for at least couple of iterations (this can later be deleted)
03_current - Holds the current latest version of a file (including tx files)

This is not the only way to do it, but for me at home, more than fine..

Cheers
Kai
Join us at the Discord https://discord.gg/G5cJj9A
mdkai
 
Posts: 309
Joined: Tue Oct 07, 2014 7:24 pm

Re: Turning on materials causes render to crash

Unread postby Ken_M » Fri Feb 22, 2019 4:06 am

Thank you, Kai. That makes a lot of sense. I appreciate the info about your workflows.

Cheers.
Ken_M
 
Posts: 59
Joined: Sun Jun 03, 2018 12:57 am


Return to General Discussion