max 2010 - mental ray and memory

Lonewolf

heya

im trying to render something really high poly using mental ray and i get this error

"The render was canceled due to insufficient memory, Adjust the scene or render settings to reduce memory usage and try again"

i have 12GB DDR3,

and i was wondering if its something i can fix if i can increase virtual memory in mental ray but i donno if there is an option like that

i remember i had that option in Vray and i worked

any idea?

Firebert

that's bananas!
you have any textures in there?
running with FG/GI, no FG/GI?
have any stuff hidden in the bkgd that you don't need any more, unnecessary stuff?
if you're running windows, check your page file size through your computer properties and adjust it through there (though this most likely will not really help or change anything, it might)
does it render at low res settings in MR? does it render in scanline?

DarthNater

I also get that issue with Mental Ray, of course I only have 3 Gig's of memory I always check that 'conserve memory' button and that doesn't seem to help at all (what does that do anyway?)

Firebert

not sure what conserve memory button does, but i do know that messing around with BSP settings will really help a lot... there's been a few instances where my scene was very massive and the only way i could get it to even render was through BSP adjustment... that was back when i was running only 2gb.... here's a good page to check out on BSP at LAMRUGif you haven't already...

DarthNater

I consider myself a pretty technical guy, and that link just melted my brain LOL Thanks though, what I could understand helped a bit, I'll have to find a scene that won't render in MR and give that a try!

divi

conserve memory puts stuff that's not used in the current bucket onto the harddrive. if you are using textures you could try the bitmap pager, if not try reducing the bucket size.
if that wont help either you could just render out regions and stitch them together in photoshop.

Firebert

divi wrote: »

you could just render out regions and stitch them together in photoshop.

if you're running FG, render region/stitching won't work as FG is calculated per regional view and will end up with visible seams, even with some blending. if FG isn't on, then this will be less of a problem

CompanionCube

whats your pagefile showing in the task manager? what material & lighting are you using? using BSP2 might help. in 2010 they removed the memory options because its meant to do it under the hood, but i use this script called Ram Optimizer V1.3 ( bottom of this page http://www.infinity-vision.de/ShaderUtilities/eng/ )

Cyrael

I'm not sure how max works, but in maya a lot of times when there is insufficient memory for rendering a still if you choose a place to save the still and batch render it will work - I was actually having this problem with my current project.

Lonewolf

thanx for the replies guys hmm.. how exactly i check that page file? lol


and yea i render with GI, no textures

CompanionCube

check the pagefile in the task manager to actually see how much memory its using when it says its out of memory. whats your lighting setup, like what type of light and shadows? are you using just GI, or FG & GI or just FG? whats your settings for the FG/GI because you might have them set too high? what type of material are you using? whats the polycount in the scene? you can try render the FG and or GI first and save them as FG/GI maps, then restart max to dump any memory its hanging onto then start max again and render the image with the saved maps. need more info basically, shouldn't be running out of memory if you got 64-bit max installed on a 64-bit machine with 12GB, means your doing something wrong.

Lonewolf

when u say "page file" iu mean just look how much memory is used under the "memory" tab in the task manager?

Firebert

what he means is the performance tab in the task manager. gives you a meter on the left hand side as to what the current memory consumption is.
it would also be beneficial to look in the processes tab and take note of the current memory consumption state before you render, which would just be the program running at it's regular state.

will it render without GI? without lights? with lights but without shadows?

there's just something in the scene, flat out, that is keeping it from rendering and i doubt it is your geometry.

try merging into a reset scene and see if that works. only merge in some of the objects to start.

keep hacking down even the silliest possibilities.

i've had scenes that wouldn't render because the shader just didn't agree with me for some unknown reason. merging into a reset scene would fix this 90% of the time.

Lonewolf

i hid one of the objects and it renders, then i looked at the memory and it was like 11.5GB O_O

i guess i need more memory

=\

System

To add to this I find it helps if you freeze operations on all objects and obviously turn off edged faces It could be that max may be using around 9-10 gigs of that 11.5 and Vista is chewing up the rest, Vista right? There's ways around that though:poly142:

ps: for me 8 gigs ram = around 8 million polys render in mr. How many did you have?

Lonewolf

where can i see how many i have during the render?

Lonewolf

just checked

57 mil


and thats just 1/4 of what the final model should be

AlecMoody

man I want to see that when its done. I havent ever worked on anything larger than 6 mil.

disanski

57 mil wooww man. Is this even possible

CompanionCube

what are you rendering? if its a model out of zbrush or something there is no need to have that polycount, use decimation master and get the polycount down by about 50-75% and you won't even really notice a difference from the original. i just rendered a high poly model that ended at being just under 60 million in zbrush, but after decimation master got it down to just under 20 million so i could render it in max.

Lonewolf

its a transformer im working on, well trying to build one, so all mechanics with TSx2



looks like im gonna have to buy more ram

Cyrael

even if you didn't build it in zbrush - bringing it into zbrush to use decimation master is a great option and money saver - decimation master works awesome at reducing poly count without it being noticeable.

Firebert

dude, come on.... 57 mil... seriously... let's talk like in a form of reality that we all can agree on, which is not in the realm of 57 mil.

if everything is at ts2, then go back and really analyze what details really need that amount of smoothing and what objects could do with less, and what objects could do with some better optimization to achieve the same result. unless you are printing out a 5ft tall to lifesize print of this thing, i highly doubt everything in your scene needs to have that much smoothing on it.

ILM created a total of 60,217 vehicle parts and over 12.5 million polygons for all of the 14 robots that were included in the movie.

Even though Barricade is one of the smaller-sized robots, he still possessed an impressive arsenal in terms of geometric pieces (3,236), polygons 644,528

The most complex robot of them all is Optimus Prime with 10,108 pieces of geometry, and 1,830,898 polygons. Prime’s face alone has 200 moving parts with 30 facial controls. An animator has the chance to move any piece. If one were to lay all of the pieces in a straight line, it would total to 6,070 feet in length.

The most complex Decepticon is Blackout with 7,742 geometric pieces, and 1,426,781 polygons. This is in stark contrast to the most simplistic robot Frenzy, who has 871 geometric pieces and 233,178 polygons.

sama.van

Lonewolf wrote:

"The render was canceled due to insufficient memory, Adjust the scene or render settings to reduce memory usage and try again"

I got the same problem importing a FBX from Maya...

Reason was about Uv set : some of polygons from my object was not attached to the second Uv set I created in Maya...

Object was the result of a combine with multiple objects and some with a 2nd Uvs set and some without...

... and it will create the same error in 3dsmax when pressing F9.

I already committed the bug last week on the Autodesk website but in the FBX section...
You could create another bug entry with your version of 3desmax.

CompanionCube

Lonewolf wrote: »

its a transformer im working on, well trying to build one, so all mechanics with TSx2
looks like im gonna have to buy more ram

12GB is loads, you don't need more, you just need to rethink the way your modeling

frubes

create MR or Vray proxies of your objects, save them down and xref them into a render pass file with just your render settings in. Not only does this reduce your file size massively but it makes it far easier to process by calling in the files from external locations. We render billions of polygons with this setup so it should work for you just fine. Also, use the correct BSP settings as someone mentioned, it will make a big difference to your memory management.

Lonewolf

yea im sure i can optimize it more and prob be good with x1 TS too

also im sure those stats reffed to trasnformers 1, with had less detailed robots and not very smooth cylinders

where did u get that info? is there anything like that about transformers 2?

CompanionCube

i think you just need to stop modeling everything in a way to be subdived, theres alot of polys being created by the TS that aren't needed. you could just be using chamfers and bevels, or even collapsing the model with TS x2 and then removing all unnecessary edge loops quickly.

in your TS settings have you set the iterations to render time only so your not wasting RAM displaying them in the scene?

Firebert

it came out of a 3dworld mag article which was from transformers 1 ...

here's info i found on transformers2

TechRadar.com: The biggest transformation is unlike any seen in the previous film, and it's for the biggest Decepticon ever. Standing 100 feet tall, the Devastator has 13 million polygons and 52,632 parts. Moreover, he's formed out of six separate robots.

the proxy idea could work. i've never tried them before so it never even crossed my mind, but it would be worth a shot.
regardless, it would be worth it to go back and optimize a few things, larger pieces that could make a big difference... i mean ts1 doesn't really get great results, so i hear where you're coming from... you gotta keep that quality up as best you can... there's probably some small pieces that you could actually roll with some simple smoothing groups on though, and those little buggers eat up some damn poly count some times without you even realizing it.

frubes

In films, shaders are more often than not used to create the edge fillets and chamfers that you are trying to achieve by subdividing the mesh. Obviously, things such as displacment also reduce the need for such extreme amounts of geometry, so whilst those poly counts are an indication of how they achieved the modelling it doesn't tell the full story. Bare in mind, chamfering in a shader only works on technical models and not so much organic ones where the face angles are variable.

However, the proxy idea i mentioned earlier should have no problems rendering your scene.

Firebert

touch

Lonewolf

thanx alot for the info guys

i can just smooth the faces and it will prob look almost the same

i just love smooth parts lol so i use TS alot even when i dont need


and yea the like u said, the base is good enough to go

Lonewolf

im surprised about devastator though, only 13mil? :O

it looks like 130mil lol

DarthNater

Lonewolf lets see some of these models you're working on WE HAS TO C DA ART!11!!!!

AlecMoody

I knew I recognized your name. I used to have your lr300 for cs

Lonewolf

haha thanx guys

ill show when ill have something good xD

Lonewolf

Lonewolf wrote: »

haha thanx guys

ill show when ill have something good xD

http://fc03.deviantart.net/fs16/f/2007/165/5/4/Optimus_Prime_Reference_Image_by_Tophoid.jpg

bumped into this pic and remember about what u guys said about the polycount

its really big pic and u can see the smallest details clearly

without TS, u would find some sharp and edge element but i cant find even 1

it like every part have TS O_O


this is really incredible work once u study every parts of it

MoP

Lonewolf: Or the model is just using Mental Ray shaders with the "round edges" capability enabled. Then you don't need turbosmooth or sub-d, it just naturally rounds off all the sharp edges, even when objects intersect.

Well worth looking into if you want to save work for highpoly renders.

r_fletch_r

Are you still having this problem? I posted a script a whileback that allows you to manually increase the memory mentalray gets to render with. it used to be a UI option prior to 2010 but they hid it in favor of an automatic method.

http://www.robertfletcher.org/public_files/Scripts/set_memory_limit.ms

It fixed the problem for me

Lonewolf

MoP wrote: »

Lonewolf: Or the model is just using Mental Ray shaders with the "round edges" capability enabled. Then you don't need turbosmooth or sub-d, it just naturally rounds off all the sharp edges, even when objects intersect.

Well worth looking into if you want to save work for highpoly renders.

i cant find that option in my max (2010), can u post a screenshot of where it is?

=\

i think it calls round corners

that might be cool stuff

thanx

Lonewolf

r_fletch_r wrote: »

Are you still having this problem? I posted a script a whileback that allows you to manually increase the memory mentalray gets to render with. it used to be a UI option prior to 2010 but they hid it in favor of an automatic method.

http://www.robertfletcher.org/public_files/Scripts/set_memory_limit.ms

It fixed the problem for me

im gonna try it and let u know

thanx!

MoP

Sorry, yes, I meant "round corners", not "round edges". My bad.

Lonewolf

ya know where it is in max 2010?

cant find it

r_fletch_r

um.. the script

round corners is part of the mentalray A&D material. found under special effects. A good scene scale helps allot when using this feature.

Lonewolf

Can anyone post a screenshot of where it is or tell me how to get to it?

I can find it =\

MoP

Lonewolf: Here you go.

r_fletch_r

there are 2 ways to get the round corners.

1: roundcorners node in the bump slot of a standard mat.
Its a hidden shader so the quickest way to get it is a script

--Create a Standard Material with a round corners shader
mat = standardMaterial()
mat.bumpEnabled = on
mat.bumpMapAmount = 100
mat.bumpMap = Architectural__Round_Corners_Bump ()
meditMaterials[1] = mat

save that as a script or drag it onto your ui to create a button out of it

2:Use Arch & Design shader
Change the Standard max material to Arch & Design. Just click on where it says standard in the material editor and select Arch & Design. Under special effects is the round corners option.

The Script:
Download and run the script.
Go to customize UI - > Toolbars
Chnage the category to "RT Render Tools"
Drag 'Set Mentalray Memory Limit' onto your top bar.

MoP

Bah, r_fletch_r is far more helpful than me
Thanks for the write-up!