Tips & tricks for working with large scenes

danshewan

Hey guys,

So, I'm still plugging away on my Japanese temple scene, and I decided to completely remodel several parts of the scene, and right now I'm working on the steps.

I tried exporting a base mesh from Max and sculpting the steps as one object in ZBrush, but it wasn't giving me the look I wanted - specifically, the deep cracks between each step slab were difficult to achieve convincingly when sculpting one single object. So, I thought I'd go with a more modular approach and sculpt two or three single step slab bricks, and assemble them in Max. This is where the problems started.

I optimized each of the three bricks in ZBrush so that they retained some of their detail, but were (I thought) still manageable. Right now, each step slab is around 12k.





So I bring them into Max, and start instancing. After some resizing and rotations, I end up with a pretty decent result.





However, now I'm at the stage where I need to bake this down in xNormal. This is where things get frustrating. Each step is still an individual object (Editable Mesh).

I converted the first step to Editable Poly and tried attaching each subsequent step, and Max crashes. I tried using Attach List, and Max crashes. I tried selecting all objects and exporting as .obj, and Max crashes. I tried selecting all the objects and applying an Optimize modifier, and guess what? Yep. Right now, I'm considering trying applying an Optimize modifier to each individual step, then trying to Attach them, but I'm not holding out much hope.

Any ideas (aside from 'buy more RAM', which isn't an option right now - I'm running a paltry 2GB) on how I can get this SOB into a single object, or bake this down without blowing up my machine?

At this point, I'm considering going back into ZBrush and sculpting at a much lower resolution to reduce the scene size, but I didn't really want to do this as the detail is pretty much as low as it can be without losing definition.

Suggestions appreciated.

MoP

What's wrong with just selecting them all and exporting them to a single OBJ file?

I don't get why you want to Attach them all or merge them into a single object in the first place?

If you want an easy way to select them all then you should do one of the following:

1. Put them in their own Layer then just select them via the layer manager.
2. Make a Named Selection Set for them.
3. Group them.

All of these would be very simple and fast basic scene organisation methods, much preferable to trying to Attach a whole bunch of objects together which don't need to be attached.

danshewan

Thanks for the swift reply, MoP.

They are grouped for ease of selection (I saved the steps out into their own file), but Max is still having a coronary trying to export the OBJ.

At this stage, I'm going to try splitting the object up into several sections, and export them out individually. Max just simply can't seem to manage anything else. I wanted to reuse the geometry for the steps to the left and right of the building, and thought that using a Quickslice to trim the front steps down to fit the side stairs would be relatively easy.



This way, I'll be able to use one of the modular OBJ's for the other stair sections, and hopefully Max won't explode.

Thanks again.

MoP

Hmm, I guess if you have 7 rows of 12 stairs each at 16000 triangles then that's already about a million triangles in total.

Are you using the Guruware OBJ exporter?

danshewan

MoP wrote: »

Hmm, I guess if you have 7 rows of 12 stairs each at 16000 triangles then that's already about a million triangles in total.

Are you using the Guruware OBJ exporter?

Yeah, and it's still pitching a fit.

eld

run it through optimizer, the drop in detail wont be noticable in the bake, especially since its a rather random stone material.

pior

Why do you subdivide them that much anyways ? This little micro detail layer will never be noticeable on the bake (roughly 1 pixel worth detail) and doesn't really looks like stone anyways (weathered stone stairs = smooth)

danshewan

eld wrote: »

run it through optimizer, the drop in detail wont be noticable in the bake, especially since its a rather random stone material.

I'm going to have to - the scene is holding Max hostage, and I can barely do anything without it crashing.

This is the decimated slab in ZBrush.



Do you think this will still give me a decent result? I'm not trying to focus on microscopic detail that won't be seen in-engine, but I guess this felt a little brutal. Still, I should be able to work with the scene if I use this resolution.

Thanks for the feedback.

Kevin Albers

I'd suggest decimating the blocks, but maybe less drastically than the image above. Also, you can delete all the polygons that won't show up in the final texture (bottom, back etc).
Good luck with it. I ran into this same issue just a couple of days ago, and didn't really come up with a solution that worked great for me.

One thing that can help with putting high poly scenes together in Max is setting objects to 'display as box'. This won't help with your problem, but in general can help with managing high poly scenes so you can move things around without bad framerate issues.

MoP

run the max scene in bounding box view mode for faster results - then import one of your more optimised meshes, and use the Align tool to replace the old bricks with the new ones.

Or, if your bricks are all instances (which they should be), you can just Attach your more optimised brick to one of them (in bounding box mode or isolate select mode!) and then delete the original higher-poly geometry from that object (use Select By Element mode). That way all your instances remain intact and updated with the new low-poly geometry. Should be pretty fast to do.

Blaizer

You can have all the ram of the world, but in Max, if you have a gamer card... you won't do anything reaching the limit XD. That's why i work with quadro and Maxtreme. My Bro have a GTX285 with OC, and it's a huge shit in Max, literally. His Max collapses with files i load in a sigh.

All the micro detail could be added to the mesh using it in a bump slot, so you won't lose all the work you put on a model. heh, In 3D, you CAN'T work in "macro mode" man XDD, you must adapt yourself to your hardware. As tip, all those bricks could have been builded using subdivs, instead of raw meshes from Zbrush, with less polygons and far better than ugly triangulated meshes with bad smooth (result of the optimization). working with subdivs it's also great for renderings. Very optimized meshes does not give good results.

What i see is what i call to work in the craziest way possible XD, that's gross man!. I can load objs with 4+ millions polygons (when i need it), but it's better to optimize the mesh using decimation master or something like polygon cruncher. Objs will load faster and all will be better. One model with 2 or 3 million of polygons should not be a problem.

Ark

Get xsi, its really stable, theres a reason Blur use's it for layout.

As Pior says, all the small details never carry over into the bake, the decimated shot you have above with the 'crackly' edges will read much better, especially when its been compressed.

I think the decimated slab you've shown should be fine for generating the normal map, atleast you will get the shape correction from the bake. Then you can just overlay some detail normal maps in photoshop for the surface detail.

danshewan

Thanks for all the feedback, everyone. Definitely a good learning experience, albeit a frustrating one initially.

How the steps look now - 284k tris, in the original scene file.




Much less hassle, and less than a quarter of the cost.

Thanks again for everyone's input - I appreciate it.

Mark Dygert

I think that will give you horrible results if you're planning on baking it but only testing it will tell.

Max runs better if they're all separate objects, it can only degrade the visuals of the scene on a per object basis when it starts to have trouble few objects to degrade the more it trouble it has.
http://www.vigville.com/forum_images/AdaptiveDegradation.jpg
First it simplifies the material, then drops it to wireframe mode then displays only the bounding box.

When merging objects don't add a huge list but a few at a time and give it a little bit to digest and clean up the scene after the merge. If your HDD is still making noise even after max gives you back control wait until its quite.

You can help it degrade by turning on the bounding box only for each block so that's the only thing visible in the viewport at all times. When importing be in wireframe mode instead of shaded. Also make sure backface culling is turned off.

It can also help to go in and turn down your texture display settings Customize > preferences > viewport Tab > configure Driver.
Turn off "Match Bitmap Size as Closely as Possible" and turn it down to 128.
Same for "Download Texture Size".

If you're running 2010 make sure the graphite modeling tools ribbon is fully hidden.

To go a step further you can move some plug-ins out of

danshewan

Vig wrote: »

I think that will give you horrible results if you're planning on baking it but only testing it will tell.

I won't be able to test bake anything for an hour or two, but why do you think this will yield horrible results? Is it the geometry itself, or the arrangement, or....?

I'm still very new to the baking workflow, and I'd certainly value any insight as to what I could be doing better.

Mark Dygert

Blaizer wrote: »

My Bro have a GTX285 with OC, and it's a huge shit in Max, literally. His Max collapses with files i load in a sigh.

That's his problem. He's over clocking a card thinking he can squeeze extra performance out of it but he's really just making it unstable.

Cards and processors have red lines for a reason. A GTX is an overclocked GTS that can handle the extra juice. If it can't they label it a GTS and sell it for a little less. By over clocking it to the point it fails repeatedly hes damaging the card and there isn't anything he can do to reverse the effects. But he can make it more stable by dialing it down so low its pointless. The more it fails the more you have to dial it down.

At the first sign of instability he should have turned it down. Now its probably too late.

Think of the processors as having billions of little rooms that hold electricity. These rooms have walls that are rated to hold back a certain amount of juice. Once they fail and the wall breaks that extra juice cascades into the other rooms, which puts stress on the others walls and breaks them down also.

It's like killing brain cells only the killed cells actually don't just die off they turn and attack the living by channeling the juice they normally store through their broken walls which breaks more walls and makes more zombie cells.
The only way to get stable performance is to dial it back to a point where the walls that are left can hold back the electricity. Which means by trying to squeeze out a few microns of performance he actually made his card worse than a 3dfx Voodoo...

The stability is only going to get worse and he shouldn't blame the software or the hardware vendors for stability. If you overclock and

Eric Chadwick

Why not just bake it in Max.

Mark Dygert

danshewan wrote: »

I won't be able to test bake anything for an hour or two, but why do you think this will yield horrible results? Is it the geometry itself, or the arrangement, or....?

I'm still very new to the baking workflow, and I'd certainly value any insight as to what I could be doing better.

I think if you turn off faceted view there might not be much to capture. There doesn't seem to be much variation in the surface. But it might make a few peaks and valleys. You might want to exaggerate the detail in the sculpt in order to get a normal looking bake. You always lose a little bit of detail and height when baking.

You don't want to sculpt micro details because they'll probably get lost, and you want to exaggerate details to make sure they show up. It's easier to wash details out of a normal map than it is to try and pull them out later.

teaandcigarettes

Hey Dan, I did not read the whole thread, so if anything has been already posted please disregard it.

Have you thought about changing your approach and creating a smaller, tileable staircase instead? Just as in the picture you posted before, except instead of creating 3 pieces you could easily achieve the same effect by creating one, tileable piece. You might find the modular walls I made interesting: http://i49.tinypic.com/6hkr9i.jpg

Basically, I have aligned the edges of my wall so that they snap to the grid. Then I have created a second row of bricks, except I moved it two squares further. Then I have created a third row, in the same way as the first, etc. This way you get a nice, repeating pattern: http://i48.tinypic.com/nv1p3c.jpg

You can easily use the same technique for your model - you will get some steps sticking out at the ends of your full staircase, but you can easily hide them by placing walls, railings, etc. Then you can easily create the lowpoly by using the Unified Skin modifier in Zbrush: http://boards.polycount.net/showpost.php?p=1037418&postcount=71

By using smaller, tileable pieces you can get a better texture resolution and more details. Also, once you align the steps in Max you can bring it back into Zbrush for further detailing, to add more variations.

If you don't want to follow this, you might also try making lowpoly steps in Zbrush (5000-6000 tris should be enough to create nice edges), bring it into Max, align and attach all steps together, bring back into Zbrush, subdivide, add more details.

I hope this post will help you in some way.

danshewan

Thanks for the suggestions, Tea. I'm going to see how the bakes come out before messing with it again, but there was definitely some useful information in your suggestions and the links you posted - would have definitely been useful prior to my starting on the steps! It's definitely been a valuable experience, and hopefully this thread will help others with similar problems.

Here's how the high-poly steps look alongside the rest of the scene. Not sure how the bake will come out, but I guess we'll see.





Thanks again for your suggestions!

Blaizer

Vig, his Videocard has OC from manufacturer, it's one of those versions with more power, you know , better for gaming. The graphic card works flawlessly, but for heavy 3d work is not very good, it's useless, it's just for games (to play them at maximum specs on 24").

MoP

Weird, I have a GTX275 gamer card (no overclocking) and Max is running great. 4-6 million poly models in and out all the time with no crashes.

I think maybe something is wrong with his card or other parts of his computer (or his Max version )

teaandcigarettes

That turned out pretty cool. Can't wait to see the bakes.

gamedev

Why not just bake it in Max.

I'm with Eric. Max can push a ton of instanced geo. Why bother exporting when you can bake in max?

danshewan

I will be now, since I reduced the count pretty significantly. Still matching the low poly geometry to the high poly steps right now - will probably post updates / bakes to my WIP thread.

Thanks again for everyone's feedback.

MoP

I don't bake in Max mainly because I find the results of Max's baker don't look quite right in most circumstances. I can get a more accurate bake in XNormal or Maya. Those basically make the normal-map look perfect in their viewports, whereas the maps Max bakes do not look perfect in the Max viewport... for some reason.

I will try and post some comparison images soon if I can.

albino_goldfish

@MoP

you posting those here? cuz I would love to see some comparison shots, Im still newish at baking, I was wondering which program would be better to learn straight off the bat, something like that would help!

Mark Dygert

Is it more or less an issue with the viewport display not showing it correctly? Or is the bake actually off in some way?

I've never been able to trust any viewport preview, it always looks off in max and maya. The only time I've gotten it anywhere close to what it should be (in max) is when using shaderFX and then using that same shader in engine, even then it was off...

MoP

Vig: Maya bakes have always been flawless for me. They seem to use the exact same tangent space when baking and displaying, so it's all good.

It's almost like Max doesn't use the same tangent-space derivation when rendering compared to the viewport, so as a result it never looks quite right. I might do some tests by baking in Maya and seeing how those maps look in Max's viewport - if they are correct then I'd say it's Max's baker that's off in some way.

I've been pretty ill the past day or two, but will try to post images tonight.

EarthQuake

The problem is that *every* app calculates tangents just a little bit differently, but atleast maya and xnormal can display normals rendered in the respective programs correctly(or "mostly correct"). I'm pretty sure that max renders specifically for offline rendering, and uses a different method for realtime display, basically making max's RTT normals useless for games work.

System

With you on that, max is hit and miss, sometimes green channel is inverted, sometimes it isn't and other times the normals aren't what you would expect. I use xnormal too, much quicker baking ambient occlusion than max and as said predictable results.

Mark Dygert

Thanks for the info guys I'll have to give Maya another look, I always assumed it was just as bad as max it would be great if its a bit more accurate. But accurate being a bit of a relative term I guess, relative to the engine being used?

EarthQuake

Yeah you still need to have your engine synced up to whatever results you get from the baker, so it doesnt really mean much if your engine is doing it differently too, you'll get poor results from everything then.

MoP

GCMP wrote: »

With you on that, max is hit and miss, sometimes green channel is inverted, sometimes it isn't

That'll be your settings at fault. Check the "Orientation" section in the "Options" dialog of the "Render To Texture" window. It doesn't just change randomly like you're implying, but it does save the setting per object (if I remember right) so if you're changing it for some but not all renders then that'll be why it's different.

Edit: I've moved my full-on normal-map response over into its own thread since I think it's needed.

System

Yes, it's set to down for green so I usually flip the channel after baking. afaik settings are loaded correctly from the ini file, also I never change the orientation but still get different results, strange.
Checking your new thread now, thanks