Zbrush 21 mil subtool Not Enough Memory OVERRIDE!
polycounter lvl 17
How to combine 32-bit displace maps + Not Enough Memory OVERRIDE
Displace Map Generation - 4K 32-bit EXR with highest settings - from one apparently too large subtool = ~22 million polys gives the error Not Enough Memory with Multi Map Exporter, despite my PC having a lot more than 4GB RAM.
(I don't want to retopo into smaller parts so it will give subtools having not so many polys and reproject again and again...)
Windows Page file size doesn't matter: you can read a post by an experienced user here:
http://forums.cgsociety.org/showthread.php?t=1131761
So if I save just this mesh w. 8 subdiv levels and load it as a subtool in an empty project file, still Zbrush gives me not enough memory all the time. The displace map must be 32-bit EXR, no 16-bit fake crap allowed.
Here is, how you can override the "not enough memory" bottleneck:
Concept:
You need to lower your lowpoly model's poly count, without changing the model.
About ~70% of the 22million polygons will fit into the 4GB memory that Z4R6 can handle. We must select a couple of the neighboring UV Islands and go over this selection using a couple of Grow Selection button presses, so Zbrush will bake the displace map over the selected border as well. Split your model physically so the unselected region will be cut, reducing your polygon count at the highest subdiv.. Now you can click on Generate All Maps. Zbrush will no longer display not enough memory.. Then we need to select the other half of the lowpoly mesh and repeat above.
Go to Subdiv level 1.
1. The subtool is an eagle body (without the large feathers), cut into multiple UV islands.
2. Polygroups--> make groups from UVs = instant polygroups per UV islands!
3. Since the memory will be enough for an about 15 million poly subtool, I select about 70% of the model using the polygroups/UV islands, then I click on Visibility -> Grow about three times and the polygon selection grows by 3x rows reaching over the border of the selected 70% (lowpoly at subdiv 1), this gives a good portion of the full lowpoly model selected. This will be the first half. You must hide the rest with CTRL+SHIFT+ALT poly selection using lasso = this hides unwanted polys
4. By this time all the unselected polys are invisible
5. Polygroups--> group visible = places all visible polys into one polygroup
You can unhide all polys now.
6. Go to Highest Subdiv level, mine is 8 at ~22 million polys
7. Subtool --> split --> Groups split = this will physically CUT the model into little new subtools defined by polygroup borders (delete the newly created little subtools, belonging to the other half of your model)
8. Displace map will now generate properly with Multi Map Exporter
9. You need to repeat the displace map gen for the other half of the mesh, using the above method: make sure you go OVER the border of the other lowpoly half (at subdiv 1) too by a couple of Grow button presses!
(10. Going over the border by one row of polys for both halves is necessary, otherwise there will be a big seam / chasm during displacemap rendering: try it out by just splitting your large subtool into all UV islands and generate a displacemap for two islands sharing a UV seam.)
11. Now you have two 32-bit Full Float EXR displace maps.
12. The question is, how to combine these displace maps so you largely create the same quality displace map (32-bit R Full float EXR) as Zbrush creates from one subtool.. but this time merging them by hand so it looks almost the same as if Zbrush did it...
13. Install ProEXR for Photoshop. Open one EXR file: I selected "alpha appears on separate layers". Generate a solid color UV Template render in MAX (Unwrap UVW modifier, Open UV Editor, Tools menu, Render UVW Template). Load this BMP in Photoshop you can select all the UV Islands with the Magic Wand tool (in 8 bit mode!), save the selection and load it in the EXR file, increase the selection border by about 10 pixels (for 4K Disp map) then do a Layer via Copy. Drag the other EXR into this file as a new layer. Repeat the selection process in the solid color UV Template file using MAX 3D viewport and Zbrush to see, which UV islands you need to select for this other half of the model and save this selection too to load it into the first EXR file to use it for the second layer you just dragged in (use SHIFT+Drag to center the image precisely).
Increase the selection border by about 10 pixels (for 4K Disp map) then do a Layer via Copy again. You now have two layers via Copy containing only the good displace map data for all the UV Islands of the model. Delete the two original EXR layers. Make a new layer for the background and fill it with black, place it to the bottom and flatten the file. Save the EXR file, tell ProEXR to use 32-bit data and Zip compression.
14. Get Nuke and pull in a merge node, check the input R G B channel boxes for all of your EXR files - naturally you must have divided your lowpoly into separate meshes and created highpoly sculpts using these parts or "subtools" and have rendered EXR displace maps for all subtools so you have multiple EXR files by this time. Check only the R channel box for output, because Zbrush creates beautiful R Full Float 32-bit EXR-s too and it has less size, because the G and B channels will be empty. Pull in a Write node, select EXR file type as output and also name your file using .EXR as extension.
Press "rendering" in NUKE and it will write a beautiful main EXR file, just as Zbrush 4R6 would have done it, if the Merge Maps for All Subtools button would work and if it could handle more than 4GB memory..
Use this new 4K or 8K or 16K EXR to render your lowpoly model in MAX with displacement mapping. Have fun!
Displace Map Generation - 4K 32-bit EXR with highest settings - from one apparently too large subtool = ~22 million polys gives the error Not Enough Memory with Multi Map Exporter, despite my PC having a lot more than 4GB RAM.
(I don't want to retopo into smaller parts so it will give subtools having not so many polys and reproject again and again...)
Windows Page file size doesn't matter: you can read a post by an experienced user here:
http://forums.cgsociety.org/showthread.php?t=1131761
So if I save just this mesh w. 8 subdiv levels and load it as a subtool in an empty project file, still Zbrush gives me not enough memory all the time. The displace map must be 32-bit EXR, no 16-bit fake crap allowed.
Here is, how you can override the "not enough memory" bottleneck:
Concept:
You need to lower your lowpoly model's poly count, without changing the model.
About ~70% of the 22million polygons will fit into the 4GB memory that Z4R6 can handle. We must select a couple of the neighboring UV Islands and go over this selection using a couple of Grow Selection button presses, so Zbrush will bake the displace map over the selected border as well. Split your model physically so the unselected region will be cut, reducing your polygon count at the highest subdiv.. Now you can click on Generate All Maps. Zbrush will no longer display not enough memory.. Then we need to select the other half of the lowpoly mesh and repeat above.
Go to Subdiv level 1.
1. The subtool is an eagle body (without the large feathers), cut into multiple UV islands.
2. Polygroups--> make groups from UVs = instant polygroups per UV islands!
3. Since the memory will be enough for an about 15 million poly subtool, I select about 70% of the model using the polygroups/UV islands, then I click on Visibility -> Grow about three times and the polygon selection grows by 3x rows reaching over the border of the selected 70% (lowpoly at subdiv 1), this gives a good portion of the full lowpoly model selected. This will be the first half. You must hide the rest with CTRL+SHIFT+ALT poly selection using lasso = this hides unwanted polys
4. By this time all the unselected polys are invisible
5. Polygroups--> group visible = places all visible polys into one polygroup
You can unhide all polys now.
6. Go to Highest Subdiv level, mine is 8 at ~22 million polys
7. Subtool --> split --> Groups split = this will physically CUT the model into little new subtools defined by polygroup borders (delete the newly created little subtools, belonging to the other half of your model)
8. Displace map will now generate properly with Multi Map Exporter
9. You need to repeat the displace map gen for the other half of the mesh, using the above method: make sure you go OVER the border of the other lowpoly half (at subdiv 1) too by a couple of Grow button presses!
(10. Going over the border by one row of polys for both halves is necessary, otherwise there will be a big seam / chasm during displacemap rendering: try it out by just splitting your large subtool into all UV islands and generate a displacemap for two islands sharing a UV seam.)
11. Now you have two 32-bit Full Float EXR displace maps.
12. The question is, how to combine these displace maps so you largely create the same quality displace map (32-bit R Full float EXR) as Zbrush creates from one subtool.. but this time merging them by hand so it looks almost the same as if Zbrush did it...
13. Install ProEXR for Photoshop. Open one EXR file: I selected "alpha appears on separate layers". Generate a solid color UV Template render in MAX (Unwrap UVW modifier, Open UV Editor, Tools menu, Render UVW Template). Load this BMP in Photoshop you can select all the UV Islands with the Magic Wand tool (in 8 bit mode!), save the selection and load it in the EXR file, increase the selection border by about 10 pixels (for 4K Disp map) then do a Layer via Copy. Drag the other EXR into this file as a new layer. Repeat the selection process in the solid color UV Template file using MAX 3D viewport and Zbrush to see, which UV islands you need to select for this other half of the model and save this selection too to load it into the first EXR file to use it for the second layer you just dragged in (use SHIFT+Drag to center the image precisely).
Increase the selection border by about 10 pixels (for 4K Disp map) then do a Layer via Copy again. You now have two layers via Copy containing only the good displace map data for all the UV Islands of the model. Delete the two original EXR layers. Make a new layer for the background and fill it with black, place it to the bottom and flatten the file. Save the EXR file, tell ProEXR to use 32-bit data and Zip compression.
14. Get Nuke and pull in a merge node, check the input R G B channel boxes for all of your EXR files - naturally you must have divided your lowpoly into separate meshes and created highpoly sculpts using these parts or "subtools" and have rendered EXR displace maps for all subtools so you have multiple EXR files by this time. Check only the R channel box for output, because Zbrush creates beautiful R Full Float 32-bit EXR-s too and it has less size, because the G and B channels will be empty. Pull in a Write node, select EXR file type as output and also name your file using .EXR as extension.
Press "rendering" in NUKE and it will write a beautiful main EXR file, just as Zbrush 4R6 would have done it, if the Merge Maps for All Subtools button would work and if it could handle more than 4GB memory..
Use this new 4K or 8K or 16K EXR to render your lowpoly model in MAX with displacement mapping. Have fun!