[Tool Dev] Smart Maps Baking

hbahnassi

Hi Polycount,

In my last project I developed a new tool that drastically enhances the quality of baked textures (less pixilation and blurriness) without increasing the texture size and sacrifice your game memory.

The tool is based on an AI algorithm that captures many factors from the 3D scene/object in order to generate best baked results. If you are interested in learning more about the technical background, please refer to this blogpost.

I recorded a few cases with very low-res textures to show the differences easier:

Baking Lightmap with Global Illumination for Sponza
- Baked Texture Size: 1024x1024
- Number of Triangles: 110,324 triangles
- Processing Time: ~4 hrs
Video on YouTube

Baking Lightmap with Final Gathering for a City Proxy
- Baked Texture Size: 512x512
- Number of Triangles: 11,406 triangles
- Processing Time: ~10 mins
Video on YouTube

Baking Ambient Occlusion from Hi-Res to Low-Res Model
- Baked Texture Size: 512x512
- Hi-Res Number of Triangles: 77,280 triangles
- Low-Res Number of Triangles: 10
- Processing Time: ~20 secs
Video on YouTube

I'm planning to make this tool available for public so I would appreciate hearing from you where you like it to be available?
Maya, Max, xNormal, Unreal or Unity.

And what would be your main use for such tool?
- Baking Hi-res to low-res maps (e.g. Normal Maps, AO Maps, … etc)
- Baking lightmaps (Global Illumination, Ambient Occlusion, … etc)

Thanks,
Homam

Eric Chadwick

This is a really neat idea.

So does it basically look for areas with highest rates of change, and allocate more pixels to those, with less for areas of less change?

What other heuristics does it use? Maybe overlap UVs for instances with similar lighting?

Personally I'd like to see this in Unity and Unreal.

kosmoski

Actually it's quite cool to see some one working in XSI these days  

supaclueless

If I had a choice on what platform this could work on, I would most certainly choose having this working with Unreal's Lightmass to somehow work together and produce cleaner lightmap qualities without needing denser texture resolutions or insane bake times.

arrangemonk

Why dont you just bake with higher resolution and scale the uv islands according to the chanfes inside the rendered area, and then repack tge changed uvs and render the lower resolution kightmap from the laeger one, you could also tesselate the affected poligons so the areas with high contrast are smaller

I think you could also do iteratiins of that algorithm:
Render large lightmap (way larger than desired result)
Calculate the contrast (sobel for example)
Sum the pixels of the uvislands and
Tesselate the ones with with higher values
Repeat last 2 steps for the newly created trangles
Scale according to contrast
Repack uvs
Render desired resolution lightmap from large and the 2 different uvs
Done

... Thinking again rotation of trangles is also important, fibding lines in the imafe and rotate the trangles so the lines are straight would cause better lines too

hbahnassi

Eric Chadwick said:

This is a really neat idea.

So does it basically look for areas with highest rates of change, and allocate more pixels to those, with less for areas of less change?

What other heuristics does it use? Maybe overlap UVs for instances with similar lighting?

Personally I'd like to see this in Unity and Unreal.

Thanks Eric for your feedback. The heuristic is a combination of several things, including rate of change as you have guessed as well as color importance plus other information related to the object topology.

Currently the algorithm doesn't overlap UVs, but it is definitely an interesting idea to consider

hbahnassi

kosmoski said:

Actually it's quite cool to see some one working in XSI these days  

hehe... Still resisting the switch to other package. It is not easy since our pipeline and game engine are integrated with XSI. I'm not sure how long we can take it. Hopefully another couple of years

hbahnassi

supaclueless said:

If I had a choice on what platform this could work on, I would most certainly choose having this working with Unreal's Lightmass to somehow work together and produce cleaner lightmap qualities without needing denser texture resolutions or insane bake times.

Thanks for the feedback. Integrating the tool with Unreal's Lightmass will definitely produce much cleaner and efficient lightmaps. However the baking times won't get shorter as Lightmass will still be part of the baking process.

ENODMI

Very cool!

supaclueless

@hbahnassi That would be fine. Even though the baking process speed is not improved, improving the bake accuracy without more performance overhead sounds like a natural plus.

hbahnassi

arrangemonk said:

... Thinking again rotation of trangles is also important, fibding lines in the imafe and rotate the trangles so the lines are straight would cause better lines too

Yes the current implementation does rotate triangles when packing UVs, but it is not considering rotating UVs to match a specific feature. Currently I'm piggybacking on Softimage's automatic packing algorithm. Which is out-of-the-box very fast and efficient. I believe it can be enhanced, but I'll keep this effort to Unreal or Unity.

and thanks for proposing another solution