Is it possible to bake hard edges or selected edges somehow in Maya, Mental Ray, V-Ray or anything ? I want a baked image with stated edges white and all other black background.
Here is the stuff I want to achieve before you ask. I want to extract an uv image for hard edges with normals. So if the edge is part of full convex face it will be white and if the edge is part of concave face it will be black or inverse of this and others will be interpolated gray. So I can make a normal map with that texture. I extracted all edges without that info and converted that grayscale image to normal map and concave edges become black as I already expected.
You're not going to be able to fake a high poly with this idea if that's what you're after. Sounds like you're after a curvature map? I can't think of a good way to only do selected edges directly, maybe mask of what you don't need with an AO or even delete faces that you don't want contributing, filling the gap in with grey.
Curvature map was out of my mind. The idea was if we can have the curvature of hard edges from low poly, we can fake a fast normal map. Xnormal curvature map gives all edges.
Alright, I have managed to make quick normal maps that only softens hard edges without creating high polys. It is not that hard actually. There is no need for curvature map information. I will explain in case of anyone might want to try. Just select every single hard edges and cut out in UV Editor to seperate UV Shells for different hard edges and pack it that way. If the shape is complex it will increase shell count more than you may think. Also make sure there must be at least 1 pixel padding between shells. Now we need a simple selection mask whose UV shells must be white and background must be black. Transfer the mesh into Substance Designer and convert it to UV SVG and set resolution 2x more than you want to achieve. This step is important because Substance Designer anti aliases uv shell edges smartly which makes hard edges disappear more. If you have any other software to make selection masks make resolution 4x more than you want to achieve and downsample 2x to get similar effect. Then take it to the Photoshop for Quixel NDO and make sure the curve is Half Round. Make Size 1 to 8 depending on resolution and Contrast 50 to 100 depending on size and resolution. Try to make soft but also discernable with contrast. To make it work with values higher than 8 you may need bevel to fit more otherwise it wont look natural. If you have made black shells and white background then make Bevel to Outer and Stand to Down. Then finally downsample the normal map 2x to get antialiasing effect. This step is also important to reduce hard edges and get soft transition effect. These are my results.
The method you posted reminded me of the Chamfer Node for Substance Designer which was posted a while back. It does something similar to your technique whereas it uses the UV islands, among other things, to generate a normal with bevels. I have yet to get around to testing it. For now, if I need really quick bevels and am working in Painter, I have a bevel filter (simply the bevel filter from Substance Designer, packaged for use in Painter) which I apply to the mask of a layer with height information and adjust that with levels if necessary.
Replies
I want to extract an uv image for hard edges with normals.
So if the edge is part of full convex face it will be white and if the edge is part of concave face it will be black or inverse of this and others will be interpolated gray.
So I can make a normal map with that texture.
I extracted all edges without that info and converted that grayscale image to normal map and concave edges become black as I already expected.
I can't think of a good way to only do selected edges directly, maybe mask of what you don't need with an AO or even delete faces that you don't want contributing, filling the gap in with grey.
The idea was if we can have the curvature of hard edges from low poly, we can fake a fast normal map.
Xnormal curvature map gives all edges.
It is not that hard actually.
There is no need for curvature map information.
I will explain in case of anyone might want to try.
Just select every single hard edges and cut out in UV Editor to seperate UV Shells for different hard edges and pack it that way.
If the shape is complex it will increase shell count more than you may think. Also make sure there must be at least 1 pixel padding between shells.
Now we need a simple selection mask whose UV shells must be white and background must be black.
Transfer the mesh into Substance Designer and convert it to UV SVG and set resolution 2x more than you want to achieve.
This step is important because Substance Designer anti aliases uv shell edges smartly which makes hard edges disappear more. If you have any other software to make selection masks make resolution 4x more than you want to achieve and downsample 2x to get similar effect.
Then take it to the Photoshop for Quixel NDO and make sure the curve is Half Round. Make Size 1 to 8 depending on resolution and Contrast 50 to 100 depending on size and resolution. Try to make soft but also discernable with contrast.
To make it work with values higher than 8 you may need bevel to fit more otherwise it wont look natural.
If you have made black shells and white background then make Bevel to Outer and Stand to Down.
Then finally downsample the normal map 2x to get antialiasing effect. This step is also important to reduce hard edges and get soft transition effect.
These are my results.
I have yet to get around to testing it. For now, if I need really quick bevels and am working in Painter, I have a bevel filter (simply the bevel filter from Substance Designer, packaged for use in Painter) which I apply to the mask of a layer with height information and adjust that with levels if necessary.