Clean bakes
polygon
I made this thread because i heard a lot of ways to bake maps and i was courious what ways you guys use and wich one gave you best results?
I have some hard times with the bakes ,because i tried all the ways in the poll and the best result until now i had with the xnormal and the model smoothed,but i got some wierd shadows on the model.On the other side if i do hard edges the normal looks flat and it makes a wierd line on the hard edge.i've tried with cage too but i've got the same horrible results.
ps:i use maya 2014
I have some hard times with the bakes ,because i tried all the ways in the poll and the best result until now i had with the xnormal and the model smoothed,but i got some wierd shadows on the model.On the other side if i do hard edges the normal looks flat and it makes a wierd line on the hard edge.i've tried with cage too but i've got the same horrible results.
ps:i use maya 2014
Replies
This should clear it up : https://www.youtube.com/watch?v=ciXTyOOnBZQ
9/10 I find you will get a better baking method using a couple programs with overlays, and whenever I bake I use maya/max/zbrush as a default. Really lean away from xnormal for normals, usually only quick n dirty results.
Usually it'll just come down to workflow, and what you personally get the best results with. There's no real "clear-cut" winner, or else that would be the only way to do things.
Sorry about the early reply, the edit is me saying what I actually want to say.
Yes this is a common hard surface normal baking concept. But things can get complicated when using something like handplane because handplane does not care about the tangents of the low poly model. You should learn the fundamentals of normal maps from the wiki page.
http://wiki.polycount.com/NormalMap?highlight=%28%5CbCategoryTexturing%5Cb%29
As for the programs I personally bake everything in xNormal, and I will use handplane when it wants to work for me, but I still bake in xNormal. Also keep in mind baking goes deeper than normals, you should also bake out several other maps for lighting information.
[ame="
This is not a matter of opinion - this is math
- Now usually frustration comes from experimenting with that kind of stuff while working on a model which already took a lot of time to make. So, simply grab an old model of yours, and give yourself a few days to fully experiment.
- Also, at the end of the day it only really depends on the level of tolerance of the artist. Some do not notice small artifacts, and some do. And of course, once in game some things are noticeable and some are not. With Dota using a RTS perspective, a lot of problems go unnoticed anyways so that's good
- Now after years of working with that kind of stuff I have seen artists jump through all kinds of time wasting hoops to "fix their bakes", resorting to the most dirty tricks like trying to paint over seams in their normals, tweaking their models after the fact to hide problems, adding weird unnecessary hard edges, and so on. Every single time it came from a misunderstanding of how baking actually works, and was quite easily avoided by setting up the bake properly
If you want maximum quality, especially on smooth hard surfaces like armor plates, you really want to be perfectly synced - that is to say, converting your OS to Source-synced TS, like here :
But if you don't mind these kind of small artefacts, then of course you can use the default Xnormal TS - It will be 90% there
- All this is dependent on what you use to actually view these maps with, since every display environment (just like any baking environment) uses different syncing. A map perfectly synced to Source will not necessarily look "proper" in UDX or Crysis or Maya viewport. And arguable, the 3DSMax shader I sued above is not perfectly synced to Dota either hehe
- When it comes to Dota2 we are lucky enough to have access to a perfectly synced solution (OS to Source-synced TS handplane), which does not require any hard edges and does not need any weird "trick" to make it work, so, my recommendation would be for you to simply keep at it and keep testing it until it works. It really is a very smooth workflow.
- Lat but not least : while experimenting with all this, especially with Maya, make sure to triangulate your low poly meshes before baking. Maya (unlike Max) seem to have a weird logic when it comes to exporting non fully triangulated meshes. Put simply, while Maya might internally think [/] when displaying a quad of your lowpoly in its viewport, it might actually export [\] when writing an OBJ/FBX. Meaning that the resulting normalmap will not match the lowpoly in your viewport. So just to be sure, triangulate everything.
- - - - -
@ Kite : I would say that it get's *less* complicated with Handplane really
So, artists all over the industry had to find a workaround for this generalized lazy engineering, and found out that placing a hard edge in the lowpoly model where strong surface changes where happening (and of course, splitting the corresponding UVs to avoid color mixing) would take care of 90% of the issue (there is usually still a very faint seam left but it is hardly noticeable).
Now once you get rid of the problem by using a perfectly synced solution (which is a pretty rare thing - IDtech and Handplane-based workflows being the only ones I ever ran into), you do not need any hard edges tricks *at all*. So, if you are using a Handplane-based workflow, there should be no hard edges in your model, because the reason for them to be there is not here in the first place
- - - - -
Personally there are still a few things I want to test with that workflow. It gives me very good results but that is using OBJs and if I understand the format correctly it does remove some important information that should be necessary for a perfect bake (tangent-basis, which is something that one can display in Maya under polygon display options, it's interesting to look at). I need to test it all further using a worst case scenario mesh, a better file format, and then check the final result in Dota. I'll hope to have some time for that in the future !
If anyone is having problems getting handplane to work for them in dota2 send me an email (alecmoody@gmail.com). Especially if you are newer to baking, there are still some basic rules you need to follow for normal maps to function correctly (things like correct UV and mirroring setup). I can usually diagnose those things for people in a couple of minutes. Make sure your email includes your lowpoly model (the file you are sending into handplane) your object space bake, and an image of your issue.
Also, I just had someone buy a freelance license for dota2 use. Just to clarify for people here, you do not need a paid handplane license to use handplane for workshop items.
I was trying to look at this from a perspective of a user that has little to no normal map baking knowledge, and I should have given more detail on what I meant. If someone does not understand what a tangent space normal map does, how is it easier to understand that you can use an object space normal, a mesh with garbage tangents, and have handplane give you perfect tangents with the correct weighting for your engine? If you understand all the fundamentals of normal maps and baking, of course its less complicated with handplane, but understanding why it is this way can be more complicated. I think that is what I was trying to say :poly142:
Alec thanks for the email address, I will be reaching out to you later. I have consistent issues with my outputs, but when I get a good output, nothing has changed in my workflow,I was convinced it was my workflow, but no matter what I do I get the same results
By less complicated I meant, less of a convoluted process, not relying on "tips and tricks". But yeah I agree that some of the concepts may seem alien at first. Now luckily enough this is Polycount and all the information is here for anyone to grab - 11 pages of discussions on the subject right there :
http://www.polycount.com/forum/showthread.php?t=68173
here you can see my model,with diferent types of edges(smooth,hard and avarage).i did them as this:unlock normals,set normal to faces and after for each individual model i set avaraged edges or hard edges or smooth edges.
As you can see i get some wierd shadows on model and i suppose thats the main reason my bakes gets out ugly.
here are the bakes done with the Xnormal object/normal with each type of edges
and here are the bakes that i got using handplane for the 3 types of objects maps that i baked earlier and using as lowpoly mesh smooth or avaraged
So the best object space map i got was the one where i used a hard edges lowpolly and the best tangent space was using xnormal with a model with avaraged normals.
The handplane seems to make wierd colors and it screw up the entire model(note:i didn't use any kind of symetry on this bakes,just the unwraped models)
As a matter of fact ... with everything unlocked (green), "smooth" and "average" should be exactly the same. So, even before baking there is an issue here. And we can disregard the third option (all facetted) since it is of no relevance anyways.
The OS bake looks great, so there is no problem there.
So now all you have to do is to track what is going on with the "smooth" and "average" ones and work from there.
Good luck, and of course keep us updated.
Also, because of the extreme angle changes in your lowpoly, even a perfectly Source-synced map will always display with artifacts in Maya, since Maya does not use the same tangent-basis than Source anyways. So from there on, once you have the above problems solved and a good TS map from handplane, you should really show your end results inside Dota, in game. (a rather simple way to do that is to export your test asset as a Lone Druid summon, or even, a courier.)
I am glad that you are posting pictures - it makes it much easier for us to help you out
I would recommend you watch this entire video as it should fill in any gaps in your baking workflow:
[ame="
The second issue here is how you are approaching smoothing on your low. Without going into a lot of detail, the easiest way to set up smoothing is to only harden edges on UV border edges. If you are doing organic objects where UV seems need to transition across smooth surfaces you will want to keep those edges soft as well. The rule here is- no hard edges without a UV split AND UV padding.
@ Alec :
"no hard edges without a UV split AND UV padding" is totally true ; however :
"Without going into a lot of detail, the easiest way to set up smoothing is to only harden edges on UV border edges" is, as far as I know, incorrect ; or, to be more exact, it is unnecessary and introduces potential errors. I mean, normalmap calculation and display is an extremely sensitive thing ; introducing the harshest thing possible (= a hard edge all the way down the UV seam of an arm, for instance) can only produce more approximations and errors ...
I keep hearing about people doing that but I think this is mostly because of a misunderstanding. I personally never ran into a single case where it was necessary. I would love to see an example tho !
And i always triangulate my models before exporting them for bake so the problem isn't here
This is what I was referring to eariler. If you use the obj sapce > Handplane workflow, these normals dont matter. Handplane does not use them, that is why you can feed a mesh with garbage tangents, one smoothing group but model has hard edges, and it will fix these errors in the output tangent space normal. If you look at the handplane example any area with bad shading on the model would be the garbage tangents I'm talking about.
(This of course assumes that we will manage to help each other successfully on this, but I am confident that we will
Also : I would love to see a screenshot of your two models ("smooth" and "average") under viewport 2.0 with a Blinn applied. Regular viewport with the matte default Lambert make things hard to debug visually. Thanks !
Another thing : I have encountered some very weird meshes exported out of Maya before, to a point where they made Max crash because of their weird tangents. If I remember correctly I fixed that issue one time by exporting the mesh out of Maya, importing it to Zbrush, and re-export from Zbrush as OBJ as it seemed to "clean up" everything. Then the model just needed to be set to smooth again (in Maya or Max) and then everything was fine from there. So maybe it could be worth trying.
I am having a hard time following exactly what the issue is. Can you send me your files so I can check it out? I just need a maya file with your lowpoly, and your object space bakes.
Also, the only weird colors I see on your normal maps are from a lack of padding in your object space bake.
This might be a case where in the short term you should not worry about any hard edges at all. Just set unlock your normals, select all the edges, and smooth them. Personally, I never use maya's average normals button. When you bake for source using handplane you should not ever need hard edges to correct a shading error.
on the other side they don't look as good
@alec i'll send you mail with the files
This vid covers xnormal and maya:
[ame="
bent normal got no artifact
Nannou : I am not sure to understand what you are saying here. You "fix" this by simply giving handplane your fully smooth lowpoly model, your OS map, and let it convert it all to an accurate Source-synced TS. Also I am not sure to why you are mentioning Bent Normal Maps here, as they have absolutely no relevance to the Dota2 workflow...
Hey nannou, what does the object space map look like from the tip of the blade there?
Also, feel free to send over that object space map and your 3d file. Ill look at it and let you know what is causing the broken output on the end.
Here's the files Alec if you wanted to take a look. https://www.dropbox.com/s/67jaryzaoblhjxv/test.rar
This is my normal map with hard edges. I had to split the tip in 2 and add hard edges to get it to work.
(haha And beat me to it
To check off double sided on your mesh go to the Attribute editor for you mesh under the shape tab and look under Render stats. Also make sure "opposite" isn't checked. That will auto happen if you negatively scale something and then combine.
i always have the backface culling checked so if the normals would be inverted i would see,also i don't think that dota 2 engine support double sided meshes so i would have to reverse the normals manually
i'm 99% sure that the problem is from the smoothing and the way i smooth the mesh because the OS is baking fine so the mesh itself doesn't have problems and the artifacts that i got into the tangent are exactly where i got the wierd shades.
so? anyone? XD
Freeze transforms > Reset transforms > Normals-unlock all > Normals-Set to face (that should make it all facetted) > Normals-Smooth normals. All while having the face or vertex normals display enabled, to check how it changes from yellow to green.
edit: but if i unlock normals again in the end all the yellow normals become green
1. smoothing split needs uv split
2. hard edges where ever possible
3. uv mapping, uv mapping, uv mapping
part 3 takes a bit of practice and the process itself takes a bit of time, but imho its essential for a good quality bake, and i am still learning that little spaces matter in the end, which means use the little spaces as much as you can and make your uvs as big as possible without overlapping eachother. i think it was jeremy who had a pretty nice example picture of a sven sword uv map in his thread.
Every hard edge introduced in the model means that the normalmap will have to compensate for it, on top of doing its real job of faking the surface quality of the highpoly model. Why introduce that ?
i got that info btw from a pc thread on normal maps. seemed to the standard work that is suggested to folks when they have questions ^^ For me, i never could use too many hard egdes in a model anyway. maybe cause I simply do more organic work then hardsurface modeling/sculpting
However we are *very* lucky with Source and Dota2, because thanks to Handplane these two are perfectly synced. Thats actually pretty awesome and I think we should try to use it at its fullest !
- - - - -
Now for a bit of history : ironically one of the first games to ever use normalmaps, Doom3, was perfectly synced back then. Baking was taking place inside the game client itself using a command line tool (!), and the results were flawless regardless of sharp angles and whatnot. Not only was the syncing perfect, but it also used face-weighted normals for increased precision. This was waaaaay ahead of its time. Maybe that's why I am so picky when it comes to bake quality now