Home Quixel Megascans

SOLVED: Normal Map/AO Question

Hey Guys!

So, I'm still learning the workflow for ddo and had a question about a couple of the maps and how they look. So the maps I've generated for this particular model are Specular, Normal, Albedo, Gloss and Bump. I have just begun mixing and mashing some materials and I think it's looking good so far, however whenever I take a look at my Normal Map and Occlusion map, they both change significantly.

dzj7t.jpg

As you can see, the Normal map is muddied up and shiny after all the layering of my materials, Is there a way to change it so it actually resembles a generic normal map or is that just how quixel generates it? If so, can I just use my original normal map I brought in and not worry about the new shiny/muddied normal map it's generated for me?

9790jl.jpg

As far as the occlusion map goes, it's very blurry and smeared so pretty much the same question applies, can I just use my original AO map instead of this blurred/smeared one?

24e3u2s.jpg

All in all, I guess it's a matter of appearance of how I'd like my maps too look. I was just wondering if someone could shed some light on this for me. Thanks!



P.S. This is going too sound newbie and stupid but where would I plug my gloss map in to say like Maya or Unreal Engine 4? I'm very new to the PBR workflow. Thanks guys!

Replies

  • Synaesthesia
    Offline / Send Message
    Synaesthesia polycounter
    Your images aren't loading. Please try uploading them to Imgur.com and edit your topic. I'll gladly take a look at them when I can see them. :)

    In regards to your layers, I've never seen that issue with DDO but I know you can simply fiddle with the opacity of the offending layers on the AO/normal maps to reduce or eliminate that problem altogether. It may simply be an incorrect blend mode that inadvertently got set by the current quirky version of DDO.
  • Ladygrace
    Offline / Send Message
    Ladygrace polycounter lvl 5
    I have a feeling you have placed a layer in the incorrect order or blended it in Photoshop.
    And yes you can import your own normal map when you start a project in DDo.

    Using PBR in maya i forgot where to plug it in lol, but make sure when you use the PBR shader in maya to change the limit of lights it can use as by default it only allows 1 light at a time.
  • ILoveArt
    Hey Guys I really appreciate the feedback and help!

    @ Synaesthesia Yea I'll be happy to re-upload the images via Imgur.com gimme a second. Also, I looked through my normal map and found at the bottom there was my Texture Base (the normal map) and it had an effect attached to it called Color Overlay. I'm not sure why it was there, I haven't fiddled with the auto-generated layer setup that ddo has but after I turned this off, it removed the discoloration. The shininess is not there anymore except on the edge padding which I guess I can settle with as it does not effect the normal map directly. The AO unfortunately is still smudged and whatnot even though I tried turning off all of the layers individually.

    @ Ladygrace Thanks for the heads up on the lighting setup I should use in maya for PBR it works like a charm! I of course also still don't know where to plug in my gloss map lol. However, the end game of this is to plug everything into UE4 anyways so I'll settle with Marmoset Toolbag for now.
  • ILoveArt
    JuXJt0o.jpg

    dh3qD1X.jpg

    UvIA8xm.jpg


    There ye go reuploaded!
  • Synaesthesia
    Offline / Send Message
    Synaesthesia polycounter
    Check and see if you have topmost layer called "Normal Scaler" in your normal PSD. If you don't, that's why your normals look like that. I've had my Normal Scaler layer get transferred to another map channel before - I'm not entirely sure what causes it, though. Put it back where it belongs and it should fix that part of it.
  • ILoveArt
    I looked around the layer setup and unfortunately there's no layer called 'Normal Scaler'. I followed the tutorials on youtube and I just don't see why my setup is different than others.
  • Synaesthesia
    Offline / Send Message
    Synaesthesia polycounter
    As I mentioned, it's an error in DDO currently. I haven't run into it with 2.0 yet, so Quixel has fixed it - at least, that's what it looks like so far! :)

    A temporary workaround for now: Create a new project with the same input maps (just to speed this process up) and duplicate that Normal Scaler layer into the top of your normal PSD. That will solve one aspect of the problem. I would also copy the layer style of the base normal layer and paste that back into your original base normal layer while you're doing this, just to ensure consistency. DDO adds a slight leveling effect - it's worth keeping just to toggle on and off and see what you prefer.

    As far as the AO is concerned, I have run into that issue but I suspect it's blurring off into the empty space between your UV shells. If it isn't, post your AO layers from Photoshop and I'll take a closer look at it.
  • ILoveArt
    Hey Syn, so I did as you suggested by creating a new project and the Normal Scaler is there. I duplicated and layered it on top of my previous normal map and it works like a charm thanks!

    As for the AO here are the images showing my layered setup that DDO generated for me, haven't messed with it at all.

    078SV9W.jpg


    1V5s0v5.jpg

    Thanks so much for your help by the way!
  • Synaesthesia
    Offline / Send Message
    Synaesthesia polycounter
    Looks like Dusty Edges might be the culprit.
  • ILoveArt
    Removing the dust edges helped remove the unblur/smudge effect atleast where the uv shells are. However the open space between each shell and the edge padding remained messy. So after removing my edge padding, everything looks great. My question is now then, to avoid this kind of circumstance again should I just export my normal map from xNormals without edge padding for ddo setup? I always assumed edge padding was good but it seems to be the main culprit in the messy AO.

    A workaround I just did was too clean up the normal map ddo generated and then map convert the normal map to an AO map using ndo so there's no blurriness/smudged areas, is this an ok workaround?
  • Synaesthesia
    Offline / Send Message
    Synaesthesia polycounter
    You should be checking every map that DDO works with to ensure that it's consistent with the style you need. Edge padding shouldn't be the issue - it's just another set of dark and light pixels to DDO, as far as it's concerned. I actually prefer to add padding myself with the xNormal Photoshop plugin, since I tend to render my AO map out about twice as large as my intended resolution. So if I'm working in 2k, I render out my AO at 8k. Then I grab the alpha, use it as a mask, duplicate the AO from the mask, shrink it to 2k, and pad it out by about 8 to 16 pixels depending on my layout. This virtually eliminates AO seams and the ones left over can be easily painted out in Mudbox or Photoshop.

    It's perfectly legitimate to use a narrow AO map (one generated only from normals, where you're not getting a lot of model occlusion but rather occlusion from smaller details that were painted in or modeled in) but I find the better result is to combine your rendered AO with your normal AO from NDO - it gives DDO more to work with and helps isolate areas in Dynamask when you mask with AO.


    In regards to your original question concerning gloss maps, UE4 doesn't use gloss. It uses roughness. When you export your project from DDO, it will automatically invert your gloss map and call it a roughness map. UE4 treats white pixels as completely rough, and black pixels as completely smooth. The values obviously fall between those ranges for most materials. :)

    If you plan to use UE4 for anything, you'll want to get used to the idea of putting roughness, metalness, and AO into one texture - I usually do metalness in the red channel, roughness in the green, and AO in the blue.
  • ILoveArt
    Oh wow thanks for opening my eyes that much more with lots of good info! Yeah I'm very new to the pbr setup so it's definitely been a learning experience but hell of a lot of fun! With your input I think I'm going to create a new DDO project with the parameters you just stated concerning exporting my maps using the xNormal Photoshop plugin. I've finally got a computer that doesn't take forever to bake that stuff out anymore so why not hop on board with that!

    Also, concerning the combination of AO maps, when you say rendered AO you mean the one from xNormal correct?

    I actually just noticed like a noob that you can export the maps. I originally had been saving out the PSD's as Targa's instead of exporting so thanks for clearing that up.

    Again thanks so much Syn you've been a life saver and really helpful!



    P.S.

    Where might the xNormal Photoshop Plugin be located in xNormal?
  • Synaesthesia
    Offline / Send Message
    Synaesthesia polycounter
    You could just reimport the reworked AO map too, if you don't want to lose what you already have. DDO's reimporter is pretty solid even in 1.8, where many bugs abound. :)

    Yes, by rendered AO I mean having a program render the physical model into an AO map - doesn't matter which one does it. xNormal is fast if you use GPU acceleration but the AO baker in 2.0 is just as fast, if not faster, which is also GPU-powered.

    You can certainly save out the PSDs, but I feel it's better to allow the program to export them into the format you prefer. It was designed around exporting your maps, after all. Remove some work for yourself! If the final output is different than what you were expecting, then don't hesitate to export it yourself.

    xNormal's Photoshop plugin is actually located inside of Photoshop. Filter > xNormal > Dilation

    It requires the layer it gets used on to have transparency, so that's why I select the alpha mask (ctrl+click on the alpha's thumbnail) in the AO you generated from xNormal. Then I press CTRL+J to duplicate the layer directly from the mask, so only the selection is duplicated. After it's shrunk to 2k, running the filter on it gives it proper padding.
  • ILoveArt
    Thankfully what I did with DDO only took 1-2 hours too accomplish so not too much loss, I figured I'd like to redo everything the right way this time instead.

    Ahhhh I found my xNormal plugin didn't have the files in the proper folder when it was installed so now xNormal plugin shows up and it works fantastic! Good lord I've been missing out! Much appreciated!

    Edit: Oh so I'm exporting from xNormal and are you saying while my AO should be 8k, my normal map should be 2k or same res? I'm just wondering if I should generate them separately in two different resolutions or both as 8k and then resize them to 2k after I'm done with the xNormal plugin.
  • Synaesthesia
    Offline / Send Message
    Synaesthesia polycounter
    You'll downsize the AO to 2k following the technique I mentioned earlier. Don't work at 8k unless you absolutely have to lol
  • ILoveArt
    Yup I understand that bit. Things are working much smoother and looking better thanks so much Syn! My only other question I guess is when you say plugging my roughness, metalness and AO into one texture (RGB) you do that process within UE4 correct?
  • Synaesthesia
    Offline / Send Message
    Synaesthesia polycounter
    You'd have to export using the RMA-packed UE4 profile in DDO, or manually combine the maps with the regular UE4 export. I prefer manually doing it. I've noticed compression artifacts in the RMA-packed export and prefer to avoid that.
Sign In or Register to comment.