Home Technical Talk

Physically based rendering in games

134567
ngon master
Offline / Send Message
ZacD ngon master
*******
NINJA ADMIN EDIT
*******
Some of the recent conversation in the UE3 thread has centered around physically based rendering for games, this is really a topic that deserves it thread so here we go!

First, some links:
Image based lighting, which is essential for a PBR workflow - Paul Debevec, Debevec's research is probably the basis for most IBL implementations out there today: http://www.pauldebevec.com/Research/IBL/
Adopting a physically based shading model (mostly math) by Sebastien Lagarde: http://seblagarde.wordpress.com/tag/physically-based-rendering/
PBR in Remember Me: http://www.fxguide.com/featured/game-environments-parta-remember-me-rendering/
PBR in UE4: http://blog.selfshadow.com/publications/s2013-shading-course/karis/s2013_pbs_epic_slides.pdf
Kodde's PBR Maya shader: http://www.polycount.com/forum/showthread.php?p=1505129

Let me know if anyone else has good resources and I'll link them here as well.

I've pulled some posts out of the UE3 thread, and combined them in this thread, so some comments may look a little confusing.

(Sorry zac, since this is the oldest post it will always be up top, hence the editing)

-EQ
*******


I would like to get a head start on learning how to create assests for physically based lighting, since is seems like many major next gen games are going to start focusing on it.

Replies

  • Obscura
    Offline / Send Message
    Obscura grand marshal polycounter
    Its possible in UDK with a material, that is using custom lighting.
  • Computron
    Offline / Send Message
    Computron polycounter lvl 7
    You watched the Guerilla Games Shadow Fall presentation perchance?
  • joeriv
    Offline / Send Message
    joeriv polycounter lvl 7
    Someone is working on a shader for unity:
    http://www.polycount.com/forum/showthread.php?t=120093&highlight=unity+shader

    There is also a shader for maya by kodde
    http://www.kostas.se/?p=30

    When I heard about it, I also tried to make on for UDK, but my knowledge in shaders is/was a bit to limited to get it working properly, maybe if someone can/wants I can throw it online to fix it/get it working as it should.

    That's basicly the only things that are out there as far as I know untill the new unreal/udk4 comes out or a new version of cryengine
  • ZacD
    Offline / Send Message
    ZacD ngon master
    Computron wrote: »
    You watched the Guerilla Games Shadow Fall presentation perchance?

    and the Fox engine demo. I kinda wish I used maya right now :\
  • Kwramm
    Offline / Send Message
    Kwramm interpolator
    kodde is a good start. On the UE4 website/docs are some good links to PBS, but stick to kodde first since it is an actual implementation and shares many similarities with other implementations. From an artist point of view the difficulty is getting used to working with the new texture types, not painting in light info, etc. and finding good workflows that allow you to create each of the map types in an efficient way.
  • leleuxart
    Offline / Send Message
    leleuxart polycounter lvl 10
    Like everyone else has said, the Maya plugin and Unity shader are the only two options right now. UE4 will switch to physically-based lighting and Crysis 3 used it, so whenever 3.5 comes out for CryEngine, I think we'll see the feature.

    I've been reading everything I can(and watching all of the videos), but it's almost useless if I can't try it out in an engine. I'm gonna give the Maya plugin a shot though.
  • iniside
    Offline / Send Message
    iniside polycounter lvl 6
    Publicly ? None.
    CryEngine 3 is closest thing to PBR but it still quite off. You can start using CE3 right now. Although remember that default ToD have nothing to do with PBR, and you will need to create your own from scratch.
  • ZacD
    Offline / Send Message
    ZacD ngon master
    Grrr, guess I'll need to download Maya, people need to get to work making shaders :P
  • Money
    Offline / Send Message
    Money polycounter lvl 8
    lpcstr posted shaders for UDK if you want to try it. I actually got the same idea so played around with them last night.

    http://www.polycount.com/forum/showthread.php?t=116169&postcount=10

    http://www.polycount.com/forum/showthread.php?t=117463

    read also that blog entry on how to chose vales for textures.
  • EarthQuake
    Something I've wondered about this sort of shader... Is there any sort of standardization on how these shaders should work?
  • ZacD
    Offline / Send Message
    ZacD ngon master
    EarthQuake wrote: »
    Something I've wondered about this sort of shader... Is there any sort of standardization on how these shaders should work?

    Apparently there is some standard of how materials are defined, that their roughness and what not is measurable in the real world.

    [ame="http://www.youtube.com/watch?v=FQMbxzTUuSg"]Hideo Kojima GDC 2013 Panel - MGS5 & Fox Engine - YouTube[/ame]
    Technical talk starts at 19min




    [ame="http://www.youtube.com/watch?v=_29M8F-sRsU"]Digital Dragons - Micha? Drobot (Guerrilla Games) - Killzone: Shadow Fall - Lighting - YouTube[/ame]




    Siggraph presentations http://blog.selfshadow.com/publications/s2012-shading-course/

    Database of BRDF materials http://www.merl.com/brdf/
  • radiancef0rge
    Offline / Send Message
    radiancef0rge ngon master
    http://filmicgames.com/archives/547
    http://seblagarde.wordpress.com/2011/08/17/feeding-a-physical-based-lighting-mode/

    Its measurable. IMO having worked with a PBL system - in a 8 bit workflow its ridiculous to clamp the values like they typically are. You end up with maybe 15 values to express the variation in spec and there are very few materials in the world that can fit into that standard.

    http://seblagarde.wordpress.com/2012/04/30/dontnod-specular-and-glossiness-chart/
  • artquest
    Offline / Send Message
    artquest polycounter lvl 13
    http://filmicgames.com/archives/547
    http://seblagarde.wordpress.com/2011/08/17/feeding-a-physical-based-lighting-mode/

    Its measurable. IMO having worked with a PBL system - in a 8 bit workflow its ridiculous to clamp the values like they typically are. You end up with maybe 15 values to express the variation in spec and there are very few materials in the world that can fit into that standard.

    http://seblagarde.wordpress.com/2012/04/30/dontnod-specular-and-glossiness-chart/

    Here's another link while we're at it! I really liked the setup here. It seems to be very intuitive and easy to work with from what I can gather.

    http://disney-animation.s3.amazonaws.com/library/s2012_pbs_disney_brdf_notes_v2.pdf


    On a side note they offer a solution for what you are talking about radiancef0rge in that paper. I've run into this myself aswell, painting a specular map in terms of 1 -15% brightness value is a real pain in the ass because thats where the human eye has a terrible time distinguishing between value shifts.

    For those who don't know, when working in a PBR system the specular values are not at all what you would imagine. For instance diamonds in real life are one of the things that reflect the highest percent of light and they are only about 15%)

    Also I know that for the BRDF there are far more values of glossiness in real life then 255 so I'm curious to see how different studios will handle these sort of setups. I really love working with PBR though, as it really makes getting the end result I want a lot quicker.

    EQ, any idea on when marmoset will add this type of shader functionality? Is it something you guys are even looking into?

    Being able to visualize the BRFD will be key for the coming generation of games I believe. We're really going to need the ability to see this stuff in real time as we edit it.
  • EarthQuake
    Thanks for the links guys, I'll take a look.
    artquest wrote: »
    EQ, any idea on when marmoset will add this type of shader functionality? Is it something you guys are even looking into?

    Being able to visualize the BRFD will be key for the coming generation of games I believe. We're really going to need the ability to see this stuff in real time as we edit it.

    I can't tell you anything concrete other than we're looking into it. Its something that I personally feel is going to be pretty important moving forward, so we'll see how it shakes out.

    Any personal experience that people have working with this sort of shader setup would be great to hear, what you liked about it, and what you didn't, how easy it is to work with compared to a traditional diffuse/normal/spec/gloss workflow, and any suggestions that you think would make this sort of shader setup easier to work with.
  • leleuxart
    Offline / Send Message
    leleuxart polycounter lvl 10
    Not sure how useful this is, but it seems like it provides a good understanding of how the new shading model works, including the code. Even though I don't do any coding, I love learning about new things we may see, as well as staying up to date.

    [ame=" Reflectance Distribution Function - YouTube[/ame]


  • Obscura
    Offline / Send Message
    Obscura grand marshal polycounter
    How good or bad if it would works with cubemaps? I think it would be ok for static scenes. Or not? Its not too expensive to use real time reflections(captured in real time) on everyting?
  • ZacD
    Offline / Send Message
    ZacD ngon master
    Obscura wrote: »
    How good or bad if it would works with cubemaps? I think it would be ok for static scenes. Or not? Its not too expensive to use real time reflections(captured in real time) on everyting?

    Watch the Kill Zone video, they use a combination of special cube maps with real time reflections.
  • radiancef0rge
    Offline / Send Message
    radiancef0rge ngon master
    In my experience cubemaps were generated for each staging area (open world)
    There were realtime reflections in highly localized areas but I do not recall how they were done :/ If my memory serves me they were reflection cards
  • fearian
    Offline / Send Message
    fearian greentooth
    forgive what may be a stupid question - In a PBL engine, would you still be able to have materials in a scene using a standard diff/spec setup? I woud imagine that in some cases the trade off between time spent making th different textured vs difference in quality might not be worth it. (small props or some materials)


    Also, I'm unsure on the advantages/quality differences beween a PBL texture settup and fully utilizing Diff, Spec, Gloss and Image based Lighting with Cubemaps. Is the diffrence that profound?

    Edit: OK, the Guerrilla talk explained quie a bit. I'm still not sure how much of the diffrence is their new lighting engine vs the material set up, but from the sounds of things it's actually quite the pipeline improvement once you get into it.
  • Santewi
    artquest wrote: »
    On a side note they offer a solution for what you are talking about radiancef0rge in that paper. I've run into this myself aswell, painting a specular map in terms of 1 -15% brightness value is a real pain in the ass because thats where the human eye has a terrible time distinguishing between value shifts.

    For those who don't know, when working in a PBR system the specular values are not at all what you would imagine. For instance diamonds in real life are one of the things that reflect the highest percent of light and they are only about 15%)

    Also I know that for the BRDF there are far more values of glossiness in real life then 255 so I'm curious to see how different studios will handle these sort of setups. I really love working with PBR though, as it really makes getting the end result I want a lot quicker.


    You could always just work in the 0-1 range, and then just divide the final value by (1/max specular value).

    So for something that reflects 15% of the light, you'd get the final value by dividing the full-range image by (1/0.15) = 6.67. And its easier for your eyes too.
  • equil
    All that really changes is the spec and gloss maps. you can think of the usual spec maps as specular masks, they directly mask the specular contribution on the object. the "new" specmaps describe the amount of light (as a %) that gets reflected, which is calculated as the ratio between the indices of refraction of the mediums light passes from and to. this has to do with the specular term having fresnel added to it.

    gloss maps (i really prefer to call this roughness) are technically the same as always, but since the specular term is now normalized, the roughness also affects the brightness of the specular highlight.
  • iniside
    Offline / Send Message
    iniside polycounter lvl 6
    fearian wrote: »
    forgive what may be a stupid question - In a PBL engine, would you still be able to have materials in a scene using a standard diff/spec setup? I woud imagine that in some cases the trade off between time spent making th different textured vs difference in quality might not be worth it. (small props or some materials)


    Also, I'm unsure on the advantages/quality differences beween a PBL texture settup and fully utilizing Diff, Spec, Gloss and Image based Lighting with Cubemaps. Is the diffrence that profound?

    Edit: OK, the Guerrilla talk explained quie a bit. I'm still not sure how much of the diffrence is their new lighting engine vs the material set up, but from the sounds of things it's actually quite the pipeline improvement once you get into it.

    I've been digging PBR myself recently and it is much easier to setup than you think for first time.

    Non-PBR will work. To some degree, but they will not look good.
    In general making PBR textures is much easier and faster. You need to keep them as simple as possible. It's job of renderer to make them look good not yours. Yours job is to only provide information about material in texture.
    Here is nice info about it:
    http://seblagarde.wordpress.com/2011/08/17/feeding-a-physical-based-lighting-mode/#more-107

    Tl:dr version is simple object should look good only with normal,spec and gloss map. Color map is nice addition that should enchance look, but shouldn't be needed to make object look good.

    Another advantage is, once you made correct material you can use it in any other PBR setup.


    As for cubemaps. I've been thinking about semi automated solution (in this case for cryengine), where we cloud add many Env probes and automate baking them. Let's say bake cube map for every hour (for night cycle), and then blend between then as the time pass.
    Fortunetly IBL is not that intrusive like lightmaps, so it cloud be nice solution for day-night cycle, even with destruction enviroment. Doubt many players will see much of a diffrence if part of cube would be destroyed in world but not in cube map.
  • leleuxart
    Offline / Send Message
    leleuxart polycounter lvl 10
    equil wrote: »
    gloss maps (i really prefer to call this roughness) are technically the same as always, but since the specular term is now normalized, the roughness also affects the brightness of the specular highlight.

    I remember seeing a post in WAYWO of a gun using different maps. If I remember correctly, they were Diffuse/Albedo(the flat color, with no AO/Cavity added), Substance, Gloss/Roughness, and one more that I'm forgetting. 
  • passerby
    Offline / Send Message
    passerby polycounter lvl 12
    anyone have any good examples of a asset using these kinda maps, like for the specular mask, would i still be having to highlight edges and things like that, or would the shader do it all
  • marks
    Offline / Send Message
    marks greentooth
    passerby wrote: »
    anyone have any good examples of a asset using these kinda maps, like for the specular mask, would i still be having to highlight edges and things like that, or would the shader do it all

    If your geometry and/or normal/parallax map create an "edge" in the gbuffer, then yes it will "do it for you". In an ideal world you wouldn't be painting very much surface-geometry based (ala edges and indents) information into the specular MASK at all.

    As Equil said, all that really changes is your specular level and roughness textures. Using the 0-255 range in the texture as your entire value ranges though means you're leaving (often large) parts of your histogram unused which is wasteful and likely won't compress as well. Coupling your PBR specular textures with a scalar multiplier value in your shader may be a better solution, albeit a little less simple to understand.
  • ZacD
    Offline / Send Message
    ZacD ngon master
    passerby wrote: »
    anyone have any good examples of a asset using these kinda maps, like for the specular mask, would i still be having to highlight edges and things like that, or would the shader do it all

    You wouldn't be painting highlights on edges unless the material was different in those areas, or the edge was worn darker/lighter/smoother.
  • passerby
    Offline / Send Message
    passerby polycounter lvl 12
    ZacD wrote: »
    You wouldn't be painting highlights on edges unless the material was different in those areas, or the edge was worn darker/lighter/smoother.

    thanks exactly what i wanted to know, so no longer have to fake thigns like edge highlights, if it is there on the mesh and in the normal, it will act like a edge would in rl
  • radiancef0rge
    Offline / Send Message
    radiancef0rge ngon master
    marks wrote: »
    Using the 0-255 range in the texture as your entire value ranges though means you're leaving (often large) parts of your histogram unused which is wasteful and likely won't compress as well.

    THIS. this was my biggest gripe unless you use some new compression the bands it creates are ridiculous. Although I ended up finding a work around by putting some shitty noise in the normal maps.
  • ZacD
  • cman2k
    Offline / Send Message
    cman2k polycounter lvl 17
    SO much good info...really want to start messing with this stuff again...thanks guys!
  • Norman3D
    Offline / Send Message
    Norman3D polycounter lvl 14
    THIS. this was my biggest gripe unless you use some new compression the bands it creates are ridiculous. Although I ended up finding a work around by putting some shitty noise in the normal maps.

    You could normalize the histogram in photoshop and de-normalize it with a multiply and add operation in the shader.
    This way you would use the full range in the texture and still have the proper values in the shader.
  • ZacD
    Offline / Send Message
    ZacD ngon master
    That still could be an issue if your texture is using near white and near black.
  • radiancef0rge
    Offline / Send Message
    radiancef0rge ngon master
    There was no access to the shader but I understand what youre saying, its a good idea. :)
  • mLichy
    http://www.outerra.com/

    I've been working a vehicle in here. They are still far off from what they want, but it's getting there. Visually is quite impressive so far, even at it's current state. But the shaders are still not near final and missing many features.

    But it's super super easy to get assets in the engine, and it runs well.
    They also push out a chunk of updates/new features pretty often.
  • Brandon Kern
    Offline / Send Message
    Brandon Kern polycounter lvl 11
    This is a really really interesting discussion. I have a question though to you guys who watched the Guerrilla Games presentation: The presenter said that metals don't really scatter light, which is true, there is no sub-surface scattering on a material that is that dense... so there should be no diffuse on the material/should it just be black?

    EDIT: Here's the part of the video when he explains it: http://www.youtube.com/watch?feature=player_detailpage&v=_29M8F-sRsU#t=830s

    Does this only apply to next-gen physics based lighting or does it apply to this generation too? Should the only maps on a material that is metal be a specular colour map and a gloss map? I mean assuming that you're going for photo-realism?
  • ZacD
    Offline / Send Message
    ZacD ngon master
    You'd need cube maps or other reflections to be able to pull off metals without diffuse, it might work depending on the engine. You'd also need a way to blur the reflections for the different levels of roughness.
  • Brandon Kern
    Offline / Send Message
    Brandon Kern polycounter lvl 11
    ZacD wrote: »
    You'd need cube maps or other reflections to be able to pull off metals without diffuse, it might work depending on the engine. You'd also need a way to blur the reflections for the different levels of roughness.

    So say that it was your typical engine like UE3 for example with a basic shader and cube maps, would you just use specular colour and gloss for say a sheet of steel with a rusty patch with variation in both maps to define those materials accurately? I'm actually tempted to go try it now that I'm typing this out.

    Also, is a roughness map comparable to a gloss map?

    EDIT: is an albedo map just another name for diffuse?
  • EarthQuake
    ZacD wrote: »
    You'd need cube maps or other reflections to be able to pull off metals without diffuse, it might work depending on the engine. You'd also need a way to blur the reflections for the different levels of roughness.

    I think pretty much any system using a PBL lighting is also going to be using some sort of image based lighting for the specular reflections, so not sure this is really a concern.
  • ZacD
    Offline / Send Message
    ZacD ngon master
    I was addressing Bradon's questions for a non PBL engine.

    roughness kinda equals gloss
    albedo kinda equals diffuse

    If you are using UDK or something similar, you might want to put some reflection in the emissive.
  • EarthQuake
    Well image based lighting isn't really next gen or anything, we'll be releasing some shaders soon that do IBL on mobile.

    So really, I don't think you would want to do PBL without IBL, it just doesn't make much sense to try and get all physically accurate without doing accurate specular reflections as well.

    Edit: Ah wait, nevermind. Misunderstood what he was asking. Disregard.
  • artquest
    Offline / Send Message
    artquest polycounter lvl 13
    This is a really really interesting discussion. I have a question though to you guys who watched the Guerrilla Games presentation: The presenter said that metals don't really scatter light, which is true, there is no sub-surface scattering on a material that is that dense... so there should be no diffuse on the material/should it just be black?

    EDIT: Here's the part of the video when he explains it: http://www.youtube.com/watch?feature=player_detailpage&v=_29M8F-sRsU#t=830s

    Does this only apply to next-gen physics based lighting or does it apply to this generation too? Should the only maps on a material that is metal be a specular colour map and a gloss map? I mean assuming that you're going for photo-realism?

    If you have a shader written (or built in UDK) to account for this setup then you could do it current gen for sure. In any setup where you can use a color specular and you have reflections (cubemap or realtime) I imagine you could make your diffuse black and have it setup to use the specular color in this manner.
  • Brandon Kern
    Offline / Send Message
    Brandon Kern polycounter lvl 11
    EarthQuake wrote: »
    Well image based lighting isn't really next gen or anything, we'll be releasing some shaders soon that do IBL on mobile.

    So really, I don't think you would want to do PBL without IBL, it just doesn't make much sense to try and get all physically accurate without doing accurate specular reflections as well.

    Edit: Ah wait, nevermind. Misunderstood what he was asking. Disregard.

    Are you referring to the marmoset shaders that use IBL that you guys are bringing to Unity? Looking forward to seeing that!
    Sorry to drift off topic. But yeah, Guerilla Games presentation does a really good job of illustrating things with their screenshot examples.

    EDIT: Cool! thanks for the answers to my questions guys!
  • EarthQuake
    Are you referring to the marmoset shaders that use IBL that you guys are bringing to Unity? Looking forward to seeing that!

    Yep, we should have some more info on it soon, we've got it running on iOS, which is pretty cool.
  • marks
    Offline / Send Message
    marks greentooth
    IBL is a cubemap, basically. It's nothing "holy crap new technology". It's just a fancy (generic) term for stuff like cubemaps and (in offline rendering) HDRI's.
  • EarthQuake
    marks wrote: »
    IBL is a cubemap, basically. It's nothing "holy crap new technology". It's just a fancy (generic) term for stuff like cubemaps and (in offline rendering) HDRI's.

    Sort of.

    Image based lighting in games is generally composed of two cubemaps, one for the diffuse lighting, and one for the specular reflections. If you've got a good IBL system, you can control glossiness by saving various specular cubemaps with different levels of blur and then blending between them for the matte through mirror reflection spectrum.

    So really you just need two cubemap lookups, which means you can do it on current gen, next gen, even some mobile platforms as well.

    HDR refers to files with high dynamic range, and isn't really specific to cubemaps or IBL. A photograph can be HDR, a texture can be HDR, etc. Though generally high quality IBL systems will use HDR files for the cubemaps.

    HDRI, Cubemaps, and IBL aren't really interchangable terms, though they do tend to hang out in the same whitepapers. =D
  • marks
    Offline / Send Message
    marks greentooth
    Sweet explanation, thanks. I hear that the cool kids blend between different mip levels of the environment cubemaps for controlling glossiness of reflections. Wasn't there a whole epic thread around here somewhere with perna schooling people about HDR cubemaps? ;D

    Cubemap for diffuse lighting is confusing me though - what's the craic with that? I'm not down on in-engine lighting methods.
  • EarthQuake
    marks wrote: »
    Sweet explanation, thanks. I hear that the cool kids blend between different mip levels of the environment cubemaps for controlling glossiness of reflections.
    Yeah, but its good to do a little bit more to it than just pick the lower mips, ideally you want to actually blur the spec, which will give you better quality in the lower gloss levels.
    Wasn't there a whole epic thread around here somewhere with perna schooling people about HDR cubemaps? ;D
    Yes, the conclusion of that thread wasn't so much HDR = useless, it was Per talking about how he compressed an HDR image into the range of an LDR image, and used fancy maths to extrapolate it back to HDR-ish in the shader. Skyshop will ship with RGBM support for IBL cubemaps, which is similar (storing HDR data in an LDR image + alpha channel) and gives good memory savings over something like 16 or 32 bit float HDR files which can be huge.
    Cubemap for diffuse lighting is confusing me though - what's the craic with that? I'm not down on in-engine lighting methods.
    If you've ever used Toolbag, the default lighting is from a diffuse (blurry) cubemap. The specular reflection cubemap is view dependent, but the diffuse cubemap is static (well, its tied to the sky direction, but you get it I'm sure).
  • artquest
    Offline / Send Message
    artquest polycounter lvl 13
    marks wrote: »
    Cubemap for diffuse lighting is confusing me though - what's the craic with that? I'm not down on in-engine lighting methods.

    I recently had the privilege of implementing this sort of setup in a shader for the ambient lighting. The results are quite good. You can create an irradiance map to use for diffuse lighting via the amd cube map generator found here.
  • fearian
    Offline / Send Message
    fearian greentooth
    marks wrote: »
    Cubemap for diffuse lighting is confusing me though - what's the craic with that? I'm not down on in-engine lighting methods.

    The simple way I've always thought of it, is if you have a cubemap of a field with blue sky and green grass, it will use those colour values to ambiently light the model, so that the bottom lights slightly green, the top slightly blue. Much like 'bounce' light IRL.
  • Joopson
    Offline / Send Message
    Joopson quad damage
    Silly question about cubemaps in UDK.

    EDIT: Found it out, of course. Right in plain sight. Sorry guys! I swear it's not my fault.



    Interesting discussion so far.
134567
Sign In or Register to comment.