Home Technical Talk

Animating Normal maps to achieve rain drops in 3DS Max or Unity3D

2

Replies

  • D4N005H
    Offline / Send Message
    D4N005H polycounter lvl 4
    Obscura wrote: »
    That would be the same thing as what I suggested on the previous page :P The animated thing, you would just control it with physics instead of hand setting the parameters in an animation.The material setup could be the same or almost the same and you could control the material parameters with C#. So I think it should be possible, if you can get physics values and do some math with them.
    When rendering, you set up a normal map texture as your render-target (everything you draw will appear on that texture), then you generate a single mesh consisting of several quads that represent each particle. All quads have the same "water drop normal map" texture like in post #34.
    Then you clear the normal map with a neutral colour and draw that mesh so the particles are stamped on it.
    The resulting normal map texture with the water drops is applied to the windshield and is what the refraction shader uses.

    The more particles, the slower it'll be. If it's extremely slow for the desired number of particles (since their logic would be written in C#) then it's possible to pass on all processing to the GPU in the form of "GPU-based particles" which is a technique that stores (and processes) particle data on textures, but it's much more complex.

    These are just ideas of course.
    Okay. I'll share the results here soon.
  • D4N005H
    Offline / Send Message
    D4N005H polycounter lvl 4
    Obscura wrote: »
    So this is how a flipbook texture looks. Sprite sheets works the same way... You have frames rendered out and put next to each other, and are simply stepping on then with modifying the current texture coordinate.
    Flame_FlipCard01.jpg
    Okay lets forget the metaballs for now, I want to make a flipbook of baked normal maps of waterdrops in 3DS Max.

    I want to simulate rain in RealFlow and then import the animated mesh into 3DS Max and then bake a normal map for this. I then will make a flipbook of those images. Then I should do what Kryzon said:
    Kryzon wrote: »
    When you want to clear the normal map (for example when the wiper sweeps) you can also render some primitive into the texture like a polygonal fan and using a neutral normal map colour (127, 127, 255):

    Windshield_normal_map_clear.png

    03fig02.jpg
    but I have doubt if the clearing normal map parts with flipbook is possible or not? is this a good way?
  • skankerzero
    Just commenting so I can visually distinguish this thread from the others.

    I'm loving watching this thread develop.
  • Obscura
    Offline / Send Message
    Obscura veteran polycounter
    Is it possible to make a "for cycle" in HLSL?
  • D4N005H
    Offline / Send Message
    D4N005H polycounter lvl 4
    Obscura wrote: »
    Is it possible to make a "for cycle" in HLSL?
    Well logically it should support. although I haven't seen cycles in HLSL.
    EDIT: Here is a function with "For" : http://www.developerblogger.com/376_12764064/
  • D4N005H
    Offline / Send Message
    D4N005H polycounter lvl 4
    Obscura wrote: »
    Hey, I just made an example for you for the windshield wiper+dynamic bubbles [...]

    Aaaa... I just understand your post :D now that's not only easy but I managed to solve the merging normals,too.

    Here is what I did, there is two separated normal maps for 2 wipers. these two will be used separately in Animator so that if one wiper got stuck or got damaged. the appropriate normal map won't get animated (rotated). the background is a normal map BUT I assumed that the background normal map would be a flipbook one for animating water drops. I then after all should do something for the metaballs. I think I should kiss rigidbody and say goodbye to it and start creating C# for it later, but for now, these normal maps should be created.

    NaiveSaltyKingfisher.gif
    The material is assigned to a cube, you are watching the top side of the cube.

    here is how I merged two normal maps without problem, however took some hours to get this result, but it is working like a charm:

    2edxowy.jpg

    As I said before, the "background" normal map should be animated later via backed normal maps that I will make them later.
    Please note that ShaderForge shows normal maps more bluish.
  • Obscura
    Offline / Send Message
    Obscura veteran polycounter
    By merging normals, I meant something like metaballs with normalmap inside the material. Sorry if it wasn't clear. That should be also possible. With black-white images, its easily possible but I couldn't figure out yet how to do that with normal maps. Maybe it would be better to actually start with only alphas and generating the normals inside the materials. Hmm. This should work.
  • D4N005H
    Offline / Send Message
    D4N005H polycounter lvl 4
    I think those metaballs should be particle 2D, although that is possible to have something like metaball in shader but in the DriveClub's videos, I think those metaballs were actually metaballs because they moved and merged together. I was wondering if you have any Idea of how to make them behave like a real metaball in the shader code?
  • Obscura
    Offline / Send Message
    Obscura veteran polycounter
    Here we go, I made a quick test...It is not perfect but it does work for the basics. It needs a lot more smoothing, but probably simply a better alpha would work...If not, then it needs a better solution for generating the normals. There are a few formulas. I used the "basic" setup from here.
    How it works:
    On the left side, you can see a black plane, and some white sphere alphas moving around. I used a scene capture, to capture this into a dynamic texture (render target 2d), and then I'm generating normals from the dynamic texture inside the material, so its similar to what Kryzon did. I animated the planes of the sphere alphas with hand but they can use physics too.
    [ame="
  • Obscura
    Offline / Send Message
    Obscura veteran polycounter
    Okay here is V2 which already looks so much better. Now the example on the right side use the Sobel filter, but it doesn't look good on the edges... However it has a bit more liquid like look when the balls are merging. The trick for the smooth look was to render the dynamic texture at very low resolution , something like 32x32... Maybe it is also possible to sharpen the edges somehow.
    [ame="

    They look disgusting haha.
  • Obscura
    Offline / Send Message
    Obscura veteran polycounter
    Here is V3 with sharp borders... For this, I made an another scene capture and render target, and rendered a high resolution image instead, and I used this to lerp between the generated smooth normals, and a neutral flat normal color (0,0,1) based on the high res mask.

    Maybe it would be better to down-size the high res rendertarget in the material, because using 2 scene captures are dropping down the framerate the same as the particle emitters. Its like -4 fps per scene capture. At least on my pc.
    [ame="
  • D4N005H
    Offline / Send Message
    D4N005H polycounter lvl 4
    Well done, specially the third one. but for the particles, I used colliders to interact with not only gravity but also the G-forces of the car and the wipers. How do you want to make those water drops interact with physics? that would be great if this question answered.
  • D4N005H
    Offline / Send Message
    D4N005H polycounter lvl 4
    I was surfing the net for this and found a good article, it's about the water drops, they used objects there. (Just sharing this.)
    http://www.indiedb.com/games/autumn-game/news/simulating-dynamic-weather-effects-rain
  • Fingus
    Offline / Send Message
    Fingus polycounter lvl 9
    The metaball stuff is really cool. But I'm not sure how performant it is.

    Couldn't you conceivably do this with a panner and a flow map?
  • D4N005H
    Offline / Send Message
    D4N005H polycounter lvl 4
    Fingus, what a cool idea, Can't wait for other people comment on this idea.

    I personally think that this idea is the best solution for the "exterior view" because the metabells are going away from the center of the screen to the sides. But for the windshield... I'll just wait for others to express their ideas about it.... .

    ShadowyIdleElectriceel.gif
    Look at the water drops in exterior view of Forza6 game.
  • gilesruscoe
    Offline / Send Message
    gilesruscoe polycounter lvl 7
    Not sure if you'll find this useful or not. I wrote a pretty long blog post for my studios dev blog. Starts with some simple principles and ends up doing metaball stuff for various effects. Might be worth a read.

    https://www.oysterworldgames.com/unity-shader-tutorial-2-5d-imposters/

    If i were trying to implement this kind of windscreen effect though, i'd use flowmaps and masks, not metaballs. Use vertex colours to store rain flow velocities based on the motion of the car and the wipers and masks to control the look of the rain at various velocities/positions on the windscreen.
  • D4N005H
    Offline / Send Message
    D4N005H polycounter lvl 4
    gilesruscoe Thanks for your reply, I'll read that page. But would you explain a bit more about this part of your reply:
    [...]Use vertex colours to store rain flow velocities based on the motion of the car and the wipers and masks to control the look [...]

    EDIT: After skimming the blog post, I should say that that is a great way to make metaballs, I do practice that and thanks for sharing that with us. But that is useful for some other cases.
  • Obscura
    Offline / Send Message
    Obscura veteran polycounter
    D4N005H - Since the original balls are actual 3d planes, you can add colliders or use as rigid body or anything. Later today I'll continue with that. It seems like it isn't completely clear what I did so I'll share the full setup too.

    Fingus - The formula for generating the normal use around 70 instructions so its fine. Even the scene capture is more expensive hehe. So its a lot more cheaper than I thought.

    Anyways, my first idea on the first page worked with controllable panners. I controlled it with anim but you could control it with anything, maybe even with a flow map.
  • Obscura
    Offline / Send Message
    Obscura veteran polycounter
    Okay here is the version with rigid bodies. I replaced the planes to spheres so they can't overturn. The collider of the sphere has to be smaller than the sphere itself so it can partly go inside an another sphere so you get the metaball effect. I think this is the final version, It looks enough good in my opinion. The further work can be done with coding the rigid bodies to spawn/destroy etc.
    [ame="



    D4N005H - The exterior drops can be done with my first idea (controllable panners). As you see there is a max amount of drops because they disappear after time, probably they get 0 opacity as I also did, and get new position, then setting the opacity to 1 again.
  • Obscura
    Offline / Send Message
    Obscura veteran polycounter
    I made a script for spawning/destroying the physics spheres so I can maintain any amount of spheres at the same time. I used 150 for this video. It would be also possible to destroy every spheres which has been fallen down from the glass. I also added the windshield wiper to interact with.
    [ame="
  • D4N005H
    Offline / Send Message
    D4N005H polycounter lvl 4
    Prefect, awesome Obscura. But please read this, I think there is relatively a big misunderstanding.

    Did you do something like what I did? I mean did you do this things below:

    Created 3D Spheres
    added rigid bodies to the spheres
    used a RenderTexture
    applied some shaders to the rendertexture so that those spheres look like water

    am I right? if so... that's exactly what I did except your shader is way better than the one I used. right?

    so if the structure of this is same as mine, there is still one big problem,please look at the video below to see what the problem is:
    [ame]

    I hope everything there, was clear.
  • Obscura
    Offline / Send Message
    Obscura veteran polycounter
    Don't move the meshes that you are rendering from. Or if you do, then do it slowly. Rotation should work though. Damping could also help, or maybe physics material. There you could control friction. Maybe you don't need to move at all, you could just use force fields, so it would be more controllable.

    It looks like you do almost the same as me by the way.
  • D4N005H
    Offline / Send Message
    D4N005H polycounter lvl 4
    Hehe yes, we did the same concept but with different shaders, I was going to think about the normal maps and refractions... but with your shader, I wouldn't think about them because everything is okay with it.

    So... we conclude that physics of the water drops (lets call it particle emitter) should be fixed, whether these particles are using your shader or the shader that I've used for them.

    (More info of the shader that I used is in the Mr.Chadwick's post in here)
    Don't move the meshes that you are rendering from. Or if you do, then do it slowly...
    How on the earth I haven't thought about it... wow... Yesssss that will just solve this problem! simply I will add force to a Kinematic rigid body with 1/100 or lower the actual speed of the car, and then bang the problem will get solved, hopefully. About the damping and other stuff... those will work if I do what I just said, Mr.Chadwick said about damping,too.

    But before everything, I should replicate your shader, Obscura.
  • Obscura
    Offline / Send Message
    Obscura veteran polycounter
    Oh one thing, I didn't use particle emitters. I made a script for spawning actual rigid bodies inside a rectangle shape. Don't know if this makes any difference. Maybe the only difference is that then I can control them individually like destroy the fallen ones.
  • D4N005H
    Offline / Send Message
    D4N005H polycounter lvl 4
    Yes, I did the same thing, I spawned them by script, I meant generally call them particle emitter. I really thank you for your help, will do some thing in the next days and share results here, I was wondering if you could explain a bit more about the craetion processes of the metaball shader.
  • Obscura
    Offline / Send Message
    Obscura veteran polycounter
    Sure. First you need the same mesh setup as I have. One black plane or shape to spawn the rigid bodies to. The rigid bodies has to be spawned randomly inside a rectangle shape to get realistic look. For the sphere rigid bodies, I used this material, so only their center will be white, and they go invisible on the edges:
    NnwB67g.jpg

    I used 2 scene captures for getting high and low res heightmap too, but you could do it with 1 highres+ more tricks in the main shader (it would need filtering too). Then I used them in this way in the main material. I used the exact same setup as what I linked, its a realtime normalmap generator with sobel filter. There are some options to control the resolution, smoothness etc, but these also depends on how big texture you are rendering, and how you set up the sphere material. You can get differently looking effects and blending with playing with the parameters... I used translucent blend mode (alpha blend) for the window, and for the spheres too.

    VPufHyW.jpg

    - You can see a part of the background on the rendertarget2d because the UDK scene capture cant record in 2:1 aspect ratio... So basically the half of the capture is useless. For making that not visible on the windshield, I adjusted the uvs of it to use only the necessary part of the texture.
  • D4N005H
    Offline / Send Message
    D4N005H polycounter lvl 4
    Really thanks Obscura!!
    Unfortunately ShaderForge doesn't have a node for converting height map to normal map, so I did it without this node (shown below), Also one weird thing is that ShaderForge doesn't have Distortion, so I used refraction option but I can't connect the refraction to the multiply node, do you know any other way to do so? I just used some numbers and also a mask for it which means this shader is applied on a plane which is on an other plane, the plane below this plane, is the windshield, look at the images below:

    59873_sf_rendertexturenew_8152015.png

    zlwumt.jpg
    However this is what I got now: (I didn't only haven't had used refraction at the time I was creating this GIF)
    LargeCourageousIndochinahogdeer.gif

    However, I hopefully will be rewriting the spawning spheres script and I will simulate gravity for each sphere and also I will be adding artificial gravity for the windshield so that the spheres would behave as water drops, tomorrow. I am searching how to manipulate gravity of rigid bodies in Unity3D right now.

    EDIT: Found a way to make gravity for each water drops and the windshield collider in this video:
    https://www.youtube.com/watch?v=gHeQ8Hr92P4

    EDIT: Decided not to add gravity to each spheres , because it leads to the high usage of CPU. Frame just got below 20! , so I just removed the gravity.
  • Obscura
    Offline / Send Message
    Obscura veteran polycounter
    1. Why you would need double glass mesh? Maybe you want clear coat effect? That can be done without double mesh. Layering transparent meshes is expensive, so I'd recommend a shader based solution, if possible. You don't use PBR right? Then its easy, you need double specular, which can be done with custom lighting.
    2. Why you would need alpha test if you already use transparency with alpha blend?
    3. Do ShaderForge has "custom" node? You could write hlsl code into it, as I did.
    4. Distortion and refraction should be the same thing.
  • D4N005H
    Offline / Send Message
    D4N005H polycounter lvl 4
    Obscura wrote: »
    1. Why you would need double glass mesh? Maybe you want clear coat effect? That can be done without double mesh. Layering transparent meshes is expensive, so I'd recommend a shader based solution, if possible. You don't use PBR right? Then its easy, you need double specular, which can be done with custom lighting.
    Okay I didn't know that, thanks. I actually want to have refractions only for the water drops, in ShaderForge, there is a refraction option but it affects all the surfaces that has the alpha value below 0.5, since the glass it self is under this value, then the refraction affects it, which results a refracted windshield, which is not good. And it is PBL , is it bad or good to let this be PBL? I thought PBL makes the object more realistic by real time reflection (then after using this option this happened:
    105w8eb.jpg
    No shadow, no reflections from other objects, only it reflects the skybox apparently. )


    2. Do you mean Alpha Clipping? I used it to remove the refraction for whole glass, ... I'm new to the shaders,though. Here is without alpha clipping:
    317jypj.jpg
    3. Yes it has. Okay, Might be a weird question, but what do you recommend me to do now?

    EDIT: All I did right now was removing the glass mesh and removing the refraction.
  • Obscura
    Offline / Send Message
    Obscura veteran polycounter
    3. Make the same setup as I made with the custom node.

    Refraction setup should be the same as my distortion setup, then it will be affected by the normal map and fresnel.

    2. Yeah you don't need the alpha clip thing at all.

    1. Using custom lighting is not really possible with PBS, at least its not recommended because you would have to redo the whole PBS.. So maybe stick with the double mesh then.
  • D4N005H
    Offline / Send Message
    D4N005H polycounter lvl 4
    I did tried to do exactly what you did, but the "Multiply" node that was connected in the distortion is not possible to be connected in Shader forge. Here is what will happen:

    110at0w.jpg

    Author of ShaderForge says refraction needs vector2.
  • Obscura
    Offline / Send Message
    Obscura veteran polycounter
    I think I understand what you mean. Then I think you can make custom distortion. It is fairly simple, it doesn't do more than adding things into the coordinates of the scene texture. I'm making an example right now.
  • Obscura
    Offline / Send Message
    Obscura veteran polycounter
    Here it is... I made it in a new material so I couldn't access to the generated normals, but you can replace the normalmap from my example with the generated normals.
    Custom distortion:
    n0rBLkY.jpg
  • Obscura
    Offline / Send Message
    Obscura veteran polycounter
    Well... Now I see your issue, and you need a component mask with masking the r and g channel. Then it will work. Or use vector2. So it is not the multiply that is causing the issue but the vector3.
  • D4N005H
    Offline / Send Message
    D4N005H polycounter lvl 4
    3. Okay.
    2. Okay.
    1. You've said it is not optimized, I should do something for this.
    Okay let me try this.
  • Obscura
    Offline / Send Message
    Obscura veteran polycounter
    I would say just keep the double mesh. It would be incredibly hard to recreate the PBS for you.

    How much cars you want in the scene? How much of them will be visible at the same time? Cause having overlapping transparency only once is ok. There are just better solutions, but if you want it only once, then its not a problem.
  • D4N005H
    Offline / Send Message
    D4N005H polycounter lvl 4
    Super weird... the component mask that is connected to the Fresnel only has R output...
    2aeyvf7.jpg
  • Obscura
    Offline / Send Message
    Obscura veteran polycounter
    Fresnel returns a dot product between the camera vector and normal vector, so its an "1 channel value", and not a vector, this is why you get only red channel. You should put the component mask after the multiply. But if you use Vector 2 in the multiply then you don't even need the component mask.
  • D4N005H
    Offline / Send Message
    D4N005H polycounter lvl 4
    Obscura wrote: »
    I would say just keep the double mesh. It would be incredibly hard to recreate the PBS for you.

    How much cars you want in the scene? How much of them will be visible at the same time? Cause having overlapping transparency only once is ok. There are just better solutions, but if you want it only once, then its not a problem.
    One car, the players car only when he is in CAM1,CAM2 or CAM3.
    CAM1 = Interior camera;
    CAM3 = An exterior camera that lets the player orbit it so that he can see the car.
    CAM2 = Rear view camera;
    CAM4 = Cinematic camera;

    other cameras and other cars only use the animated normal map (flip book normal map, still haven't made it)
  • D4N005H
    Offline / Send Message
    D4N005H polycounter lvl 4
    Obscura wrote: »
    Fresnel returns a dot product between the camera vector and normal vector, so its an "1 channel value", and not a vector, this is why you get only red channel. You should put the component mask after the multiply. But if you use Vector 2 in the multiply then you don't even need the component mask.
    Ah okay, I just wanted to recreate exactly the photo that you've showed.

    Still haven't got time to study shaders deeply... I should study them.
  • Obscura
    Offline / Send Message
    Obscura veteran polycounter
    D4N005H wrote: »
    One car, the players car only when he is in CAM1,CAM2 or CAM3.
    CAM1 = Interior camera;
    CAM3 = An exterior camera that lets the player orbit it so that he can see the car.
    CAM2 = Rear view camera;
    CAM4 = Cinematic camera;

    other cameras and other cars only use the animated normal map (flip book normal map, still haven't made it)


    Okay, the double mesh can be still a problem regardless of what kind of normalmap you are using, if you want clear coat on the other car's windows. So maybe forget clear coat, or use it only on the player's car. Or maybe there is an option for clear coat in SF? If there isn't, then use it on the player's car, or don't use it at all. At least for now... Later, when you have better understanding in shaders, you can get back to this, and recreate pbs with including option for clear coat using custom lighting.
  • Obscura
    Offline / Send Message
    Obscura veteran polycounter
    To make it clear, this is what the clear coat effect does:
    sMvVkFr.jpg

    And this can be the only reason to use double mesh. I THINK you need additive transparency with the double mesh for getting correct reflection values at the end, but I'm not sure in this.
  • D4N005H
    Offline / Send Message
    D4N005H polycounter lvl 4
    Yes you are right, I've lost too much hours for this because I'm very new to shaders and I'm super tired now.
    I think I better forget the distortion for now, since it is not working, although I tried the way that you've just provided. Anyways, thanks Obscura, if you wouldn't be here lots of things for me would be too much unclear.

    Obscura wrote: »
    To make it clear, this is what the clear coat effect does:
    sMvVkFr.jpg

    And this can be the only reason to use double mesh. I THINK you need additive transparency with the double mesh for getting correct reflection values at the end, but I'm not sure in this.

    No no... why should I need this? I just don't want all the windshield have refraction - as I've mentioned in my recent post here, otherwise this clear coat effect is useless I think... otherwise I'd say nonsense if I have it :D .
  • Obscura
    Offline / Send Message
    Obscura veteran polycounter
    Yeah its a non-sense saying this. It goes against how it works in the real life :D There, if you have a windshield for example, and there is water on top of it, the windshield will still have specular reflection under the water, and the water will have an another. The windshield's one will get distorted by the windshield's shape, and the water's one will be distorted based on the water's shape.

    Distortion and clear coat are 2 different things btw.

    Anyways, if the refraction doesn't work for some reason, the use the custom one that I made for you, but actually I can't imagine it not working. An image would help as always.
  • D4N005H
    Offline / Send Message
    D4N005H polycounter lvl 4
    :D yep, Okay just let me retry making this custom one tomorrow, I believe if some questions get answered, this would be super easy for me.(Please look at the image below)
    t7c3de.jpg
  • Obscura
    Offline / Send Message
    Obscura veteran polycounter
    Scene Texture Sample - No, its actually the render of the game camera or the viewport. SF should have scene texture node.

    Fresnel's channels- Thats a good question... The fresnel values should be between 0 and 1, and it should be a float and not a vector. I take a look at it now.

    Edit - Ok I took a look at it, and it looks like UDK gives some channels to fresnel, and it has the same values in every of them. Don't ask why, they are the exact same values...I think its because the creator of the fresnel node didn't include a component mask into the fresnel node's code.
  • D4N005H
    Offline / Send Message
    D4N005H polycounter lvl 4
    Okay. So there would be no need for for mask then? and ShaderForge doesn't have the render texture sample. I've searched the Internet, nothing came up.
    I am trying to do other approaches to see what will happen.
  • D4N005H
    Offline / Send Message
    D4N005H polycounter lvl 4
    Obscura wrote: »
    Well... Now I see your issue, and you need a component mask with masking the r and g channel. Then it will work. Or use vector2. So it is not the multiply that is causing the issue but the vector3.

    Well... converting it to vector2 causes compiling error:

    24fb79y.jpg

    would you do me a favor? would you just give me the shader that you've made? (The shader that you've used for the final version of the meta balls, I mean) however I really like to know what's wrong with my shader, seems it is a ShaderForge's fault. Only refraction doesn't work,though.
  • Obscura
    Offline / Send Message
    Obscura veteran polycounter
    I took a look at the SF nodes and the equivalent of scene texture is scene color :P
    Now you should be able to recreate the custom distortion.
    The shader I made wouldn't work at you because its a UDK file.
  • Obscura
    Offline / Send Message
    Obscura veteran polycounter
    Also. Have you tried a lower value in the multiply? Something smaller than 1. Maybe the formula that SF use for refraction is different than the one in UE , maybe something like mine , so it expects a small value.
2
Sign In or Register to comment.