RGB blending shader\tool?

Drav

Hey dudes. I am doing some work in quite a few different engines at the mo all involving RGB masks to blend textures together on objects, and in each one there are sod all tools to help you do this in engine, and a monster ragedown is building the more I work in this way.......

So, my first question is, is there a tool that can help me do this?

Anticipating the answer will be no, what is the best way for me to build one?

I have shaderFX and this could be a solution, but ideally (eventually) I would like to make something simple and easy to use....probably a max shader?

However, the big problem with this is that I still cant paint ON the model, I am still stuck drawing Red in PS, hoping for the best, saving, oh thats shit cos i cant see, repeat.......


Really, in short, I want a feature like UDKs vertex paint mode, but with RGB. So you draw on the model in the correct colour, it shows up as the correct detail texture as per the shader, and also updates my flat mask texture with the colour I've drawn.....think something like this, but with a mask texture not vertex info:


[ame]http://www.youtube.com/watch?v=o8y4m6YnEzM[/ame]


So, considering the Max shader is of limited use, what option have I got?

Can I use something like Zbrush, Mudbox or Bodypaint to work as described?


Considering quite a few games are using RGB masks to blend textures, I am amazed there seems to be no non in-house tools to do this in an artist friendly manner. It could be such a powerful tool it seems, but unless I am missing something really obvious I cant see a way to do it without writing a tool.....

Please, if I am being retarded and its actually just 'tick this box and this box', please enlighten me!!!!

Drav

Thinking aloud for a sec, I could probably paint a texture on in mudbox, and extract the alpha channel as colour info for a mask......

I'd need a way of keeping the texture scale of what I draw the same as what would be masked on......hmmm this sounds promising

Right, right, polycount is not my blog, I'll shut up and investigate......

Ben Apuna

Sorry can't help you with a shader, just wondering why your using a texture based mask rather than a vertex color based mask?

It seems like if this was a level we're talking about then you'd end up with another giant texture perhaps the same size as a fully unique lightmap.

A question to those in the know: what would be more efficient? Image based or vertex color based mask? Yeah I know "it depends...", how about a 200000 triangle level VS. a 2048 texture? (sorry don't mean to hijack your thread Drav, just curious...)

It would also seem that you would then want your level editor, be it your 3D app or engine based editor to allow you to paint whatever type of mask you need.

Having tried to paint a RGB texture based mask for a level in Mudbox in the past, it was a major PITA due to camera clipping issues. I had a slightly better experience painting that mask in Modo due to the way it dynamically handled camera clipping.

Drav

Ah its not for a level, its for adding detail and dirt to buildings, on a seperate UV channel to the base texture.

In short, there aint enough vertexes to do it well with vertexes, and the image based method allows you to use different brushes, rather than do the edges with the shader. Swings and roundabouts, but the idea is the same, allows you to have a tiled base texture, which you then detail in with other tiled textures to make it non repetitive.....


By all means hijack away Ben, I'm interested too.....

animax

It's really easy to create that shader in ShaderFX.I had recently done that for my work, which had a similar requirement.I was using 3dsmax 2011's viewport canvas to paint the RGB mask and see it right there.

I was using 3 diffuse and normal textures on uv channel 1 and blending them together with a RGB mask in channel 2.

Are you familiar with shaderFX?

Drav

Outstanding animax, thats just what i was looking for......the paint feature in max makes the shaderFX solution work, and I can replicate the shader perfectly this way.......lovely stuff

Ye I am doing exactly the same as you......3 tiled dirt\crack textures on UV1, mask on uv2.

Thanks very much!

Drav

Hey Animax, you mentioned you were drawing the RGB mask in max 2011 and using the shader to display the textures correctly....

I have got this shader working as it should using the mask and the 2 channels, but have no idea how you'd get 2011 rgb mask to update to draw the correct texture from the shader in realtime. I have the mask in shaderFX loaded as a texture, which is only updated when saved, rather than realtime.

Did you ever get you setup to work so that when you draw red in max 2001, it updates in realtime by drawing brick in the viewport, or did you draw, save, update the shader?

I cant see it being possible with the little i understand about shaderFX to be able to do it in realtime, but then I am a bit of a noob when it comes to technical stuff!

animax

Hey Drav,

Sorry for the late reply.Yep i did the way you mentioned at the end.It was not a real time feedback.I just setup the mask texture in viewport canvas and after each paint over it would save and update.

Sorry for the misunderstanding.

Drav

Ye thats what ive got too......I'm looking for a way to get it to work in realtime but i dont think i can do it as viewport canvas only works with standard materials and I dont think theres a way to make it work inside a directx shader.

Not a bad solution anyway tho, thanks for the help!

animax

Ok.I am confused.Viewport canvas can work on directx shaders.I just recorded a video to show what i meant.

What i did is put the mask map into the "software renderstyle" material also.So from viewport canvas it choose automatically the mask map to paint on.



Just wanted to make sure we are on the same page.

If anyone does figure out how to do the realtime painting.Please post here.

Drav

Ye animax we're on the same page.....paint on the mask, preview in the shader. When I said didnt work on directx shaders I meant you couldnt insert it into a directx shader, to paint using the shader materials in realtime. You still have to paint with the mask visible, and see its effect after.

Its still a million times better than painting masks in photoshop, thanks for pointing me in the right direction. I'll post up some stuff here when I've played a bit more......

animax

Glad that got cleared.Good luck mate.