Vertex colors not working, 3ds or UDK problem?

Razgriz

So, I've been trying to export vertex colors on my mesh using FBX into UDK, so I can use a vertex color node in the material editor to effectively paint bricks different colors on my mesh. It was working for a bit, I think it stopped working when I hooked up my cubemap reflections, but I can't seem to get it working again no matter what I try.

It seems like there shouldn't be any problem, in max it paints just fine, even though all the vertices are welded, there are UV seams on every brick (it's really one brick pulling from the same texture information), so there should be separate UV's to be painted individually, correct? Like I said, it looks fine in max, but now doesn't work in UDK. The UDK vertex painting works fine, but that doesn't allow me to paint brick-by-brick, and I just get a blurry mess.

Any ideas? Thanks!

toddiej

I think there is an option to 'replace existing vertex colors' or something when you imort. Don't have UDK here at work so I cant confirm where it was. I think it defaults to off. If you just re-import the model by right clicking it I think it'll default to not replacing vertex colors. You'll have to manually re-import each time and make sure that the replace vertex color is selected.

Razgriz

I just found the option you mentioned, and reimported several meshes, to no avail. It seems like it might have been the problem, since it was set to default (presumably because I've clicked "reimport mesh" on nearly everything"), but it doesn't seem to have helped at all.

toddiej

I'd probably try using .ase then unless you needed to use .fbx for a certain reason.

Jeff Parrott

I thought there was a bug or something where you have to export as an .ase.

Razgriz

.ase doesn't work either, which leads me to believe I'm doing something wrong in UDK. And, like I mentioned, I had it working for a brief moment... are there known issues when using a cubemap reflections, or anything like that?

Ace-Angel

http://www.polycount.com/forum/showthread.php?t=90447

Razgriz

I read through that topic, Ace, as well as some of the links to stuff Virtuosic was working on, but I don't entirely understand how it relates to my problem, or what I should change based on the information in there.

sprunghunt

you probably have a transform node on the reflection map? correct?

that's what's interfering with the vertex colours.

Ace-Angel

You can check Pixel Camera box in your material to allow Vertex and Transform to work, but inorder for that to work, you must have a Reflection Node connecting into your Transform.

Either Mutiply/Add your Reflection to whatever is entering your Transform, or subtract reflection by itself and again, Multiply/Add to whatever is going into Transform.

The solution Vailias posted up in that thread is the pure math version, so if you're not doing anything fancy with Transform node, you might as well cut it out and use the pure math version.