Help me figure out how to texture this 3d tiles system

Skinpop

I'm more of a programmer than a 3d artist and this is more of a system design question than a 3d art issue so I'm not sure if this is the right place to post but I'm hoping someone here might be able to help me

Anyway, I'm currently programming a 3d "tile" system for a personal project. Each tile is uniform in x/y and can be fitted next to a number of other tiles as is usual in these kinds of systems. 

To have some variation each texture is also broken up into 4 tiles (actually 4x4 but that's not important here), so in the engine the UV coordinates of a tile is offset by 0.25 * (TileCoordinate MOD 4). Basically the subtexture for each tile moves one step for each step in x,y and z on the map grid and then wraps around ever four steps. Hopefully this picture illustrates this concept adequately:



The tile models use two sets of uv projections (plus one for lightmaps). One set for top down "ground" cover and the other one for "walls" or the sloping part of a tile. Finally vertex colors and some shader trickery is used to blend ground and wall textures together in a pleasant way which is the standard way to do terrain texture blending.

The problem I have is what to do with the corners.
Since the sub-texture offset is dependent on the tile position on the grid corners end up not lining up most of the time.  For example, as shown in the picture above the left side of the corner is a "1" while the right side is also is "1", ideally we'd want a 1,2,3 but that's not going to be possible here.

My first naive solution is to somehow generate all transitions between tiles for the corners and keep them in a separate texture map. This solution is quite nasty because some of my tiles are 2-3 unit tall combinations of other tiles and I'm not sure how to do that for them:



Another thought is to use two vertex color channels for the wall and then store the subtexture offset instead of the texture ID corresponding to that channel - and then blend between the subtextures on the corner using vertex colors like any other blend texture. Thinking about it, this might actually be the best option.  

I'd love to hear suggestions.
Maybe this problem has a super easy solution I haven't thought about. If so I'm dying to hear what that might be. 

icegodofhungary

I read this a couple of times and I'm not sure what you're trying to accomplish from a 3D modeling standpoint. So let me try to put it in my own words and tell me if I'm right. You're programming a modular (you're calling them tiles) asset generator, where the program takes meshes and assembles them, then applies a texture via some algorithm. You have it worked out so that your vertical surfaces and horizontal surfaces are in separate UV spaces. You use programming magic to blend them together when applied to the 3D mesh. As part of a way to break up tiling textures, you have it set to automatically pull from a portion of a UV space, and move onto the next section in succession. So your first vertical surface gets 1, second gets 2, etc. Then it repeats after 4.

Your problem is that on the corner, they don't really fit the texture space as you have two vertical surfaces meeting at a corner. You can't draw two squares of the texture across the object for some reason. Is that correct?

If you can't use unique textures for every object, and must draw from some atlas, why not just use tiling textures (textures that repeat in U and V so that they have no seams) rather than trying to draw from one piece of an atlas? Then you could use tri-planar texturing (project a texture from 3 directions on the mesh, and blend them so there's no stretching). This eliminates a need to look up a texture from an atlas at all. Well, other than storing your whole tiling texture in an atlas of tiling textures. Then you can still use vertex coloring and/or subtextures to cover up repetition and add variety. This would take care of your corner piece and cut down on the texture space used as you wouldn't be using 4 squares for what essentially is the same kind of object (wall/barrier thing).

rollin

Why do you want to do it like that? 
As I understand your issue it's not a modeling issue but a logic issue. 
This tile-id related issue is really sounding like a basic misconception or flaw of your approach.
Imo your problem arises because you have 4 tiles on your texture but you have 6 tiles on your model. If you straighten it out you go from ?1234?1234?1234.... but your texture is 123412341234....

My suggestion would be: re-think your intentions. Why do you want to do it like you said and not like the "usual" approach which is more or less what @icegodofhungary is describing.

Skinpop

icegodofhungary
It's not for offline generation, the system generates tiles in real time on the fly(basically by copying a base-tile and offsetting its position and texture coordinates) to be used directly in the engine on the same frame.

But yeah thinking about it I should have used tri-planar projection from the start. I didn't want bloated vertices and tried something clever(foolish).

I find that breaking up a tiling texture into several sub textures parts helps a lot with variety, especially for cliffs and things like that. Probably less so for a wooden barrier but this is a low-res project so textures are pretty much free.

But thanks a lot, it works perfectly now! 

@rollin
As I prefaced in the original post, this is indeed a system design (logic as you say) issue rather than a 3d art issue/modeling. I should have clarified that the ? is nothing special it's the same as the other tiles, a 1,2,3 or 4. I put it in there to highlight that I wasn't sure what to do about them corners(since x and y vary independently).

poopipe

Personally I think I'd add specific corner textures to the atlas and treat them as a special case

Triplanar is easy but rounded corners like those illustrated breed blending artefacts and you're also left with no real way to add any unique information so I'd expect that once art happens you'll find there are limitations you don't like.

As an aside, I wonder if some sort of wangtile type logic could be applied to this