Home Technical Talk

Vertex Colour Bleeding, 3DS Max Biped?

polycounter lvl 10
Offline / Send Message
FiftyTifty polycounter lvl 10
Edit: Me questions concerning Biped are a bit further down the page. Ya'll find 'em.

Hullo lads.

Spent a few hours the other night modelling a super low poly version of an elf, aiming for ye oldé visuals of the late 90s (N64, PS1, Morrowind, early Runescape). And boy, low poly modelling is a pain in the arse. I spent less time sculptin' a skinny human model inta a muscular brute, fer crying out loud.

Anyhow, I've hit a snag with the vertex colours. Namely, for the eyes.


As you can see, the right eye is a solid black (good), but is bleeding into the eyelids and brow (bad).

With the left eye, the black isn't solid and is bleeding into the bottom eye lid. Aye, this is also bad.

What I want to do, is just use vertex colours to only colour the eyes solid black. I've got one of the ol' Runescape models, that I've been using as a reference fer polygon density 'n' topology referencin'. Trying ta get the same effect as with the eyes:


Now, I know the eyes on me own low poly model are, err, horrific. Believe me when I say a lotta that, is 'cause I was trying to see if splitting the quads would make any difference. Obviously not, and I'll pro'lly have to redo them. Bleh.

So aye, any advice for this lil' guy? I'd like ta avoid using textures wherever possible, as I'm trying to imitate that antiquated art style as much as possible, as well as a learnin' experience fer getting to grips with vertex colours.


Edit: Ah right, linking me model would help. 'Ere it is lads. https://www.mediafire.com/?mfl13fu3yz47enk

Replies

  • PolyHertz
    Options
    Offline / Send Message
    PolyHertz polycount lvl 666
    Select the eye tris in the Vertex Paint modifier, then you can paint just them. Just tried it on the file you posted without issue. If it doesn't work, your modifier stack may be the issue (never seen that many Vertex Paint modifiers used at once on a single mesh). Also you should work with consistent colors + make sure shadows and AO are turned off in the viewport so you don't misunderstand what's going on with the colors.
  • FiftyTifty
    Options
    Offline / Send Message
    FiftyTifty polycounter lvl 10
    PolyHertz said:
    Select the eye tris in the Vertex Paint modifier, then you can paint just them. Just tried it on the file you posted without issue. If it doesn't work, your modifier stack may be the issue (never seen that many Vertex Paint modifiers used at once on a single mesh). Also you should work with consistent colors + make sure shadows and AO are turned off in the viewport so you don't misunderstand what's going on with the colors.
    Was trying to layer the vertex paints (paint one vertex, new modifier, paint other vertex, etc.), to see if that made any difference. Didn't, so I just chucked 'em out right this moment.

    I've tried selecting the tris and painting 'em, but it doesn't work for me. 'Eres me steps:

    Right-click Head mesh -> Convert To Editable Poly (to remove modifiers, whilst keeping current vert colours)
    Make sure no vertices/polys/lines are selected in Editable Poly modifier
    Add VertexPaint modifier
    Select polygon in the VP modifier
    Paint with brush

    It does nothing; no amount of clicking affects the bizarre bleeding. If I select an adjacent polygon, such as one of the solid black eye triangles, the colour will change, but that cocks up the eye.

    Bit bloody odd. You using 3DS Max 2014 as well?

    Edit: Ta show what I mean:
    Below image has no change.

    But if I select an additional poly and paint:


  • PolyHertz
    Options
    Offline / Send Message
    PolyHertz polycount lvl 666
    No I'm on Max 2016. I know Max 2012 had some rather bad vertex color issues, but no idea on 2014...
    The steps seem fine, assuming your selecting subobject level 0 first before creating the vertex paint modifier too?
  • FiftyTifty
    Options
    Offline / Send Message
    FiftyTifty polycounter lvl 10
    PolyHertz said:
    No I'm on Max 2016. I know Max 2012 had some rather bad vertex color issues, but no idea on 2014...
    The steps seem fine, assuming your selecting subobject level 0 first before creating the vertex paint modifier too?

    Subobject? That's where you choose to select either verts, polys, edges or elements, aye? I've tried having the Editable Poly modifier in vertex selection mode and then applying the vertex paint. Did the same for poly selection mode, as well as having no selection modes active.

    Nada.

    Bloody pain I tell ya. You'd think Autodesk would have fixes for the older software, for legacy support and all that.
  • PolyHertz
    Options
    Offline / Send Message
    PolyHertz polycount lvl 666
    It's how max refers to the component modes internally. Goes like this:

    Sub-object level 0 = Object level , 1 = vertex , 2 = edge , 3 = border , 4 = face , 5 = element.

    Many of the modifiers determine how they're supposed to work based on the sub-object level the user is in before selecting them, not just what components are selected. In fact, component selections are completely ignored if your not in their component mode (sub-object level) before entering the modifier.

  • FiftyTifty
    Options
    Offline / Send Message
    FiftyTifty polycounter lvl 10
    Aye, made sure to just have the object selected like so:



    Nae dice, it still doesnae paint the individually selected polygon.
  • cptSwing
    Options
    Offline / Send Message
    cptSwing polycounter lvl 11
    Have you tried coloring the vertices directly in the Editable Poly? (Also, in-engine your eye-triangles will likely need to be detached from the remainder of the mesh to prevent color bleeding)
  • RN
    Options
    Offline / Send Message
    RN sublime tool
    Some of those solid black triangles are already detached from the mesh, otherwise there would be that same transition effect in the skin triangles.
    Select all vertices of the eye region and use Weld with a low threshold value to join all duplicated vertices, then select the triangles of the eyes and detach to new elements (they should stay in place, but become physically separate).
    Then you can get the hard edges you want with vertex colouring.

    When you colour a vertex, that colour is distributed to all triangles that share that vertex.
  • FiftyTifty
    Options
    Offline / Send Message
    FiftyTifty polycounter lvl 10
    cptSwing said:
    Have you tried coloring the vertices directly in the Editable Poly? (Also, in-engine your eye-triangles will likely need to be detached from the remainder of the mesh to prevent color bleeding)
    Just tried. Ya do it by selecting the vertex -> Vertex Properties -> Edit Vertex Colors -> Color and changing it, aye? Got the same result as vertex painting; those polygons refuse ta stop being eejits.

    Gruh, most engines don't do decent vertex colors? The engine I'm looking at is Amazon Lumberyard; solid CryEngine performance 'n' completely free. Bummer, but I guess it'll be fine. Just got to make sure the vertices at the disconnected edges are in the exact same position, yeah?


    These vertex colours are gettin' frustrating. Would be a bummer if it's not some weird problem on my end, and is an actual issue with 3DS Max. Any significant problems with detaching small details when using Biped, animating, skinning, etc.? 'Cause it's looking like that I'll either be doing that, or using some bastardized, tiny (2x2? 16x16? 32x32?) uv map to get the colours sorted.


  • FiftyTifty
    Options
    Offline / Send Message
    FiftyTifty polycounter lvl 10
    RN said:
    Some of those solid black triangles are already detached from the mesh, otherwise there would be that same transition effect in the skin triangles.
    Select all vertices of the eye region and use Weld with a low threshold value to join all duplicated vertices, then select the triangles of the eyes and detach to new elements (they should stay in place, but become physically separate).
    Then you can get the hard edges you want with vertex colouring.

    When you colour a vertex, that colour is distributed to all triangles that share that vertex.
    Hmm. At first glance, the example model has duplicate vertices as the edge of the eye (by the nose) disconnected, with the upper half of the eye also being disconnected from the bottom half of the eye. Funky.

    Aye, seems like I'll be detaching the polygons. Going to have to tidy up the polys around the eyes, ta make up for me barbaric plastic surgery.

    Anything I need ta be aware of with disconnected polygons, as far as animating/skinning/etc goes?
  • RN
    Options
    Offline / Send Message
    RN sublime tool
    Provided that you assign the same bones and weights of the original vertex to the duplicate vertices, it should animate fine.

    The same thing happens when you UV map your model. You're duplicating vertices like that, at the seams, but 3ds hides this from you and shows you only one vertex. It also happens with smoothing groups so the model can have different normals at the same point.
  • FiftyTifty
    Options
    Offline / Send Message
    FiftyTifty polycounter lvl 10
    RN said:
    Provided that you assign the same bones and weights of the original vertex to the duplicate vertices, it should animate fine.

    The same thing happens when you UV map your model. You're duplicating vertices like that, at the seams, but 3ds hides this from you and shows you only one vertex. It also happens with smoothing groups so the model can have different normals at the same point.

    UV Maps are unnecessary if I'm sticking to using just vertex colours, aye? 'Tis what me low poly reference models have; characters and creatures use vertex colours, whereas architecture and vegetation have actual uv maps (to utilize textures).

    On a different note, I've been screwing around with 3DS Max biped. It's bloody tough to get it to fit into body mesh. Did me best, and applied a physique modifier to me model. Struck a pose to highlight problem areas, and I ended up with this beast:


    Tried looking through Autodesk's documentation about biped and fixing errors. Supposedly, ya can edit what are known as "envelopes", which are these capsule-shaped wireframes, that are related to the biped's limbs. Found absolutely nothing on how to actually manipulate 'em; it's like reading an ikea manual without any pictures.

    'Eres me current scene, including the biped, me mesh and the hi-poly body I used as a reference. https://www.mediafire.com/?r7mjj627c98na7j

    Mind pointing me to any decent docs on how to fix these poly-gore-gons? Dinnae fancy me lasses' feet to keep dismembering themselves.

    Edit: So, I figured out how to see these envelope things. What I did, was select all of the character parts (not the biped), then apply the skin modifier. With all of the objects still selected, I added all the bones to the skin modifier. Then, after clicking "Edit Envelopes", I can select the bones and see how they affect the other objects.

    This alright, or a bad idea due to all the bones being added to all the objects, even if they shouldn't be affected by 'em? Or is this a good thing, as it keeps in line with all the constraints, or whatever the term is for stopping bones from limitlessly moving through 3D space.
  • PolyHertz
    Options
    Offline / Send Message
    PolyHertz polycount lvl 666
    MajinCry said:
    On a different note, I've been screwing around with 3DS Max biped. It's bloody tough to get it to fit into body mesh. Did me best, and applied a physique modifier to me model.
    Use the Skin modifier, Physique only exists in Max these days for legacy support reasons.
  • FiftyTifty
    Options
    Offline / Send Message
    FiftyTifty polycounter lvl 10
    PolyHertz said:
    Use the Skin modifier, Physique only exists in Max these days for legacy support reasons.


    Figures. I've just spent a wee while adjusting the weights in a skin modifier after applying the physique modifier, and bloody hell, the hand went off into another bloody dimension. Urghk.

    I'll try me hand at using just the skin modifier. Dunno how this'll work with biped, but hey.

    Edit: Argh, the weighting in 3DS Max is clunky as all hell. Ya can paint weights on, as if you were painting in Mudbox, but you can't erase weights by just using negative weight values with the paint tool. And the envelopes do bugger all, as far as adding weights are concerned.

    Ya can shrink 'em, or make 'em bigger, and they'll affect parts like the hip or thigh. Which is bad when I'm trying to skin the right hand. But does the right hand envelope affect the right hand when I lengthen the envelope? Nope; the bottom half of the hand is still yellow, despite having more envelope coverage.

    Is there something I'm missing, or is weight painting in 3DS Max just...Bad? If it 'tis, are there any good programs dedicated to just skinning?
  • PolyHertz
    Options
    Offline / Send Message
    PolyHertz polycount lvl 666
    For a simple model like yours these are the steps I would take:
    1. Apply skin modifier, and add all bones that need to be bound.
    2. Set one bone to be a value of 1 for ALL verts on the mesh.
    3. Go through the bones one by one and select the verts you want them to influence to give a value of 1.
    4. Check the skinning to make sure everything works without issue, keeping in mind there should be NO blending atp.
    5. Select a group of verts that need to blend between two or more bones, select the bone that will be TAKING influence from the other, and set the value to something like 0.5 , then repeat for the rest of the groups of verts getting blended.

    When you skin a more complex mesh, only then would I bother with envelopes and brush-based weight painting.
  • FiftyTifty
    Options
    Offline / Send Message
    FiftyTifty polycounter lvl 10
    Took me a while to figure out the first part of the process. When ya said to stay away from envelopes, I thought you meant to not click the "Edit Envelopes" toggle. What ya gotta do, is toggle it but just stay away from messing with the actual envelopes. Cool beans.

    One snag, is that there are a couple parts of the biped that aren't linked. Namely, the head and the neck. Unlike, say, the arm and hand, the head can veer off and float in space, and the neck doesn't affect the head when moved.

    Is it possible to change that? I don't know what the term is, "constraint"? Want to have it so that when I move the neck, the head moves, as well as not being able to move the head so that it's practically floating in the air.

    I thought I could achieve that by setting the neck to have 1.0 influence on the mesh's head vertices, whilst doing the same but with the biped's head and not the neck. Nada. Tried setting the biped head -> head vertices to 1.0, then biped neck -> head vertices to 0.5, but that just made the head collapse in on itself.

    Is there some sorta constraint modifier?
  • PolyHertz
    Options
    Offline / Send Message
    PolyHertz polycount lvl 666
    If the verts have a weight of 1 then they should follow what they're bound to perfectly, almost like they were parented directly to said bone.
    Make sure that 'normalize' is enabled, and check in the weight tool that even if other bones are affecting them the overall weight adds up to 1.0
  • musashidan
    Options
    Offline / Send Message
    musashidan high dynamic range
    Also, in the Skin mod you can hide the envelopes so working at the vert level is less painfull, visually.
  • FiftyTifty
    Options
    Offline / Send Message
    FiftyTifty polycounter lvl 10
    PolyHertz said:
    If the verts have a weight of 1 then they should follow what they're bound to perfectly, almost like they were parented directly to said bone.
    Make sure that 'normalize' is enabled, and check in the weight tool that even if other bones are affecting them the overall weight adds up to 1.0


    Aye, Normalize is enabled.

    Might have just been me mucking up. I'll give the head a do-over once I've got the workflow down. Still at the learnin' stage with this whole shebang.
    Also, in the Skin mod you can hide the envelopes so working at the vert level is less painfull, visually.

    Aha! The Display category! Waaaay easier to mess with the verts now.


    And, alas, I've hit another snag; the elbow and bicep collapse in on each other. I remember finding an issue like this with Maya waaaaay back in the old days (2010? 'Twas when I was 12), and I think the fix had to do with there being two types of bones, one bendy and squishy, the other rigid.

    Anyhow, 'eres a picture of the problem.

    While I can see why the bicep would deform, and that's fair enough, but the elbow is squished into non existance, which is bloody bad enough fer a trip to the ER and yer local welder.

    'Eres how the weights look:





    Maybe the verts at the UpperArm -> Forearm joint should be at 0.5 for both?

    Edit: Yup, that did the trick. Have to make the elbow a bit thinner, as it looks unnaturally large when bendin'. Bleh.
  • JedTheKrampus
    Options
    Offline / Send Message
    JedTheKrampus polycounter lvl 8
    You might want to invest in two more edge loops, one on each side of that elbow.
  • FiftyTifty
    Options
    Offline / Send Message
    FiftyTifty polycounter lvl 10
    You might want to invest in two more edge loops, one on each side of that elbow.

    I've seen that in a few models, actually; sandwiching the point bits like knees 'n' elbows. Does it just make the deformation smoother? Or are the added verts utilized for some weight-painting trickery?
  • JedTheKrampus
  • musashidan
    Options
    Offline / Send Message
    musashidan high dynamic range
    A good analogy: grab the cardboard core from a used  kitchen roll and bend it an half.....looks a lot like your skinned mesh arm, eh? 

    Yes, as mentioned already you need better topolgy to support the deformation. 
  • FiftyTifty
    Options
    Offline / Send Message
    FiftyTifty polycounter lvl 10
    Well I'll be, the middle o' the arm looks quite a bit better. Had to use a new skin modifier after changing the verts, otherwise the arm just wouldn't move alongside the biped. Have to mirror it an' redo the skinning again. Blah.



    And I just noticed that the shoulder warps when the bicep is rotated. Augh. Skinning is a right pain in the arse.
  • FiftyTifty
    Options
    Offline / Send Message
    FiftyTifty polycounter lvl 10
    It's only been close to a year, and I'm back to working on the model. Pleeeeurgh. Gave up since whenever I make a tiny change in the precedent Editable Poly modifier, the skinning is totally undone. Might just be a quirk with 3DS Max 2014, but that made me ragequit after the upteenth time of tweaking stuff.

    Anywho, spent the past day on skinning again, since the burn out is gone. Trouble is, all my quality skinning on the torso was nulled when I had to correct a problem with the vertices at the base of the neck (caused by accidentally rotating the neck bone). I swear, this stuff is nigh impossible. No wonder game artists charge an arm and a leg.

    Edit: Right, had a bizarre problem where only a couple verts on the torso would move along with the biped. Also occurred in 3DS Max 2015 when loading the scene file. The fix? Copy the skin modifier, convert the torso into an editable poly (same as it always has been), paste the skin modifier, et voila.
    My new problem is that the biped's waist bone isn't connected to the spine. Any idea how I would do that?
  • monster
    Options
    Offline / Send Message
    monster polycounter
    MajinCry said:
    My new problem is that the biped's waist bone isn't connected to the spine. Any idea how I would do that?
    In Biped's Structure Mode disable Triangle Pelvis and enable Triangle Neck. Those options exist for the old Physique modifier, but no one uses that anymore.

    I didn't read this entire thread, but if you are able to use a newer version of Max, the new Geodesic Voxel Binding feature will pretty much do the weighting for you.

    https://youtu.be/Kimn7CIZ5bI
  • FiftyTifty
    Options
    Offline / Send Message
    FiftyTifty polycounter lvl 10
    monster said:
    MajinCry said:
    My new problem is that the biped's waist bone isn't connected to the spine. Any idea how I would do that?
    In Biped's Structure Mode disable Triangle Pelvis and enable Triangle Neck. Those options exist for the old Physique modifier, but no one uses that anymore.

    I didn't read this entire thread, but if you are able to use a newer version of Max, the new Geodesic Voxel Binding feature will pretty much do the weighting for you.
    Didn't seem to work, the biped functions the same as before. Rotated the pelvis bone, and it still doesn't affect the upper half:



    Aye, I've a new version of Max, but 2014 is the latest Mudbox version I've got. Runnin' out of install space as well. I'll probably uninstall 2012 since it dies when displaying a bunch of bones (~300).

    Still, I'd like to get a grip with the Ol' Fashioned Way, before relying on an automatic tool for it. Or so I say right now, but I'll probably give up and switch after another three hours of agony.

Sign In or Register to comment.