Maya Help with toggle script

throttlekitty

I'm having trouble with a relatively simple script here, can I get some help?
It toggles a selection constraint for all or hard edges only.
the -q flag isn't documented for polySelectConstraint, but it works, and -sm returns an integer. But, Maya returns an error of:
Error: Cannot convert data of type int[] to type int.

global proc tkToggleHardEdgeSelect()
    {
        int $CurMode = `polySelectConstraint -q -sm`;
        if ($CurMode == "0")
            {
            polySelectConstraint -t 0x8000 -sm 1;
            }
        else if ($CurMode == "1")
            {
            polySelectConstraint -t 0x8000 -sm 0;
            }
    }
    tkToggleHardEdgeSelect();

PolyHertz

well the reason its giving you that error is because your declaring $CurMod as an int instead of an int array, when the return is an array. Thats what it looks like to me at first glance at least.
So just do something like this:


global proc tkToggleHardEdgeSelect()
{
int $CurMode[] = `polySelectConstraint -q -sm`;
if ($CurMode[0] == 0)
{
polySelectConstraint -t 0x8000 -sm 1;
}
else if ($CurMode[0] == 1)
{
polySelectConstraint -t 0x8000 -sm 0;
}
}

tkToggleHardEdgeSelect;

throttlekitty

That gives:
Error: "$CurMode" is not an array.

I don't think that's the case, "-q -sm" returns a single value of the current constraint of ede smoothness: 0,1 or 2.

PolyHertz

Oops, sorry about that, try the (edited) above code again.

throttlekitty

I caught that a bit late myself With that change, I don't get any feedback from the script, and nothing happens.

Edit: actually, the command isn't working on it's own now. What could I have accidentally hit or done to change that now?

PolyHertz

Are you just trying to make it so it'll only let you select hard edges? If so you need to switch it to the appropriate mode as well. Mode 3 is probably what you want which will make it so all selections from that point forth will be only on hard edges until yo toggle it off. ex:

On:
polySelectConstraint -t 0x8000 -sm 1 -m 3;

Off:
polySelectConstraint -t 0x8000 -sm 0 -m 0;

But borders will also count as 'hard edges', and when you toggle it on all hard edges will automatically be selected. If you'd explain a bit more about what your trying to achieve would probably be able to help more.

throttlekitty

Oh, thank a lot for that! I could've sworn it was originally working without the mode flag.
And yeah, border edges are fine, this is just to speed up setting UV seams on hard edges. Selecting edges quickly can be difficult, maybe it's just my ATI card, looking to speed that process up at least.