[ SOLUTION ] [ MAYA ] Replace context sensitive marking menus

rares
polycounter lvl 9
Offline / Send Message
rares polycounter lvl 9
In this thread, i'll show you how to replace the context sensitive marking menus with marking menus of your own, that you can edit in the marking menu editor.

Scroll down for the short story if you don't want to read about the struggle
**** Long story ****

The age old problem of how to edit the context marking menus that maya comes with (the ones bound to shift and ctrl in your viewport and uv editor).
These pesky marking menus are the only ones you cannot edit via the marking menu editor,and for ages maya users have been vexed by this limitation (or maybe it's just been me).
These marking menus are stored in the following mel files (we'll tackle only the ones that are bound to shift, because these are the ones i wanted to change the most):

for the viewport :

- contextPolyToolsEdgeMM.mel
- contextPolyToolsFaceMM.mel
- contextPolyToolsVertexMM.mel
- contextPolyToolsObjectMM.mel
- contextPolyToolsUVMM which was never implemented

And the mel file that tells maya which marking menu to run depending on what's selected : contextPolyToolsMM.mel

for the uv editor:

texturePanel.mel

Maya does not display these marking menus in the marking menu editor for two reasons :

1. maya needs a file to have a "menu_" prefix and for it to be stored in the "markingMenus" folder to recognize it as a marking menu

2. the mel files need to follow a certain "formula" so that the marking menu editor can "understand" them.

At this point you might think that it's possible to just copy the first 4 files i've listed to your markingMenus folder, add the "menu_" prefix to them and then just edit them in the marking menu editor once you open Maya. However, if you do this , you will see them in your marking menu editor, but they will be empty, because they are not "written" like regular marking menus.

So now that we have defined the problem we can get to the solution :

**** Short story ****

1. Go to your maya instalation folder and search for these .mel files :

contextPolyToolsMM.mel

texturePanel.mel

2. Copy these files to your documents/maya/scripts folder

3. Open contextPolyToolsMM.mel

4 Search for this line of code "global proc int contextPolyToolsMM(string $parent)"

This is the procedure that decides what marking menu will pop up when you select a component or object and you press shift and right click

5. Replace the implicit marking menus with your own marking menus

Say i want to replace the marking menu that pops up when i have a vertex selected with a marking menu that i've created and named "raresVertexTools" .. i scroll down until i find this snippet :

[COLOR="Red"]//
		// Poly vertex is selected.
		//[/COLOR]
		string $vertexList[] = `filterExpand -expand false -selectionMask 31 $selection`;
		if (0 != `size($vertexList)`)
		{
			[COLOR="Red"][B]contextPolyToolsVertexMM;[/B][/COLOR]

			// allow user to define a proc with additional menu items
			if (`exists contextPolyToolsVertexUserMM`)
				contextPolyToolsVertexUserMM $parent;

			return 1;
		}

		//

And i replace contextPolyToolsVertexMM; with this "source menu_raresVertexTools"

so now the code will look like this :


[COLOR="Red"]//
		// Poly vertex is selected.
		//[/COLOR]
		string $vertexList[] = `filterExpand -expand false -selectionMask 31 $selection`;
		if (0 != `size($vertexList)`)
		{
			[COLOR="Green"][B]source menu_raresVertexTools;[/B][/COLOR]

			// allow user to define a proc with additional menu items
			if (`exists contextPolyToolsVertexUserMM`)
				contextPolyToolsVertexUserMM $parent;

			return 1;
		}

		//

If you wanted to replace the marking menu that pops up when a face is selected you would need scroll further down and find "contextPolyToolsFaceMM" and replace it with "source menu_whateverYourMarkingMenuIsCalled"


6. Open texturePanel.mel

7. Search for "global proc textureWindowCreatePopupContextMenu( string $parent ) "

this is the procedure that creates the context sensistive marking menus in the uv editor

For the marking menus that pop up in the uv editor things are slightly trickier because the marking menu are hardcoded into this procedure. So you have to delete the code that defines these marking menus and in it's place write "source menu_yourMenu"

so now, if you want to replace the marking menu that pops up when you have uvs selected you scroll down untill you find :
//////////////////////////////////////////////////////////////
		//
		//	UV
		//
		//////////////////////////////////////////////////////////////
if (`gmatch $firstSelected[0] "*.map*"`)
{


Now what follows is the "definition" of the marking menu that maya comes with (mine looks different because it's been written a while back by a friend at work, before we were aware of the trick i'm showing you...essetially this was the old school way of modifying context sensitive marking menus )


menuItem
				-enableCommandRepeat 1
				-radialPosition "S"
				-annotation "Horizontal rotate"  
				-label "ctRotate"  
				-command "ctOrizontalRotate();" 
				;

			menuItem
				-enableCommandRepeat 1
				-radialPosition "SW"
				-annotation "Rotate UVs to the left by 90 deg"  
				-label "Rotate 90 Left"
				-command "AMCRotateUVCmd(90)" 
				;
				
			menuItem
				-enableCommandRepeat 1
				-radialPosition "SE"
				-annotation "Rotate UVs to the right by 90 deg"  
				-label "Rotate 90 Right"
				-command "AMCRotateUVCmd(-90)" 
				;

			menuItem
				-enableCommandRepeat 1
				-radialPosition "E"
				-annotation "Flip UVs localy on V axis"  
				-label "Flip V"  
				-command "polyForceUV -flipVertical -local;" 
				;

			menuItem
				-enableCommandRepeat 1
				-radialPosition "N"
				-annotation (uiRes("m_texturePanel.kWindowContextUnfoldUVsAnnot"))  
				-label (uiRes("m_texturePanel.kWindowContextUnfoldUVs"))  
				-command "unfold -i 500 -ss 0.001 -gb 0 -gmb 0.4397 -pub 0 -ps  0 -us off" 
				;

			menuItem
				-enableCommandRepeat 1
				-radialPosition "N"
				-optionBox 1
				-annotation (uiRes("m_texturePanel.kWindowContextUnfoldUVsOptionsAnnot"))  
				-command "performUnfold 1" 
				;

			menuItem
				-enableCommandRepeat 1
				-radialPosition "W"
				-annotation "Flip UVs localy on U axis"  
				-label "Flip U"  
				-command "polyForceUV -flipHorizontal -local;" 
				;
/*
			menuItem
				-command  "vrnUVSnapper();"
				-annotation "Snap UV shells together" 
				 //-label "Shell Match Exact"
				-radialPosition "NE"
				;*/

			menuItem
				-command  "cpMatchShellCenterSimple();" 
				-annotation "Shell Match" 
				-label "Shell Match" 
				-radialPosition "NW"
				texSmudgeUVButton
				;

you delete the entire "definition" untill you reach this code :
// allow user to define a proc with additional menu items
			if (`exists contextUVToolsUVUserMM`)
				contextUVToolsUVUserMM $parent;

		}

the above snippet is harmless, since autodesk put it there in case someone wanted to write their own context sensitive marking menus(like the ones for the PolyTools), but who wants to deal with that shit when you have a perfectly good marking menu editor

8. Add your won marking menu that pops up when uvs are selected in the uv editor window : "source menu_myMenu;"

so now the code should look like this :
//////////////////////////////////////////////////////////////
		//
		//	UV
		//
		//////////////////////////////////////////////////////////////


		if (`gmatch $firstSelected[0] "*.map*"`)
		{

			[B][COLOR="SeaGreen"]source menu_myMenu;[/COLOR][/B]
								

			// allow user to define a proc with additional menu items
			if (`exists contextUVToolsUVUserMM`)
				contextUVToolsUVUserMM $parent;

		}

		

		

rinse and repeat for poly vertex and edge

IMPORTANT : After you save these files, and you play with your new context sensitive marking menus, you might decide that you want to add or remove elements from them. After you make these changes with the marking menu editor you must close maya and open it again for the changes to take effect


THE END

Replies

  • Eric Chadwick
  • rares
    Offline / Send Message
    rares polycounter lvl 9
    UPDATE : for maya 2016 you have to edit texturePanelMenus.mel instead of texturePanel.mel ,
  • ValN84
    Offline / Send Message
    ValN84 polycounter lvl 5
    That's a good catch man, especially if you want to edit the MM's in Maya's Marking Menu Editor.

    But if you would rather have portability from one version to another without having to re-edit contextPolyToolsMM.mel and texturePanel.mel for each Maya version (they usually change), I would recommend just creating the user version of each marking menu mel.
    This piece of code updates the vertex contextual marking menu with code found in the "contextPolyToolsVertexUserMM.mel" file. The guys at Autodesk just added this code recently in each of their contextual marking menu mels for improved customization. I stumbled onto this myself, can't remember seeing this in any documentation...
    // allow user to define a proc with additional menu items
    			if (`exists contextPolyToolsVertexUserMM`)
    				contextPolyToolsVertexUserMM $parent;
    
    			return 1;
    

    So you can create your own file with commands, and use the -e (edit) flag with the popupMenu command to change the default marking menus and replace/add your own stuff.
    Here's my example which replaces the "Delete" command with a submenu that has two different versions of the delete command, one deletes just "hanging" vertices, one deletes vertices and connected edges.
    The copy elements from the original contextPolyToolsVertexMM.mel, modify them and add them here.
    global proc contextPolyToolsVertexUserMM( string $parent )
    {
    	if (`popupMenu -query -exists $parent`){
    	popupMenu -edit $parent;
    			menuItem
    				-ia ""
    				-label (uiRes("m_contextPolyToolsVertexMM.kDeleteVertex"))
    				-subMenu 1
    				-tearOff 0
    				-radialPosition "SW" 
    				-allowOptionBoxes 1
    				-postMenuCommandOnce 0
    				;
    				menuItem
    					-label (uiRes("m_contextPolyToolsVertexMM.kDeleteVertex"))
    					-command "DeleteVertex" 
    					-radialPosition "W" 
    					-enableCommandRepeat 1
    					-image "polyDelVertex.png" 
    					;
    				menuItem
    					-label (uiRes("m_contextPolyToolsVertexUserMM.kDeleteVtxEdge"))
    					-command "doDelete" 
    					-radialPosition "SW" 
    					-enableCommandRepeat 1
    					;
    			setParent -m ..;
    	setParent -menu $parent;
    	}
    }
    

    Bonus info: localized labels.

    You can also copy Maya's labeling system and create this file to store your own labels for different languages.

    Here's how contextPolyToolsVertexUserMM.res.mel looks for the mods I did above:
    displayString -replace -value "Delete Vertex and Edges" m_contextPolyToolsVertexUserMM.kDeleteVtxEdge;
    

    You'll have to manually source the .res.mel files in userSetup.mel though...
  • Eric Chadwick
  • rares
    Offline / Send Message
    rares polycounter lvl 9
    Can you help me update the wiki page?
    well... the point of the thread was to create custom context sesnsitive MM's in Maya's Marking Menu Editor, so the only update to this method is "UPDATE : for maya 2016 you have to edit texturePanelMenus.mel instead of texturePanel.mel"


    Val's method is about editing the exisiting marking menus by writing code for the edits, and, while it ensures compatibility with future versions, it requires actual knowledge of mel and programming to pull off. It's a different aproach, and it should have it's own wiki entry
  • throttlekitty
    Offline / Send Message
    throttlekitty Polycount Sponsor
    Thanks for sharing!
  • Colnix
    Offline / Send Message
    Colnix polycounter lvl 4
    Dam This is awesome :D, is it possible to change the RMB menu also? the one with edge/verts/faces/object mode/... ?

    thanks for sharing
  • claydough
    Offline / Send Message
    claydough polycounter lvl 10
    dagObjectHit

    leveraging undocumented command dagObjectHit we can build our own viewport object/component under cursor aware marking menu.
    ( I use ctl-mmb to not step on any default Maya MarkingMenu or functionality )
    // EDIT: this is wrong old code! Because ctrl-mmb is used for interactively dragging normals with the move tool.
    I currently use:
    -shift
    -ctrl
    -button 1
    to not step on any default Maya MarkingMenu or functionality )
    code is edited to reflect as much.

    dagObjectHit notes: ( and a template for custom MM )
    http://tech-artists.org/forum/showthread.php?419-popUpMenu-mel-notes-quot-dagObjectHit-quot&highlight=dagobjecthit

    • object
    • object with same name hierarchy
    • and component
    • under cursor
    4473325944_0da7edf97a_o.jpg

    You can use the same template to append your own functionality ( menus ) to Maya's menu builds as well.
  • claydough
    Offline / Send Message
    claydough polycounter lvl 10
    dagObjectHit template:

    cly_objectHitMenuTemplate;
    added to yer userSetup.mel
    [php]cly_objectHitMenuTemplate;


    PHP Code:

    /*
    cly_objectHitMenuTemplate.mel v0.4
    Roger Klado was here
    because no one else will do it fer me
    Jan 17, 2011 ( God's basement vacation )

    [email protected] ( Klado/Claydough/cly_ )

    A template for setting up long named object aware view pane pop-up menus

    most folk r using the alt rmb fer scale navigation in Maya. Now that the Roger is
    alt rmb-n' wit zSwitcher...
    tis time to free the key and un-scamble my brain's viewport. ( new hot key is ctl-mmb ) ( alt-ctl-rmb/mmb/lmb is also an option/s )

    */

    global proc cly_objectHitMenuTemplate()
    {
    if( `popupMenu -exists cly_contextualTrash` ) {
    deleteUI cly_contextualTrash;
    }
    popupMenu
    //-alt 1
    -ctl 1
    -shift 1
    -button 1
    -mm 1
    -p viewPanes
    -pmc ( "string $bob[] = `ls -preSelectHilite`; "+ // ls return of pre-select hilite under cursor
    "if (`popupMenu -e -exists cly_contextualTrash`) { "+
    "popupMenu -e -deleteAllItems cly_contextualTrash; "+
    "setParent -menu cly_contextualTrash; "+
    "if ( `dagObjectHit`){"+ // dag object Hit returns int if cursor is over an object
    "string $cursorObjectSet[] = {}; "+
    "$cursorObjectSet = `cly_dagMenuRape`; "+
    "menuItem -label $cursorObjectSet[0] ; "+
    "menuItem -label $cursorObjectSet[1] ; "+

    // pre selection hilite menu begin
    "if ( size( $bob ) ) "+
    " for ( $b in $bob ) "+
    "menuItem "+
    "-label $b; "+
    // pre selection hilite menu end

    // Return needs the echo outside of the proc. Command echo is cleaned up here
    "if ( `window -exists clyInvisibleEcho` ) "+
    "deleteUI clyInvisibleEcho; "+
    "} "+
    "else "+
    "menuItem -label \"no object under cursor dude\"; "+
    "}" )
    cly_contextualTrash;
    }
    //________________________________________________________________________________________
    // end




    global proc string[] cly_dagMenuRape()
    {
    //begin

    // assure history state is echoable ( therefore recordable ) during the popupMenu
    int $echoState = 1;
    if ( `window -exists clyInvisibleEcho` )
    deleteUI clyInvisibleEcho;
    window -vis 0 clyInvisibleEcho;
    columnLayout;
    cmdScrollFieldReporter cly_echo;
    if ( !`cmdScrollFieldReporter -q -echoAllCommands "cly_echo"` ) {
    $echoState = 0;
    cmdScrollFieldReporter -e -echoAllCommands 1 "cly_echo";
    }

    // temp file
    scriptEditorInfo -historyFilename "cly_tempHistoryLog.txt" -writeHistory true;
    //record history to temp file
    dagObjectHit -mn "cly_contextualTrash"; // result of
    scriptEditorInfo -writeHistory false;
    string $historyRecordAddy = `scriptEditorInfo -query -historyFilename`; // directory

    // "r"ead history record
    $fileId =`fopen $historyRecordAddy "r"`; // dagObjectHit return as int Id

    string $dagHitGitPath[] = {};
    string $dagHitArguments = `fgetword $fileId "\""`;
    while ( size( $dagHitArguments ) > 0 ) {
    $dagHitArguments = `fgetword $fileId "\"" `;
    // argument assigned address [2] contains short mame
    $dagHitGitPath[size($dagHitGitPath)] = $dagHitArguments;
    }
    fclose $fileId;
    /*
    $fileId =`fopen $historyRecordAddy "r"`;
    string $historyRecorded=`fread $fileId $historyRecorded`; // dagObjectHit return as string
    fclose $fileId;
    */

    if ( $echoState == 0 ) { // then restore to the original state
    cmdScrollFieldReporter -e -echoAllCommands 0 "cly_echo";
    }

    //clean
    sysFile -delete $historyRecordAddy;
    // Once again, we have had our way with dagMenuProc. ( she's history ) call her a cab...
    popupMenu -e -deleteAllItems cly_contextualTrash;

    string $cly_objPathReturns[] = {};
    string $tokenizedLongPath[]; // find short names
    tokenize $dagHitGitPath[2] "|" $tokenizedLongPath; // "
    $cly_objPathReturns[0]= $tokenizedLongPath[size($tokenizedLongPath) - 1];
    $cly_objPathReturns[1]= $dagHitGitPath[2];

    return $cly_objPathReturns;
    }
    // end: insert applause here... [/php]
  • Colnix
    Offline / Send Message
    Colnix polycounter lvl 4
    Hello claydough thank you for replying, well i can really skip the context MM from RMB. what i wanted was simply to use the RMB to pop up a MM or even using the context MM but editing it, either way is k.

    Kyle Hug have done that in is Kyle's editable context MM but messes with a lot of other stuff that i dont want to.
  • claydough
    Offline / Send Message
    claydough polycounter lvl 10
    I believe after using dagMenuProc to return the context of the menu ( under cursor ):

    popupMenu -e -deleteAllItems

    is all that is used to keep a custom MM unique.
    In which case u could omit as much and you would get a normal rmb menu.
    ( or append yer own menu items to the rmb menu items. )

    if u create yer own viewport parented RMB popup menu in yer usersetup it will take precendence over maya default rmb MM.

    However to have contextually aware commands u need a way to return component/object under cursor to your custom menus and commands. for which I suggest dagObjectHit. ( otherwise how is our menu functionality contextual? )
Sign In or Register to comment.