multi threading with python in 3ds Max

hannes d

Hi

I m researching multi threading in max
Got the following python script
but multithreading is slower than creating the objects the default way.
I know there is a overhead but atm I'm trying to understand the benefit in max off multithreading.
It still seems to create them 1 by 1 instead of multiple at the same time.

EDIT: added link to code since polycount screws the formatting https://pastebin.com/qmRrhA81

<br><div>import pymxs

</div><div>import threading

</div><div>import time

</div><div>
<br></div><div>
flag = True</div><div>
counter = 0	</div><div>
<br></div><div>
def callMXSEntry():</div><div>
&nbsp; &nbsp; with pymxs.mxstoken():</div><div>
		#~ pymxs.runtime.Teapot()</div><div>
		createTeaPots(0)</div><div>
def callMXSEntry1():</div><div>
&nbsp; &nbsp; with pymxs.mxstoken():</div><div>
		#~ pymxs.runtime.Teapot()</div><div>
		createTeaPots(1)</div><div>
def callMXSEntry2():</div><div>
&nbsp; &nbsp; with pymxs.mxstoken():</div><div>
		#~ pymxs.runtime.Teapot()</div><div>
		createTeaPots(2)</div><div>
<br></div><div>
# Ex1 and Ex2 used to mark cooperator of mxstoken</div><div>
# when pymxs.mxstoken is gained</div><div>
#	python codes are concurrence, but with pymxs.mxstoken blocks are not</div><div>
#</div><div>
def callMXSEntryEx1(locker, tick, evt):</div><div>
	global flag, counter</div><div>
<br></div><div>
	try:</div><div>
		locker.acquire()</div><div>
		flag = False</div><div>
		with pymxs.mxstoken():</div><div>
			# pymxs.runtime.Teapot(Name="callMXSEntryEx1")</div><div>
			createTeaPots(1)</div><div>
			# give up lock, let Ex2 could exec codes</div><div>
			locker.release()</div><div>
			if not evt.wait(tick):</div><div>
				pymxs.print_("Error: event untiggered\nwhich indicates 'with block' in Ex2 haven't finished\n",
 True, True)</div><div>
			counter = 30</div><div>
	except:</div><div>
		pymxs.print_("Error: unexpected exception\n", True, True)</div><div>
		raise</div><div>
	finally:</div><div>
		if locker.locked():</div><div>
			locker.release()</div><div>
<br></div><div>
def callMXSEntryEx2(locker, tick, evt):</div><div>
	global flag, counter</div><div>
	while flag:</div><div>
		time.sleep(tick)</div><div>
<br></div><div>
	try:</div><div>
		locker.acquire()</div><div>
		# we expected this block is finished</div><div>
		# before Ex1 wakeup from sleep</div><div>
		for i in xrange(10):</div><div>
			# only a indicator, could just assign counter = 10</div><div>
			counter = counter + 1</div><div>
		evt.set()</div><div>
		with pymxs.mxstoken():</div><div>
			# this block won't be executed after Ex1 with block finished</div><div>
			#~ pymxs.runtime.Teapot(Name="callMXSEntryEx2")</div><div>
			createTeaPots(2)</div><div>
			if counter != 30:</div><div>
				pymxs.print_("Error: expected counter 30, got %d\nwhich indicates 'with block' in Ex2 haven't finished\n" % counter, True, True)</div><div>
	except:</div><div>
		raise</div><div>
	finally:</div><div>
		if locker.locked():</div><div>
			locker.release()</div><div>
	pymxs.print_("succss", False, True)</div><div>
<br></div><div>
def main():</div><div>
	#~ locker = threading.Lock()</div><div>
	#~ evt = threading.Event()</div><div>
	t1 = threading.Thread(target=callMXSEntry)</div><div>
	t2 = threading.Thread(target=callMXSEntry1)</div><div>
	t3 = threading.Thread(target=callMXSEntry2)</div><div>
	#t2 = threading.Thread(target=callMXSEntryEx1) #, args=(locker, 1, evt))</div><div>
	#t3 = threading.Thread(target=callMXSEntryEx2) #, args=(locker, 0.01, evt))</div><div>
	t1.start()</div><div>
	t2.start()</div><div>
	t3.start()</div><div>
<br></div><div>
if __name__ == "__main__":</div><div>
	# slow multithread method</div><div>
	main()</div><div>
	</div><div>	# fast create instant method</div><div>
	#~ createTeaPots(0)</div><div>
	#~ createTeaPots(1)</div><div>
	#~ createTeaPots(2)</div><div>
<br></div><div>
def createTeaPots(amount=0):</div><div>
	rt = pymxs.runtime</div><div>
	if amount == 0:</div><div>
		for x in range(20):</div><div>
			rt.Teapot()</div><div>
	if amount == 1:</div><div>
		for x in range(20):</div><div>
			rt.sphere()</div><div>
	if amount == 2:</div><div>
		for x in range(20):</div><div>
			rt.Tube()





</div>

hannes d

found this, saying it doesnt allow it on scene operations
https://forums.autodesk.com/t5/3ds-max-programming/python-multithread/m-p/8375289
If someone has any more info feel free to post

hannes d

some info i found useful, regarding multi threading and processes
a good talk about all of this in python https://www.youtube.com/watch?v=MCs5OvhV9S4
https://stackoverflow.com/questions/2629680/deciding-among-subprocess-multiprocessing-and-thread-in-python
https://stackoverflow.com/questions/3044580/multiprocessing-vs-threading-python

so for now i decided to try multi-processes, but this does not work directly from max
it does however work if you run it outside of max and launch several 3dsmaxbatch.exe's
i ll update with any more useful info i find since there doesn't seem to be much info out there regarding max/python

RN

If you want to create multiple copies of the same mesh, here's a suggestion:

- Read the mesh geometry into a Python object -- a Numpy array for example, with all vertex positions as 1x3 rows, as well as a a tuple of tuples as an index buffer, representing the mesh polygons formed by the vertex indices so you can rebuild the mesh later.

- As a Python object, copy the geometry array as much as you want and transform it around using Numpy's matrix multiplication (transformArray @ pointsArray or numpy.dot(transformArray, pointsArray)).

- On the main thread, put all the geometry back into a mesh, then detach all individual elements (all pieces of "connected" geometry, same as you do with Edit Poly, each element is a mesh copy).

So each copy becomes its own object. There are scripts that do that, you can read the code and see the commands needed.

If you can't call 3ds commands from threads, at least you can manipulate your own data on them (Numpy arrays etc.).

hannes d

@RN that's a interesting approach. def will remember that.
I had considered doing this for 1 dense mesh with laods of verts, but didn't thought you could use it to create new meshes

atm I managed to get "multithreading" to work between several max instances. so every max instance on its own isn't multithreaded. But they are all running in a pool and if 1 finishes another one starts.
so instead of linear batching 100 scenes and max only using up to 15% of my cpu
I now batch several max scenes simultaneously and cpu use is 90%

since max doesnt support multiprocessing I have to "hack" around this which creates some overhead. so this is not good for small operations.
1) from the pyconsole create a multiprocess pool, and manage sending scenes and scripts to each process in the pool, and keep the pool full
2) every subprocess in the pool launches a 3dsmaxbatch.exe subprocess
3) we send our python script to 3dsmatchbatch.exe

overhead atm is about 10 sec if you do nothing in your script, but batch processing multiple scenes is 50-70% faster

monster

How do you prevent 3dsmaxbatch.exe from closing when it's done, so it can continue running other operations?
When I was using it batch mode it continually exited, and constantly restarting the process was slower than the operations being handled.

Depending on what your doing, using parallel nodes in MCG might be faster.