Usually zombie process is not a big problem, but sometimes... just look at the screenshot below. It wants the whole machine as fried eggs! or maybe boiled?
Tuesday, March 25, 2014
I needed to do some multi-threading in Python. As I needed effect quick I decided to use standard threading module. However, every time I had to use it I felt it was rather complicated beast. At threading module documentation page there is a link to multiprocessing module. I was tired with threading, on the one hand, but didn't have enough time to learn about greenlet or another competing project, on the other, so I decided to take a quick glance at multiprocessing module...
... And my life became much easier, sky bluer, grass, greener, oh and scripts faster ;-).
I don't do anything special with it, so just one simple code example, but this is very good tutorial you can find much more: http://pymotw.com/2/multiprocessing/communication.html.
Main part of nearly all my scripts looks the same:
SIZE = 30 pool = multiprocessing.Pool(processes=SIZE) pool_output = pool.map(get_values, servers) pool.close() # no more tasks pool.join()
Where servers is a list with servers I need get information from, and get_values is a function (sometimes with a different name). Simple, isn't it?