Usually zombie process is not a big problem, but sometimes... just look at the screenshot below. It wants the whole machine as fried eggs! or maybe boiled?
Kind of my extended memory with thoughts mostly on Linux and related technologies. You might also find some other stuff, a bit of SF, astronomy as well as old (quantum) chemistry posts.
Search This Blog
Friday, March 28, 2014
Tuesday, March 25, 2014
Multiprocessing (in Python)
I needed to do some multi-threading in Python. As I needed effect quick I decided to use standard threading module. However, every time I had to use it I felt it was rather complicated beast. At threading module documentation page there is a link to multiprocessing module. I was tired with threading, on the one hand, but didn't have enough time to learn about greenlet or another competing project, on the other, so I decided to take a quick glance at multiprocessing module...
... And my life became much easier, sky bluer, grass, greener, oh and scripts faster ;-).
I don't do anything special with it, so just one simple code example, but this is very good tutorial you can find much more: http://pymotw.com/2/multiprocessing/communication.html.
Main part of nearly all my scripts looks the same:
import multiprocessing
SIZE = 30 pool = multiprocessing.Pool(processes=SIZE) pool_output = pool.map(get_values, servers) pool.close() # no more tasks pool.join()
Where servers is a list with servers I need get information from, and get_values is a function (sometimes with a different name). Simple, isn't it?
Subscribe to:
Posts (Atom)