I needed to do some multi-threading in Python. As I needed effect quick I decided to use standard threading module. However, every time I had to use it I felt it was rather complicated beast. At threading module documentation page there is a link to multiprocessing module. I was tired with threading, on the one hand, but didn't have enough time to learn about greenlet or another competing project, on the other, so I decided to take a quick glance at multiprocessing module...
... And my life became much easier, sky bluer, grass, greener, oh and scripts faster ;-).
I don't do anything special with it, so just one simple code example, but this is very good tutorial you can find much more: http://pymotw.com/2/multiprocessing/communication.html.
Main part of nearly all my scripts looks the same:
import multiprocessing
SIZE = 30 pool = multiprocessing.Pool(processes=SIZE) pool_output = pool.map(get_values, servers) pool.close() # no more tasks pool.join()
Where servers is a list with servers I need get information from, and get_values is a function (sometimes with a different name). Simple, isn't it?
No comments:
Post a Comment