A pool of threads that perform some list of tasks. More...

Detailed Description

A pool of threads that perform some list of tasks.

There is a function add_work() that adds a task to be performed.

Example: print the numbers from 1 to 10 in no particular order, in three threads:

 def printit(num):
   print str(num)
 with WorkPool(3) as w:
   print "three threads are waiting for work"
   for x in xrange(10):
     w.add_work(printit,[x+1])
   print "all threads have work, but the work may not be complete"
   w.barrier()
   print "all work is now complete."
 print "once you get here, all workpool threads exited"

Definition at line 84 of file workpool.py.

Inheritance diagram for produtil.workpool.WorkPool:

Public Member Functions
def	__init__
	Create a WorkPool with the specified number of worker threads. More...

def	__enter__ (self)
	Does nothing. More...

def	__exit__ (self, etype, value, traceback)
	Called at the bottom of a "with" block. More...

def	nthreads (self)
	The number of worker threads. More...

def	add_work
	Adds a piece of work to be done. More...

def	start_threads (self, n)
	Starts n new threads. More...

def	kill_threads (self)
	Kills all worker threads. More...

def	barrier (self)
	Waits for all threads to reach the barrier function. More...

Public Attributes
	logger
	a logging.Logger for log messages

	die
	If True, all threads should exit immediately. More...

Constructor & Destructor Documentation

def produtil.workpool.WorkPool.__init__	(	self,
		nthreads,
		logger = `None`
	)

Create a WorkPool with the specified number of worker threads.

The nthreads must be at least 1.

Definition at line 103 of file workpool.py.

Member Function Documentation

def produtil.workpool.WorkPool.__enter__ ( self )

Does nothing.

Called from atop a "with" block.

Definition at line 124 of file workpool.py.

def produtil.workpool.WorkPool.__exit__	(	self,
		etype,
		value,
		traceback
	)

Called at the bottom of a "with" block.

If no exception was raised, and no "break" encountered, then waits for work to complete, and then kills threads. Upon a fatal signal or break, kills threads as quickly as possible.

Parameters

etype,value,traceback exception information

Definition at line 127 of file workpool.py.

def produtil.workpool.WorkPool.add_work	(	self,
		work,
		args = `None`
	)

Adds a piece of work to be done.

It must be a callable object. If there are no worker threads, the work() is called immediately. The args are passed, if present.

Parameters

work	a callable object
args	a list of arguments to the work function

Definition at line 191 of file workpool.py.

Referenced by produtil.workpool.WorkPool.barrier().

def produtil.workpool.WorkPool.barrier ( self )

Waits for all threads to reach the barrier function.

This can only be called by the master thread.

Upon calling, the master thread adds a WorkTask for each thread, telling the thread to call self.barrier(). Once all threads have reached that point, the barrier returns in all threads.

Definition at line 329 of file workpool.py.

Referenced by produtil.workpool.WorkPool.__exit__(), and produtil.workpool.WorkPool.barrier().

def produtil.workpool.WorkPool.kill_threads ( self )

Kills all worker threads.

Can only be called from the thread that made this object.

Definition at line 300 of file workpool.py.

Referenced by produtil.workpool.WorkPool.__exit__().

def produtil.workpool.WorkPool.nthreads ( self )

The number of worker threads.

Definition at line 187 of file workpool.py.

Referenced by produtil.workpool.WorkPool.add_work(), and produtil.workpool.WorkPool.barrier().