python 2.7 - Why do my subclassed multiprocessing Processes hang? -

i have written python program employs multiprocessing via subclassed process objects. trying improve error handling, causing me no end of frustration reasons cannot comprehend.

bear me while lay out sample code:

my subclassed process worker:

# worker.py  multiprocessing import process, queue import sys import traceback # other imports necessary  class worker(process):     def __init__(self, inputqueue, outputqueue):         try:             super(worker, self).__init__()              self.inputqueue = inputqueue             self.outputqueue = outputqueue              #etc              1/0 # dumb error causes process crash              #etc           except exception e:              exc_type, exc_value, exc_traceback = sys.exc_info()              e = traceback.format_exception(exc_type, exc_value, exc_traceback, limit = 2)               # forward error message engine class              self.outputqueue.put({ 'type' : 'error', 'data' : e })      def run(self):         try:             in iter(self.inputqueue.get, 'stop'):                 # stuff                 self.outputqueue.put({ 'type' : 'result', 'data' : stuff })          except exception e:             exc_type, exc_value, exc_traceback = sys.exc_info()             e = traceback.format_exception(exc_type, exc_value, exc_traceback, limit = 2)              # forward error message engine class             self.outputqueue.put({ 'type' : 'error', 'data' : e }) 

this engine class parent of worker objects:

# engine.py # imports necessary class engine:     def __init__(self) # other arguments necessary         # initialise class         self.processors = []     def run(self):         try:             # fill inputqueue number of jobs             # , initialise output queue              # etc              # start worker processes             in range(numberofprocesses):                 p = worker.worker(inputqueue, outputqueue)                 self.processors.append(p)                 p.start()                 inputqueue.put('stop')              # process outputqueue             numberofjobs = 6 # (say)              while numberofjobs:                 result = outputqueue.get()                  if result['type'] == 'error':                     print result                     raise exception                  elif result['type'] == 'result':                     # process result appropriately                     numberofjobs -= 1              p in self.processors:                 p.join()          except exception e:             raise 

configuration file runs whole thing:

# configuration.py if __name__ == "__main__":     # initialise other parameters necessary      try:         # initialise instance of engine.engine         eng = engine.engine(arguments necessary)          eng.run()      except exception e:         print e         print 'finished unsuccessfully.'      else:         print 'finished successfully.' 

for brevity, left few imports , other things out of example code. error handling machinery in worker.py derived earlier question here.

when run program, worker processes execute , when 1 hits 1/0 error in worker.__init__ (or in worker.run(), if there 1 in there), grabs traceback , puts output queue fine. engine.run() prints error message it's supposed to. problem is ends. engine should raise new, generic, error , (i think) pass code in configuration.py, whereupon program should exit via except statement in file (printing message says 'finished unsuccessfully').

instead, actually happens engine.run() prints error message derived worker class, program hangs , cpu usage drops 0%. nothing else happens. recognise problem other worker processes not quitting (or whatever ought do).

what mystifies me if don't have except exception e block in engine class, , if don't have try-except-else blocks in configuration.py file, engine.run() prints traceback worker object , crashes (doesn't hang) generic raise exception statement.

essentially i'd engine.run() pass generic raise exception configuration file program exits more gracefully.

someone me!