Jug
A Task-Based Parallelization Framework
Jug allows you to write code that is broken up into tasks and run different tasks on different processors.
It currently has two backends. The first uses the filesystem to communicate between processes and works correctly over NFS, so you can coordinate processes on different machines. The second is based on redis so the processes only need the capability to connect to a common redis server.
Jug also takes care of saving all the intermediate results to the backend in a way that allows them to be retrieved later.
Examples
How I Use Jug & IPython Notebooks – Meta Rabbit