multitask

Multitask is an excellent software that enables Python programs to use generators (also known as coroutines) for achieving cooperative multitasking and asynchronous I/O. When using multitask, applications are made up of a set of cooperating tasks that stop to allow a shared task manager to take over whenever a potentially blocking operation – like I/O on a socket, or getting data from a queue – is carried out. The task manager temporarily suspends the task, and during that period, other tasks can run before it restarts the blocked task when the operation is complete. This approach is perfect for applications requiring concurrency that would otherwise use select() and/or multiple threads.

Multitask is a free software distributed under the MIT license. As a simple example, imagine using Multitask to enable two unrelated tasks to run concurrently by adding a printer function:

def printer(message):
while True:
print message
yield

multitask.add(printer('hello'))
multitask.add(printer('goodbye'))
multitask.run()

This program prints out “hello” and “goodbye” in turn, pretty much as you would expect it to. For a more useful example, imagine that you want to implement a multitasking server capable of handling multiple concurrent client connections. You can easily achieve this by adding some child_task functions:

def listener(sock):
while True:
conn, address = (yield multitask.accept(sock))
multitask.add(client_handler(conn))

def client_handler(sock):
while True:
request = (yield multitask.recv(sock, 1024))
if not request:
break
response = handle_request(request)
yield multitask.send(sock, response)

multitask.add(listener(sock))
multitask.run()

With the functions and classes within the multitask module, tasks can stop for I/O operations on sockets and file descriptors, add/remove data to/from queues, or sleep for a set period. Tasks can also specify a timeout when yielding. If the specified operation is yet to complete after a given number of seconds, the task is restarted, and a Timeout exception is raised at the yielding point.

What makes Multitask really special is that tasks can yield other tasks, enabling composition of tasks and reusing multitasking code already in existence. In this case, a child task runs until it either completes or raises an exception, and its output or exception propagated to its parent. To illustrate:

def parent():
try:
print 'good child says: %s' % (yield child())
print 'bad child says: %s' % (yield child(bad=True)))
except Exception, e:
print 'caught exception: %s' % e

def child(bad=False):
if bad:
raise RuntimeError('oops!')
yield 'Hi, Mom!'

multitask.add(parent())
multitask.run()

In summary, Multitask requires Python 2.5 or later, and in its latest release, it has added functionality that enables child tasks to return values to their parent by raising StopIteration exceptions, introduced the get_default_task_manager() function for accessing the default TaskManager instance used by add() and run(), and added readline(), which is extremely useful for performing non-blocking reads from the stdout of a child process.