Alexandre Bourget

I was working on a feature to implement Gmail's "Undo" feature in Pylons (which I'll present on this blog soon), and was faced with a challenge regarding the passage of variables (or functions) to a separate thread. The reason is that Pylons resolves the current request object according to the local thread. You won't find the same request object in a different thread from which it originated.

I need to be able to resolve those objects (to get the actual object, not the proxy object) before spawning a new thread. Though, I don't want to require the user to know about all this. He should only create a function that should work in the current thread context, and my code will make sure the good objects get passed on to the new thread.

In Python terms, I wanted to be able to modify the func_closure attribute of a function object (it's called __closure__ from Python 2.6 onwards). I searched around, only to find that it's really a read-only attribute, unless you want to do risky things.

So, if you do something like:

def a():
    b = 123
    def myclosure():
        print b
    return myclosure
myfunc = a()

the value of b gets referenced (as a cell object) in the myfunc.func_closure tuple. But there's no way to change it. What I need is to tweak the myfunc function's closure, and replace the internal value of b for something else. So when I'd call myfunc(), it actually prints something else than 123, let's say 'mynewvalue'.

At first, I tried the mix-and-match suggested here, which looks like:

import new
def getitem(a):
    def retclosure():
        return a
    return retclosure
tmpfunc = getitem('mynewvalue')
cell = tmpfunc.func_closure[0]
rewrittenfunc = new.function(myfunc.func_code , myfunc.func_globals, myfunc.func_name, myfunc.func_defaults, (cell,))

Basically, it takes the code, globals, name and defaults from the myfunc function, and uses the cells from another function, mixing two different functions into one.

That's great for a single reference in the closure, but it's not very flexible. I want my code to go through all of the closed in objects, and verify is they are of a certain type (namely StackedObjectProxy), and resolve those that need local resolution. It should work with a random number of arguments.

Here is my solution:

def unproxy_closure(f):
    """Rewrite function with resolved closure references (StackedObjectProxy from current thread)"""
    import new
    def create_cell(obj):
        return (lambda: obj).func_closure[0]
    return new.function(func.func_code , func.func_globals, func.func_name,
                        func.func_defaults,
                        tuple(create_cell(cell.cell_contents._current_obj())
                                  if hasattr(cell.cell_contents, '_current_obj')
                                       and
                                     callable(cell.cell_contents._current_obj)
                                  else cell
                              for cell in func.func_closure))

First, there's that create_cell function, which will return a cell object. It seems to be the only way to create those types of objects: creating a closure and stealing it's cell object.

Using the new module, it creates a new function borrowing code (globals, defaults, name, etc.) from the first function, and it goes through the cells of the passed-in closure to modify them if necessary. That way, we keep the actual object in the closure, not the proxy object that won't resolve correctly in the other thread I'm going to launch.

Voilà! That opens the road to a very cool feature I'll talk about shortly. Comments are welcome.

UPDATE 18h22: rephrased a little bit clunky sentences

UPDATE Match 18th 2010: Added link to what I've talk about shortly after

UPDATE March 20th 2010: fixed code block, thanks to John