Alexandre Bourget

geek joy

Entries tagged “closure”

Gmail's undo/cancel feature: introducing Python's WebUndo

written by abourget, on Feb 20, 2010 11:14:00 PM.

From a web point of view, if you want to stop a user from doing stupid things (like submitting a form before reviewing it, sending a heinous e-mail late at night, or clicking a button that launches a nuclear missile), you might want to do one one of three things:

  • Pop-up a confirmation dialog
  • Go to a second page, with yet another submit button asking for confirmation, with a summary of the submitted data
  • Just let it go, but offer a timed “UNDO” or “CANCEL” link, like Gmail implemented for e-mails

Of course, the best of all three is the latter. The difference here is major. It’s the difference between pro-active annoyingness (pop-ups asking confirmation when you know very well you want to send the e-mail), or reactive flexibility. It also gives your user a great boost of confidence in your application, because the user can do stupid things and knows he’ll be able to fix them in a timely manner.

Now here’s how to do it in Pylons, using the just-released WebUndo lib. You can find it on PyPI, so just pip install WebUndo from your project.

We’re lucky Pylons runs as a single application with threads, since that’s what we’ll use to delay our operations and give the user a chance to cancel. PHP apps for instance, wouldn’t give us that flexibility, since everything vanishes once the request is sent. Take a look at the source code to know how it’s built.

Let’s consider this piece of Pylons controller:

class ThController(BaseController):
    def index(self):
        return render('th.mako')

    def save(self):
        open('/tmp/dump.txt', 'w').write(request.params.get('stuff'))
        return "Everything saved"

where th.mako contains a simple form with a text field named stuff. In this case, once you submit the form in th.mako, it’s done: the file is written to disk, and there’s no way you can cancel it.

Now let’s have a look at this version:

class ThController(BaseController):
    def index(self):
        return render('th.mako')

    def save(self):
        req = request
        def do_save():
            open('/tmp/dump.txt', 'w').write(req.params.get('stuff'))
        key = webundo.launch_cancelable_job(do_save, 5)
        return 'Saving, click to <a href="%s">cancel</a>' % h.url_for(controller='th', action='cancel', id=key)

    def cancel(self, id):
        if webundo.cancel_job(id):
            return "Thread cancelled successfully"
        else:
            return "Too late, thread finished already."

We added some pieces of webundo, the save() action returns a confirmation message, and a link to cancel the operation. We also added a def do_save() wrapper, a call to the launch_cancelable_job function and the passing on of the key. Notice also the req = request, used to make sure the request object gets closed in the do_save() function. The fun part here, is that the operation will not be carried out until the 5 seconds (second parameter to launch_cancelable_job) are elapsed. The last thing is obviously, the cancel action, which will deal with the cancelation.

Using this feature, even if you close your browser window after calling the save action, the do_save() function will be executed normally, only delayed by 5 seconds.

For the Pylons addicts who read that code and say “hey, that request object can’t traverse threads, it’s a StackedObjectProxy”! See the details and answer in my other blog post.

Of course, there are cases where you want a certain operation to be carried out directly (when calling save() for example) but you want some special code to be executed if someone click an undo button for example. The WebUndo lib provides you with another pair of functions to do just that: launch_undoable_job and undo_job.

Take a look at the documentation for full details. Happy undoing!

UPDATED March 20th: typos following feedback

Writing to Python closures, Pylons threads

written by abourget, on Feb 19, 2010 10:07:00 PM.

I was working on a feature to implement Gmail’s “Undo” feature in Pylons (which I’ll present on this blog soon), and was faced with a challenge regarding the passage of variables (or functions) to a separate thread. The reason is that Pylons resolves the current request object according to the local thread. You won’t find the same request object in a different thread from which it originated.

I need to be able to resolve those objects (to get the actual object, not the proxy object) before spawning a new thread. Though, I don’t want to require the user to know about all this. He should only create a function that should work in the current thread context, and my code will make sure the good objects get passed on to the new thread.

In Python terms, I wanted to be able to modify the func_closure attribute of a function object (it’s called __closure__ from Python 2.6 onwards). I searched around, only to find that it’s really a read-only attribute, unless you want to do risky things.

So, if you do something like:

def a():
    b = 123
    def myclosure():
        print b
    return myclosure
myfunc = a()

the value of b gets referenced (as a cell object) in the myfunc.func_closure tuple. But there’s no way to change it. What I need is to tweak the myfunc function’s closure, and replace the internal value of b for something else. So when I’d call myfunc(), it actually prints something else than 123, let’s say 'mynewvalue'.

At first, I tried the mix-and-match suggested here, which looks like:

import new
def getitem(a):
    def retclosure():
        return a
    return retclosure
tmpfunc = getitem('mynewvalue')
cell = tmpfunc.func_closure[0]
rewrittenfunc = new.function(myfunc.func_code , myfunc.func_globals, myfunc.func_name, myfunc.func_defaults, (cell,))

Basically, it takes the code, globals, name and defaults from the myfunc function, and uses the cells from another function, mixing two different functions into one.

That’s great for a single reference in the closure, but it’s not very flexible. I want my code to go through all of the closed in objects, and verify is they are of a certain type (namely StackedObjectProxy), and resolve those that need local resolution. It should work with a random number of arguments.

Here is my solution:

def unproxy_closure(f):
    """Rewrite function with resolved closure references (StackedObjectProxy from current thread)"""
    import new
    def create_cell(obj):
        return (lambda: obj).func_closure[0]
    return new.function(func.func_code , func.func_globals, func.func_name,
                        func.func_defaults,
                        tuple(create_cell(cell.cell_contents._current_obj())
                                  if hasattr(cell.cell_contents, '_current_obj')
                                       and
                                     callable(cell.cell_contents._current_obj)
                                  else cell
                              for cell in func.func_closure))

First, there’s that create_cell function, which will return a cell object. It seems to be the only way to create those types of objects: creating a closure and stealing it’s cell object.

Using the new module, it creates a new function borrowing code (globals, defaults, name, etc.) from the first function, and it goes through the cells of the passed-in closure to modify them if necessary. That way, we keep the actual object in the closure, not the proxy object that won’t resolve correctly in the other thread I’m going to launch.

Voilà! That opens the road to a very cool feature I’ll talk about shortly. Comments are welcome.

UPDATE 18h22: rephrased a little bit clunky sentences

UPDATE Match 18th 2010: Added link to what I’ve talk about shortly after

UPDATE March 20th 2010: fixed code block, thanks to John