Alexandre Bourget

From a web point of view, if you want to stop a user from doing stupid things (like submitting a form before reviewing it, sending a heinous e-mail late at night, or clicking a button that launches a nuclear missile), you might want to do one one of three things:

• Pop-up a confirmation dialog
• Go to a second page, with yet another submit button asking for confirmation, with a summary of the submitted data
• Just let it go, but offer a timed “UNDO” or “CANCEL” link, like Gmail implemented for e-mails

Of course, the best of all three is the latter. The difference here is major. It’s the difference between pro-active annoyingness (pop-ups asking confirmation when you know very well you want to send the e-mail), or reactive flexibility. It also gives your user a great boost of confidence in your application, because the user can do stupid things and knows he’ll be able to fix them in a timely manner.

Now here’s how to do it in Pylons, using the just-released WebUndo lib. You can find it on PyPI, so just pip install WebUndo from your project.

We’re lucky Pylons runs as a single application with threads, since that’s what we’ll use to delay our operations and give the user a chance to cancel. PHP apps for instance, wouldn’t give us that flexibility, since everything vanishes once the request is sent. Take a look at the source code to know how it’s built.

Let’s consider this piece of Pylons controller:

class ThController(BaseController):
    def index(self):
        return render('th.mako')

    def save(self):
        open('/tmp/dump.txt', 'w').write(request.params.get('stuff'))
        return "Everything saved"

where th.mako contains a simple form with a text field named stuff. In this case, once you submit the form in th.mako, it’s done: the file is written to disk, and there’s no way you can cancel it.

Now let’s have a look at this version:

class ThController(BaseController):
    def index(self):
        return render('th.mako')

    def save(self):
        req = request
        def do_save():
            open('/tmp/dump.txt', 'w').write(req.params.get('stuff'))
        key = webundo.launch_cancelable_job(do_save, 5)
        return 'Saving, click to <a href="%s">cancel</a>' % h.url_for(controller='th', action='cancel', id=key)

    def cancel(self, id):
        if webundo.cancel_job(id):
            return "Thread cancelled successfully"
        else:
            return "Too late, thread finished already."

We added some pieces of webundo, the save() action returns a confirmation message, and a link to cancel the operation. We also added a def do_save() wrapper, a call to the launch_cancelable_job function and the passing on of the key. Notice also the req = request, used to make sure the request object gets closed in the do_save() function. The fun part here, is that the operation will not be carried out until the 5 seconds (second parameter to launch_cancelable_job) are elapsed. The last thing is obviously, the cancel action, which will deal with the cancelation.

Using this feature, even if you close your browser window after calling the save action, the do_save() function will be executed normally, only delayed by 5 seconds.

For the Pylons addicts who read that code and say “hey, that request object can’t traverse threads, it’s a StackedObjectProxy”! See the details and answer in my other blog post.

Of course, there are cases where you want a certain operation to be carried out directly (when calling save() for example) but you want some special code to be executed if someone click an undo button for example. The WebUndo lib provides you with another pair of functions to do just that: launch_undoable_job and undo_job.

Take a look at the documentation for full details. Happy undoing!

UPDATED March 20th: typos following feedback

I was working on a feature to implement Gmail’s “Undo” feature in Pylons (which I’ll present on this blog soon), and was faced with a challenge regarding the passage of variables (or functions) to a separate thread. The reason is that Pylons resolves the current request object according to the local thread. You won’t find the same request object in a different thread from which it originated.

I need to be able to resolve those objects (to get the actual object, not the proxy object) before spawning a new thread. Though, I don’t want to require the user to know about all this. He should only create a function that should work in the current thread context, and my code will make sure the good objects get passed on to the new thread.

In Python terms, I wanted to be able to modify the func_closure attribute of a function object (it’s called __closure__ from Python 2.6 onwards). I searched around, only to find that it’s really a read-only attribute, unless you want to do risky things.

So, if you do something like:

def a():
    b = 123
    def myclosure():
        print b
    return myclosure
myfunc = a()

the value of b gets referenced (as a cell object) in the myfunc.func_closure tuple. But there’s no way to change it. What I need is to tweak the myfunc function’s closure, and replace the internal value of b for something else. So when I’d call myfunc(), it actually prints something else than 123, let’s say 'mynewvalue'.

At first, I tried the mix-and-match suggested here, which looks like:

import new
def getitem(a):
    def retclosure():
        return a
    return retclosure
tmpfunc = getitem('mynewvalue')
cell = tmpfunc.func_closure[0]
rewrittenfunc = new.function(myfunc.func_code , myfunc.func_globals, myfunc.func_name, myfunc.func_defaults, (cell,))

Basically, it takes the code, globals, name and defaults from the myfunc function, and uses the cells from another function, mixing two different functions into one.

That’s great for a single reference in the closure, but it’s not very flexible. I want my code to go through all of the closed in objects, and verify is they are of a certain type (namely StackedObjectProxy), and resolve those that need local resolution. It should work with a random number of arguments.

Here is my solution:

def unproxy_closure(f):
    """Rewrite function with resolved closure references (StackedObjectProxy from current thread)"""
    import new
    def create_cell(obj):
        return (lambda: obj).func_closure[0]
    return new.function(func.func_code , func.func_globals, func.func_name,
                        func.func_defaults,
                        tuple(create_cell(cell.cell_contents._current_obj())
                                  if hasattr(cell.cell_contents, '_current_obj')
                                       and
                                     callable(cell.cell_contents._current_obj)
                                  else cell
                              for cell in func.func_closure))

First, there’s that create_cell function, which will return a cell object. It seems to be the only way to create those types of objects: creating a closure and stealing it’s cell object.

Using the new module, it creates a new function borrowing code (globals, defaults, name, etc.) from the first function, and it goes through the cells of the passed-in closure to modify them if necessary. That way, we keep the actual object in the closure, not the proxy object that won’t resolve correctly in the other thread I’m going to launch.

Voilà! That opens the road to a very cool feature I’ll talk about shortly. Comments are welcome.

UPDATE 18h22: rephrased a little bit clunky sentences

UPDATE Match 18th 2010: Added link to what I’ve talk about shortly after

UPDATE March 20th 2010: fixed code block, thanks to John

So this is the short story of my presentation at the Montreal Python user groups on january 25th 2010.

I’m preparing to give a talk on “Pylons, web development done right” (or something alike), at the Confoo conference in March. I wanted to test-drive my presentation, so I decided to jump in after seeing a couple of presentation over there - some of which were pretty so-so, thus pushing me to risk myself.

I wanted the presentation to be a performance (since this is my primary training, as a pianist), a lightning bolt filled with impressive demonstrations, speedy typing, cool technologies using a sweet programming language. I wanted to show off Pylons (http://www.pylonshq.com) and a bunch of WSGI sweetnesses (memento, WPHP, CleverCSS, Beaker, Routes, Mako, SQLAlchemy, SqlSoup, etc.)

So I packed my Karmic system with a Squid3 proxy server, to fake having an Internet connection (while everyone in the room was struggling with the bad/non-existant Wi-Fi connection). I wrote some Python scripts that would do all sorts of background mangling to accelerate the live demos* and organised the presentation in modules, so that I could dynamically add or remove pieces of the demonstration based on interests I sized in the audience.

Finally, the presentation was greatly appreciated if I believe the comments I have received. It was a very good experience for me and I received a gift, which is in fact the subject/object of this post: the book “Friends with benefits“.

Read on...

Tired of using paster serve –reload development.ini and waiting for your code to reload ?

memento allows you to dynamically reload your code – without restarting your whole server – or parts of your code.

Install it:

$ easy_install memento

To integrate with Pylons, follow these steps:

Go to your config/middleware.py, and add:

import memento

Then, a bit lower, replace:

def make_app(...):
    ...
    # The Pylons WSGI app
    app = PylonsApp()
    ...

by:

def make_app(...):
    ...
    # The Pylons WSGI app
    #app = PylonsApp()
    app = memento.Assassin('pylons.wsgiapp:PylonsApp()', ['yourpackage'])
    ...

The second parameter to Assassin() is a list of packages you want reloaded on each request. You probably want your whole package to be reloaded, or you can be more granular.

NOTE: don’t forget the () after the PylonsApp, because it has to call the object to get a WSGI app. You’ll get errors otherwise

The neat thing is I think you can disable that in real-time also, by changing the value of app.mode (’on’ = ‘on and ‘off’ = ‘off, as you’ve guessed), so I guess you’d like to keep a copy of that ‘app’ before you lose it’s reference lower in middleware.py.

Using BeautifulSoup, in one line, you can:

' '.join(h.BeautifulSoup.BeautifulSoup(some_content).findAll(text=True))

SQLAlchemy allows you to pass an optionnal `timezone` argument to the DateTime types object. However, it’s only used by the PostgreSQL backend.

I was looking for a solution that would save consistently time-offset-aware `datetime` objects.

I found something pretty simple in the PylonsHQ pasties:

import sqlalchemy as sqla
from pytz import UTC

class UTCDateTime(sqla.types.TypeDecorator):
    impl = sqla.types.DateTime
    def convert_bind_param(self, value, engine):
        return value
    def convert_result_value(self, value, engine):
        return UTC.localize(value)

Defining this in your application allows SQLAlchemy not to depend on the `pytz` package, and allows you to have simple UTC-everywhere support in your database. Best of all, it’s cross-database.

In the actual state, you wouldn’t be able to get from the database an offset-aware `datetime` object. The `convert_bind_param` should be modified to tweak the result retrieved from the database. This way, you’d have offset-aware `datetime`s throughout.

Please comment if you try this

I was looking for Timezone support in Python, especially with SQLAlchemy.

I found several things that could help me implement timezone support in my application.

First, I found the pytz package, which is simply awsome, and deal with all your timezone needs

Here is an example usage of the pytz package. Here we load the necessary stuff:

import datetime
import pytz

Then we'll need to get my timezone, which is GMT-0500 but has all sorts of Daylight saving weirdness. Happily, pytz deal with all this itself. Let's also create a normal datetime object.

>>> mytz = pytz.timezone("America/Montreal")
>>> mytz
<DstTzInfo 'America/Montreal' EST-1 day, 19:00:00 STD>
>>> mydt = datetime.datetime.now()
>>> mydt
datetime.datetime(2009, 4, 22, 20, 0)

First operation, let's create a timezone-aware datetime object from this one. With fromutc, we'll assume my naive datetime was actually UTC time (at GMT+0000)

>>> newdt = mytz.fromutc(mydt)
>>> newdt
datetime.datetime(2009, 4, 22, 16, 0, tzinfo=<DstTzInfo 'America/Montreal' EDT-1 day, 20:00:00 DST>)

What if you have a random naive datetime object, and you'd like to attach a timezone to it to make it offset-aware ? This requires you to know to which timezone it belongs.

>>> this_time = datetime.datetime(2009, 4, 25, 12, 30, 0)
>>> vanc_tz = pytz.timezone('America/Vancouver')
>>> vanc_time = vanc_tz.localize(this_time)

Now what if you want to know the time in another timezone, but you only have the time in the vanc_tz timezone ?

>>> mtl_tz = pytz.timezone('America/Montreal')
>>> mtl_time = vanc_time.astimezone(mtl_tz)
>>> mtl_time
datetime.datetime(2009, 4, 25, 15, 30, tzinfo=<DstTzInfo 'America/Montreal' EDT-1 day, 20:00:00 DST>)

Noticed the switch from 12 to 15 ?

You can also get the current time for a certain timezone:

>>> vanc_now = datetime.datetime.now(vanc_tz)
>>> vanc_now
datetime.datetime(2009, 4, 22, 18, 0, 34, 117582, tzinfo=<DstTzInfo 'America/Vancouver' PDT-1 day, 17:00:00 DST>)

or at UTC:

>>> utc_now = datetime.datetime.utcnow()

Isn't that sweet ?