Alexandre Bourget

This is part 6 of my March 9th 2011 Confoo presentation. Refer to the Table of Contents for the other parts.

The original announcement for my Confoo presentation was to use RabbitMQ to do real-time communications, but then, after finding a way to use Redis to do it, I found it was simpler and more straight-forward for a live demo.

Redis Integration

In this part, we will use Redis's PubSub features to communicate in real-time between the different web browsers connected with a Socket.IO (or a Websocket) to our Gevent-based application.

Redis is a FOSS database which supports the Publish/Subscribe metaphore. It is written in C, and is quite robust. It also comes pre-setup on your Ubuntu machine.

To install Redis, simply run:

$ sudo apt-get install redis-server

and you're ready to go! To get things into production, you'll want to learn how to secure your installation.

You can try the redis server locally if you kill the system-wide instance (through init.d/service scripts):

$ redis-server
...
[21194] 08 Mar 22:08:54 * Server started, Redis version 1.3.15
...
[21194] 08 Mar 22:08:54 * The server is now ready to accept connections on port 6379
[21194] 08 Mar 22:08:54 - 0 clients connected (0 slaves), 533436 bytes in use

Python Redis bindings and PubSub commands

Let's setup what is needed for messaging using Redis.

First of all, we'll want to connect to the Redis server and publish a new message. We'll add some code to the views.py and set up the publisher. We'll name the channel foo:

...
import redis
from json import loads, dumps
...
def encode_video(filename, request):
    ...
    if p.returncode == 0:
        r = redis.Redis()
        msg = {'type': 'video', 'fileid': str(fileid),
               'url': request.route_url('video', fileid=fileid)}
        r.publish("foo", dumps(msg))

Now, we'll want to have our Socket.IO socket listen for any incoming message from the Redis subscription:

class ConnectIOContext(SocketIOContext):
    def msg_connect(self, msg):
        ...
        def listener():
            r = redis.Redis()
            r.subscribe(['foo'])
            for m in r.listen():
                if not self.io.connected():
                    return
                print "From Redis:", m
                if m['type'] == 'message':
                    self.io.send(loads(m['data']))
        self.spawn(listener)

When we receive a message as the m object, we'll just pass it directly to the current Socket.IO socket. We've subscribed to the foo channel, so we'll receive a copy of any message sent to it.

We will also want the client to handle the new message, so we'll tweak the call to go() and the definition for the go() function, in index.html:

      <script>
        function go(url) {
          $('#video').html('<div>Video ready: ' + url + '</div><video controls preload src="'+ url + '" />');
          ...
        }
      </script>

      ...
      socket.on('message', function(obj) {
        ...
        if (obj.type == "video") {
          go(obj.url);
        }
        ...

In here, if the message is of type video, like the one sent by the encode_video() function, it will trigger the playback immediately.

Let's make sure a background process is spawned when someone asks us to encode a video. The encoding process itself could be totally decoupled from this server, sent to a render farm for example. In views.py, we would tweak:

def get_iframe(request):
    if 'file' in request.params:
        ...
        gevent.spawn(encode_video, '/tmp/video.mp4', request)

That's it! Hope you liked the series! I might add the Android app demo if I have time, so please be patient.

If you would like us to implement anything you've seen here in a real-life project, or to kick start some of your projects, don't hesitate to send out an e-mail at contact@savoirfairelinux.com and mention this blog.