Python dictionary comprehensions

I’m a little embarrassed to say that I haven’t yet used these in my own code -- they seem so obvious now that I look back on it.

>>> d = {'a': 1, 'b': 2', 'c': 3}
>>> keys = ['a', 'c']
>>> { k: d[k] for k in keys }
{'a': 1, 'c': 3}

Getting started with tmux

As a long-time user of GNU screen, I knew it would be difficult to switch, but I’ve made the move and am so far pretty happy. tmux does all the same things that screen does and was a lot easier to customize. I figured I would share my tmux configuration file and some of the things that made the switch relatively painless.

Keyboard bindings

I changed the default keyboard bindings to be more like screen:

  • Ctrl-b is the default for tmux; I prefer Ctrl-a.
  • Ctrl-(left/right) arrow to move between windows. No screen equivalent (that I know of).
  • Ctrl-a twice to go “back”.
  • Setting Ctrl-a, r to reload the config file made it very easy to debug.
  • Status bar at the bottom

I set my status bar to use the session name and my machine name on the far left, open tab names centered in the middle, and the system time and date on the far right.

Project-specific configurations (and attach tool)

There's a cool project called tat that does some neat things to make project-specific configurations really easy to set up, and attach to them later on. It also includes tab-completion for existing sessions and your project directory names.

With tat you can create a .tmux file in your project directory, which will be run with the session name as first argument. You can use this to configure your project work space easily and consistently. Here’s one of my example .tmux project files:

# The session name should have been passed in, but if not
# default to 'default-session-name'
# Create and name some windows.
tmux rename-window -t $session:1 bash
tmux new-window -t $session:2 -n webserver
tmux new-window -t $session:3 -n python
tmux new-window -t $session:4 -n templates
tmux new-window -t $session:5 -n js
tmux new-window -t $session:6 -n less
# Configure the various windows.
tmux send-keys -t $session:2 "make serve" C-m
tmux send-keys -t $session:4 "cd templates && clear" C-m
tmux send-keys -t $session:5 "cd media/ && clear" C-m
tmux send-keys -t $session:6 "cd media/less/ && clear" C-m
# Attach to the first window.
tmux select-window -t $session:1


Enabling tab-completion for tmux was easy. On Ubuntu I did the following:

$ sudo curl > /usr/share/bash-completion/completions/tmux

Task Queue support in App Engine's ext.Testbed

A while back I wrote some (now deprecated) code that allowed you to easily test your Python App Engine applications and their interaction with the App Engine APIs. When we got acquired by Google, I started working with some of the awesome App Engine engineers on making that code part of the official App Engine codebase.

We launched that, but one of the APIs that was noticeably lacking helper-methods was the Task Queue. I added one of the old methods (get_filtered_tasks()) but a bit later I noticed a pretty serious bug where timezones weren’t handled properly.

So that code hid quietly in the App Engine code base, undocumented and technically broken. Sorry :(

I wrote some extra code to fix it, but that didn’t get merged into the main repository for quite a while.

But now it’s here.

If you’re curious about the change, feel free to check out the diff (scroll to the bottom). The method is still called get_filtered_tasks() and takes several arguments as “filters” for various properties of tasks — and it should properly handle timezones when you set the eta or countdown properties when creating tasks.

Sorry for the delay. And enjoy.

PS: Since this isn’t documented on the official App Engine page, the API may change. I don’t intend to change it, but I can’t promise it won’t. Apologies in advance if you use this helper method and have it break out from under you.

Also, check out the official code.

Some example code:

import unittest
from google.appengine.api import taskqueue
from google.appengine.ext import testbed

class TaskQueueTestCase(unittest.TestCase):

  def setUp(self):
    self.testbed = testbed.Testbed()
    self.task_queue_stub = self.testbed.get_stub(testbed.TASKQUEUE_SERVICE_NAME)

  def tearDown(self):

  def testTaskAdded(self):

    tasks = self.taskqueue_stub.get_filtered_tasks(url='/path/to/task')
    self.assertEqual(1, len(tasks))
    self.assertEqual('/path/to/task', tasks[0].url)


Login required should be the default

If you're building a web-app in Python that requires people to sign in, you are probably pretty familiar with the concept of a @login_required decorator. For those that might not be, it's pretty straight forward: the decorator wraps a handler method with a check for whether the current request is coming from someone who has been through your sign in process.

When I first saw this however many years ago, I was pretty new to Python decorators and thought it was pretty awesome. I still think the whole concept is pretty awesome. This seems to fit the mold of a "decorator" pretty perfectly... almost as if decorators were built with this in mind...

But what if you forget your @login_required decorator? It’s so easy to do. When you're testing that "Edit your account" page, you're authenticated, you assume everything should go to plan. You could (and should) add a functional test to check that an unauthenticated request is redirected, but this is just a quick little side project...

So why don’t we just make @login_required the default? I think that it should be, so here’s some code to save somebody else some time.

It works by wrapping all local methods with your login_required method during object creation (aka, __new__). If you want a method to be public, you decorate it with the @login_not_required decorator -- which sets a flag on the method saying... that login is not required.

Here's the BaseHandler (and Meta class) which does the wrapping:

import types

import webapp2

from auth import login_required

class BaseHandlerMeta(type):
  """Meta class for all request handlers.

  This automatically wraps all handler methods with the login_required
  decorator. If something should be exposed publicly, it should be wrapped
  with the login_not_required decorator.

  def __new__(cls, name, bases, local):
    if name != 'BaseHandler':
      for func_name, func in local.iteritems():
        if isinstance(func, types.FunctionType):
          local[func_name] = login_required(func)

    return type.__new__(cls, name, bases, local)

class BaseHandler(webapp2.RequestHandler):
  """Base class for all RequestHandlers."""

  __metaclass__ = BaseHandlerMeta

  def user_is_logged_in(self):
    # Do some magic here to check if someone is logged in
    return False

Here are the login_required and login_not_required decorators:

def login_not_required(handler_method):
  """Allows a user to *not* be logged in.

  The login_required attribute is inspected by BaseHandlerMeta and is used
  as the flag for whether to wrap a method with login_required or not.
  handler_method.login_required = False
  return handler_method

def login_required(handler_method):
  """Requires that a user be logged in."""

  required = getattr(handler_method, 'login_required', True)
  already_wrapped = getattr(handler_method, 'wrapped', False)

  # If the method doesn't require a login, or has already been wrapped,
  # just return the original.
  if not required or already_wrapped:
    return handler_method

  def check_login(self, *args, **kwargs):
    if not self.user_is_logged_in():
      uri = self.uri_for('login', redirect=self.request.path)
      self.redirect(uri, abort=True)
      return handler_method(self, *args, **kwargs)

  # Let others know that this method is already wrapped to avoid wrapping
  # it more than once...
  check_login.wrapped = True

  return check_login

This would make your request handlers look something like this:

import base
from auth import login_not_required

class MyHandler(base.BaseHandler):
  def my_public_method(self):
    self.response.write('Hello world!')

  def my_public_method_with_a_problem(self):
    # Calling this should force a redirect to the login page!

  def my_other_handler_that_is_protected(self):
    self.response.write('Hello privately!')

  def protected_by_default(self):
    self.response.write('Once you log in, you can view this!')

Notice that if you are in a public method (@login_not_required) and you call a method that does not have that decorator, you’ll get redirected to a login page. If you want something to be public, it and all of the functions it calls (inside the handler) should be explicitly defined as public.

Friendly URL routing in App Engine with webapp2

I used to do crazy things like hard-code URLs in my templates:

<a href="/accounts/create/">Create your account</a>


<a href="/accounts/view/?id={{ the_id }}">View account {{ the_id }}</a>

Ever since webapp2 came around, I've been really into the URL routing helpers, which make it much easier to keep long lists of URLs organized. I started using a new project layout that makes routing really simple -- maybe even fun.

Before I get to the interesting part I need to digress slightly. A while back, I posted a question on Stack Overflow asking whether it was better to break your app apart into separate routes in app.yaml or inside a WSGIApplication. Put simply: should you create one WSGIApplication object and route everything there? Or many separate ones, each with their own routing in app.yaml?

  • The short answer was: It doesn't really matter.
  • The longer answer was: if you're particularly sensitive about start-up time or memory, you might break things apart.

I decided to use one big app. Having a single app makes unit testing with ext.testbed a bit easier.

In addition to making testing easier, I've been using the Python 2.7 runtime which lends itself nicely to a single WSGIApplication: I put my code in and then route /.* to and don’t think about it anymore. But how are you supposed to keep track of all the URLs, Handlers, methods, and routes in your app as it grows larger and larger?

webapp2 to the rescue.

Take a look at

from webapp2_extras.routes import RedirectRoute

from handlers.account import AccountHandler

__all__ = ['application_routes']

application_routes = []

_route_info = [
  ('account.list',   'GET', '/accounts/',                 AccountHandler, 'list'),
  ('account.create', None,  '/accounts/create/',          AccountHandler, 'create'),
  ('account.view',   'GET', '/accounts/<id:\d+>/',        AccountHandler, 'view'),
  ('account.delete', None,  '/accounts/<id:\d+>/delete/', AccountHandler, 'delete'),
  ('account.update', None,  '/accounts/<id:\d+>/update/', AccountHandler, 'update'),

for name, methods, pattern, handler_cls, handler_method in _route_info:
  # Allow a single string, but this has to be changed to a list.
  # None here means any method
  if isinstance(methods, basestring):
    methods = [methods]

  # Create the route
  route = RedirectRoute(name=name, template=pattern, methods=methods,
                        handler=handler_cls, handler_method=handler_method)

  # Add the route to the public list

There are a few things to notice here:

  1. You could type out all the boiler-plate every time. I don't think routes have enough distinguishing factors to merit that. I removed the boiler plate and put together a list of tuples.
  2. Each tuple defines a route: (name, methods, URL pattern, HandlerClass, handler_method).
  3. Routes are named as thing.action (thing is singular -- not things).
  4. The method can be 'GET', 'POST', None, or any other method (or list of methods). None simply means "I don't care".
  5. URL patterns allow fancy matching (<id:\d+>).
  6. Rather than having many handlers (CreateUserHandler, DeleteUserHandler), I use one handler per thing and have the methods correspond to the actions (UserHandler.create, UserHandler.delete). If necessary, I can use self.request.method to distinguish between intent (ie, "show me the account creation form" versus "create my account and rediect me somewhere").
  7. The handler method name (AccountHandler.list) and the action part of the route name (account.list) are identical. This makes linking easy -- you know the route's name because it lines up with your code.
  8. I break from PEP8 about lining data up vertically. This is data you'd put in a table. I don't like jagged tables.

Now to tie this all together, I just put some configuration for webapp2 into

__all__ = ['webapp2_config']

webapp2_config = {
  'webapp2_extras.sessions': {
    'secret_key': 'Put some magical secret here.', 
  'webapp2_extras.jinja2': {
    'environment_args': {
      'autoescape': True,
and hook it all up in

import webapp2

from config import webapp2_config
from routes import application_routes

app = webapp2.WSGIApplication(routes=application_routes, config=webapp2_config)
After that, your handlers might look like this:

import webapp2
from webapp2_extras import auth
from webapp2_extras import jinja2
from webapp2_extras import sessions

class BaseHandler(webapp2.RequestHandler):
  def jinja2(self):
    return jinja2.get_jinja2(

  def auth_config(self):
    return {'login_url': self.uri_for('login'),
            'logout_url': self.uri_for('logout')}

  def auth(self):
    return auth.get_auth()

  def session_store(self):
    return sessions.get_store(request=self.request)

  def dispatch(self):
    """Override dispatch to persist session data."""
      super(BaseHandler, self).dispatch()

  def render_template(self, template, context=None):
    context = context or {}

    extra_context = {
      'request': self.request,
      'uri_for': self.uri_for,
    # Only override extra context stuff if it's not set by the template:
    for key, value in extra_context.items():
      if key not in context:
        context[key] = value

    rendered = self.jinja2.render_template(template, **context)
from handlers import base

class AccountHandler(base.BaseHandler):
  def create(self):
    if self.request.method == 'POST':
      # Create the account
      # Redirect them to the dashboard page
      return self.redirect(self.uri_for('account.view'))
      return self.render_template('account_create.html')

And your templates can now create links that look like this:

<a href="{{ uri_for('account.create', my_param='something') }}">Create your account</a>


<a href="{{ uri_for('account.view', id=the_id) }}">View account {{ the_id }}</a>

Extra bonus: If you delete a URL and the template tries to look it up, you’ll get an exception — and so will your unit tests. I guess this could be a scary thing if you don’t have tests, but hopefully it is helpful.