Safe deployment on App Engine

In a previous article I wrote about the need to update your Datastore indexes on App Engine before deploying code that relies on them. To simplify the deployment process, I put together a script that waits for your indexes to be built before deploying new code.

#!/usr/bin/env python

# Need to import and fix the sys path before importing other things.
import remote_api_shell
remote_api_shell.fix_sys_path()

import time

from google.appengine.api import datastore_admin
from google.appengine.ext.remote_api import remote_api_stub
from google.appengine.tools import appengine_rpc


APP_ID = 'your-app-id'


def configure_remote_api():
  def auth_func():
    return ('your.email@gmail.com', 'your.application.specific.password')

  remote_api_stub.ConfigureRemoteApi(
      APP_ID, '/_ah/remote_api', auth_func,
      servername='%s.appspot.com' % APP_ID,
      save_cookies=True, secure=False,
      rpc_server_factory=appengine_rpc.HttpRpcServer)
  remote_api_stub.MaybeInvokeAuthentication()


def main():
  print 'Checking indexes...'

  configure_remote_api()

  interval = 10  # seconds.
  building = True

  while building:
    # Start with a fresh run: maybe we're done building?
    building = False

    for index in datastore_admin.GetIndices('s~%s' % APP_ID):
      # If any single index is building, we're not done.
      # Sleep for a bit and then this cycle should repeat.
      if not index.has_state() or index.state() != index.READ_WRITE:
        building = True
        print 'Indexes are still building... Waiting %s seconds.' % interval
        time.sleep(interval)
        break

  print 'All indexes are up to date.'


if __name__ == '__main__':
  main()

Since this script will “block” until all indexes are built and ready, you can add this line to your deployment script, which should look something like...

$ appcfg.py update_indexes .
$ ./wait_for_indexes.py  # (The script above...)
$ appcfg.py update .