John Smith's Blog

Ramblings (mostly) about technical stuff

Converting old App Engine code to Python 2.7/Django 1.2/webapp2

Posted by John Smith on

I'm borrowing the code for this blog for another project I'm working on, and it seemed to make sense to take the opportunity to bring it up to speed with the latest-and-greatest in the world of App Engine, which is:

  • Python 2.7 (the main benefit for me; I don't like having to dick around with 2.6 or 2.5 installations)
  • multithreading (not really needed for the negligible traffic I get, but worth having, especially given that the new billing scheme seems to assume you'll have this enabled if you don't want to be ripped off)
  • webapp2 (which seems to the recommended serving mechanism if you're not going to a "proper" Django infrastructure)
  • Django 1.2 templating (I'd used this on a work project a few months ago, but the blog was still using 0.96

Of course, having so many changed elements in the mix in a single hit is a recipe for disaster; with things breaking left, right and centre, trying to work out what the cause was was a bit needle-in-a-haystackish. It didn't help that the Py2.7 docs on the official site are still very sketchy, so I ended up digging through the library code quite a bit to suss out what was happening.

As far as I can tell, I've now got everything fixed and working - although this site is still running the old code, as the Python 2.7 runtime has a dependency on the HR datastore, and this app is still using Master/Slave.

I ended up writing a mini-app, in order to develop and test the fixes without all the cruft from my blog code, which I'll see about uploading to my GitHub account at some point. In the mean-time, here are my notes about the stuff I changed. I'm sure there are things which are sub-optimal or incomplete, but hopefully they might save someone else time...

app.yaml

  • Change runtime from python to python27
  • Add threadsafe: true
  • Add a libraries section: libraries: - name: django version: "1.2"
  • Change handler script references from foo.py to foo.app
  • Only scripts in the top-level directory work as handlers, so if you have any in subdirectories, they'll need to be moved, and the script reference changed accordingly: - url: /whatever # This doesn't work ... # script: lib/some_library/handler.app # ... this does work script: handler.app

Templates

  • In Django 1.2 escaping is enabled by default. If you need HTML to be passed through unmolested, use something like: {% autoescape off %} {{ myHTMLString }} {% endautoescape %}
  • If you're using {% extends %}, paths are referenced relative to the template base directory, not to that file. Here's an table showing examples of the old and new values:
    File Old {% extends %} value New {% extends %} value
    base.html N/A N/A
    admin/adminbase.html "../base.html" "base.html"
    admin/index.html "adminbase.html" "admin/adminbase.html"
  • If you have custom tags or filters, you need to {% load %} them in the template, rather than using webapp.template.register_template_library() in your main Python code.
    e.g.
    Old code (in your Python file): webapp.template.register_template_library('django_custom_tags') New code (in your template): {% load django_custom_tags %} (There's more that has to be done in this area; see below.)

Custom tag/filter code

  • Previously you could just have these in a standalone .py file which would be pulled in via webapp.template.register_template_library(). Instead now you'll have to create an Django app to hold them:
    1. In a Django settings.py file, add the new app to INSTALLED_APPS e.g.: INSTALLED_APPS = ('customtags')
    2. Create an app directory structure along the following lines: customtags/ customtags/__init__.py customtags/templatetags/ customtags/templatetags/__init__.py customtags/templatetags/django_custom_tags.py Both the __init__.py files can be zero-length. Replace customtags and django_custom_tags with whatever you want - the former is what should be referenced in INSTALLED_APPS, the latter is what you {% load "whatever" %} in your templates.
    3. In your file(s) in the templatetags/ directory, you need to change the way the new tags/filters are registered at the top of the file.
      Old code: from google.appengine.ext.webapp import template register = template.create_template_register() New code: from django.template import Library register = Library() The register.tag() and register.filter() calls will then work the same as previously.

Handlers

  • Change from google.appengine.ext import webapp to import webapp2 and change your RequestHandler classes and WSGIApplication accordingly
  • If your WSIApplication ran from within a main() function, move it out.
    e.g.
    Old code:
    def main(): application = webapp.WSGIApplication(...) wsgiref.handlers.CGIHandler().run(application) if __name__ == '__main__': main() New code: app = webapp2.WSGIApplication(...) Note in the new code:
    1. The lack of a run() call
    2. That the WSGIApplication must be called app - if it isn't, you'll get an error like: ERROR 2012-01-29 22:17:37,607 wsgi.py:170] Traceback (most recent call last): File "/proj/3rdparty/appengine/google_appengine_161/google/appengine/runtime/wsgi.py", line 168, in Handle handler = _config_handle.add_wsgi_middleware(self._LoadHandler()) File "/proj/3rdparty/appengine/google_appengine_161/google/appengine/runtime/wsgi.py", line 220, in _LoadHandler raise ImportError('%s has no attribute %s' % (handler, name)) ImportError: has no attribute app
  • Any 'global' changes you might make at the main level won't be applied across every invocation of the RequestHandlers - I'm thinking of things like setting a different logging level, or setting the DJANGO_SETTINGS_MODULE. These have to be done within the methods of your handlers instead. As this is obviously painful to do for every handler, you might consider using custom handler classes to handle the burden - see below.

Rendering Django templates

The imports and calls to render a template from a file need changing.
Old code: from google.appengine.ext.webapp import template ... rendered_content = template.render(template_path, {...}) New code: from django.template.loaders.filesystem import Loader from django.template.loader import render_to_string ... rendered_content = render_to_string(template_file, {...}) As render_to_string() doesn't explicitly get told where your templates live, you need to do this in settings.py: import os PROJECT_ROOT = os.path.dirname(__file__) TEMPLATE_DIRS = (os.path.join(PROJECT_ROOT, "templates"),)

Custom request handlers

As previously mentioned, where previously you could easily set global environment stuff, these now have to be done in each handler. As this is painful, one nicer solution is to create a special class to set all that stuff up, and then have your handlers inherit from that rather than webapp2.RequestHandler.

Here's a handler to be more talkative in the logs, and which also sets up the DJANGO_SETTINGS_MODULE environment variable. class LoggingHandler(webapp2.RequestHandler): def __init__(self, request, response): self.initialize(request, response) logging.getLogger().setLevel(logging.DEBUG) self.init_time = time.time() os.environ["DJANGO_SETTINGS_MODULE"] = "settings" def __del__(self): logging.debug("Handler for %s took %.2f seconds" % (self.request.url, time.time() - self.init_time)) A couple of things to note:

  1. the webapp2.RequestHandler constructor takes request and response parameters, whereas webapp.RequestHandler just took a single self parameter
  2. Use the .initialize() method to set up the object before doing your custom stuff, rather than __init__(self)

About this blog

This blog (mostly) covers technology and software development.

Note: I've recently ported the content from my old blog hosted on Google App Engine using some custom code I wrote, to a static site built using Pelican. I've put in place various URL manipulation rules in the webserver config to try to support the old URLs, but it's likely that I've missed some (probably meta ones related to pagination or tagging), so apologies for any 404 errors that you get served.

RSS icon, courtesy of www.feedicons.com RSS feed for this blog

About the author

I'm a software developer who's worked with a variety of platforms and technologies over the past couple of decades, but for the past 7 or so years I've focussed on web development. Whilst I've always nominally been a "full-stack" developer, I feel more attachment to the back-end side of things.

I'm a web developer for a London-based equities exchange. I've worked at organizations such as News Corporation and Google and BATS Global Markets. Projects I've been involved in have been covered in outlets such as The Guardian, The Telegraph, the Financial Times, The Register and TechCrunch.

Twitter | LinkedIn | GitHub | My CV | Mail

Popular tags

Other sites I've built or been involved with

Work

Most of these have changed quite a bit since my involvement in them...

Personal/fun/experimentation