John Smith's blog

Ramblings about (mostly) technical stuff

You are viewing a version of this site optimized for smaller devices. If you prefer, you can view the full site via this link.
Posts matching 'webapp2'

Converting old App Engine code to Python 2.7/Django 1.2/webapp2

Posted on

I'm borrowing the code for this blog for another project I'm working on, and it seemed to make sense to take the opportunity to bring it up to speed with the latest-and-greatest in the world of App Engine, which is:

  • Python 2.7 (the main benefit for me; I don't like having to dick around with 2.6 or 2.5 installations)
  • multithreading (not really needed for the negligible traffic I get, but worth having, especially given that the new billing scheme seems to assume you'll have this enabled if you don't want to be ripped off)
  • webapp2 (which seems to the recommended serving mechanism if you're not going to a "proper" Django infrastructure)
  • Django 1.2 templating (I'd used this on a work project a few months ago, but the blog was still using 0.96

Of course, having so many changed elements in the mix in a single hit is a recipe for disaster; with things breaking left, right and centre, trying to work out what the cause was was a bit needle-in-a-haystackish. It didn't help that the Py2.7 docs on the official site are still very sketchy, so I ended up digging through the library code quite a bit to suss out what was happening.

As far as I can tell, I've now got everything fixed and working - although this site is still running the old code, as the Python 2.7 runtime has a dependency on the HR datastore, and this app is still using Master/Slave.

I ended up writing a mini-app, in order to develop and test the fixes without all the cruft from my blog code, which I'll see about uploading to my GitHub account at some point. In the mean-time, here are my notes about the stuff I changed. I'm sure there are things which are sub-optimal or incomplete, but hopefully they might save someone else time...

app.yaml

  • Change runtime from python to python27
  • Add threadsafe: true
  • Add a libraries section: libraries: - name: django version: "1.2"
  • Change handler script references from foo.py to foo.app
  • Only scripts in the top-level directory work as handlers, so if you have any in subdirectories, they'll need to be moved, and the script reference changed accordingly: - url: /whatever # This doesn't work ... # script: lib/some_library/handler.app # ... this does work script: handler.app

Templates

  • In Django 1.2 escaping is enabled by default. If you need HTML to be passed through unmolested, use something like: {% autoescape off %} {{ myHTMLString }} {% endautoescape %}
  • If you're using {% extends %}, paths are referenced relative to the template base directory, not to that file. Here's an table showing examples of the old and new values:
    File Old {% extends %} value New {% extends %} value
    base.html N/A N/A
    admin/adminbase.html "../base.html" "base.html"
    admin/index.html "adminbase.html" "admin/adminbase.html"
  • If you have custom tags or filters, you need to {% load %} them in the template, rather than using webapp.template.register_template_library() in your main Python code.
    e.g.
    Old code (in your Python file): webapp.template.register_template_library('django_custom_tags') New code (in your template): {% load django_custom_tags %} (There's more that has to be done in this area; see below.)

Custom tag/filter code

  • Previously you could just have these in a standalone .py file which would be pulled in via webapp.template.register_template_library(). Instead now you'll have to create an Django app to hold them:
    1. In a Django settings.py file, add the new app to INSTALLED_APPS e.g.: INSTALLED_APPS = ('customtags')
    2. Create an app directory structure along the following lines: customtags/ customtags/__init__.py customtags/templatetags/ customtags/templatetags/__init__.py customtags/templatetags/django_custom_tags.py Both the __init__.py files can be zero-length. Replace customtags and django_custom_tags with whatever you want - the former is what should be referenced in INSTALLED_APPS, the latter is what you {% load "whatever" %} in your templates.
    3. In your file(s) in the templatetags/ directory, you need to change the way the new tags/filters are registered at the top of the file.
      Old code: from google.appengine.ext.webapp import template register = template.create_template_register() New code: from django.template import Library register = Library() The register.tag() and register.filter() calls will then work the same as previously.

Handlers

  • Change from google.appengine.ext import webapp to import webapp2 and change your RequestHandler classes and WSGIApplication accordingly
  • If your WSIApplication ran from within a main() function, move it out.
    e.g.
    Old code:
    def main(): application = webapp.WSGIApplication(...) wsgiref.handlers.CGIHandler().run(application) if __name__ == '__main__': main() New code: app = webapp2.WSGIApplication(...) Note in the new code:
    1. The lack of a run() call
    2. That the WSGIApplication must be called app - if it isn't, you'll get an error like: ERROR 2012-01-29 22:17:37,607 wsgi.py:170] Traceback (most recent call last): File "/proj/3rdparty/appengine/google_appengine_161/google/appengine/runtime/wsgi.py", line 168, in Handle handler = _config_handle.add_wsgi_middleware(self._LoadHandler()) File "/proj/3rdparty/appengine/google_appengine_161/google/appengine/runtime/wsgi.py", line 220, in _LoadHandler raise ImportError('%s has no attribute %s' % (handler, name)) ImportError: has no attribute app
  • Any 'global' changes you might make at the main level won't be applied across every invocation of the RequestHandlers - I'm thinking of things like setting a different logging level, or setting the DJANGO_SETTINGS_MODULE. These have to be done within the methods of your handlers instead. As this is obviously painful to do for every handler, you might consider using custom handler classes to handle the burden - see below.

Rendering Django templates

The imports and calls to render a template from a file need changing.
Old code: from google.appengine.ext.webapp import template ... rendered_content = template.render(template_path, {...}) New code: from django.template.loaders.filesystem import Loader from django.template.loader import render_to_string ... rendered_content = render_to_string(template_file, {...}) As render_to_string() doesn't explicitly get told where your templates live, you need to do this in settings.py: import os PROJECT_ROOT = os.path.dirname(__file__) TEMPLATE_DIRS = (os.path.join(PROJECT_ROOT, "templates"),)

Custom request handlers

As previously mentioned, where previously you could easily set global environment stuff, these now have to be done in each handler. As this is painful, one nicer solution is to create a special class to set all that stuff up, and then have your handlers inherit from that rather than webapp2.RequestHandler.

Here's a handler to be more talkative in the logs, and which also sets up the DJANGO_SETTINGS_MODULE environment variable. class LoggingHandler(webapp2.RequestHandler): def __init__(self, request, response): self.initialize(request, response) logging.getLogger().setLevel(logging.DEBUG) self.init_time = time.time() os.environ["DJANGO_SETTINGS_MODULE"] = "settings" def __del__(self): logging.debug("Handler for %s took %.2f seconds" % (self.request.url, time.time() - self.init_time)) A couple of things to note:

  1. the webapp2.RequestHandler constructor takes request and response parameters, whereas webapp.RequestHandler just took a single self parameter
  2. Use the .initialize() method to set up the object before doing your custom stuff, rather than __init__(self)

About this blog

This blog (mostly) covers technology and software development. It runs on a custom engine hosted on Google App Engine - the blog code is under continued development - theoretically at least :-) -so please excuse any bugs or glitches you may encounter.

RSS icon, courtesy of www.feedicons.com RSS feed for this blog

About the author

I'm a web developer for a London-based equities exchange. I've worked at organizations such as News Corporation and Google. Projects I've been involved in have been covered in outlets such as The Guardian, The Telegraph, The Register and TechCrunch.

Twitter | LinkedIn | GitHub | My CV | Mail

Recent posts

  1. Test post to verify migration to App Engine High-Replication Datastore worked OK
  2. Fixing slow emacs startup on Linux under VMWare
  3. gl.enableVertexAttribArray() gotcha
  4. nVidia Linux v302 drivers and dual-head/rotated monitor setups
  5. Hassles with array access in WebGL, and a couple of workarounds
  6. Reinvented the wheel and built my own IP address checker
  7. Parallax starfield and texture mask effect in WebGL
  8. In praise of help() in Python's REPL
  9. Enhanced version of Python's SimpleHTTPServer that supports HTTP Range
  10. What's the best way of including SVGs in a responsive web page?

Popular tags

  1. javascript (10 posts)           
  2. python (10 posts)           
  3. linux (8 posts)         
  4. browsers (7 posts)        
  5. app engine (6 posts)       
  6. dell (6 posts)       
  7. fedora (6 posts)       
  8. svg (6 posts)       
  9. windows (6 posts)       
  10. chrome (4 posts)     
  11. html5 (4 posts)     
  12. ipad (4 posts)     
  13. netbook (4 posts)     
  14. ux (4 posts)     
  15. firefox (3 posts)    
  16. fonts (3 posts)    
  17. github (3 posts)    
  18. html (3 posts)    
  19. http (3 posts)    
  20. newspapers (3 posts)    

Other sites I've built or been involved with

Work

Personal/fun/experimentation

Latest tweets

Worrying how people who clearly have no idea what they're talking about are making decisions affecting the rest of us http://t.co/CQI9fdxo4p
Went a week w/o GoT being spoiled (in US at an HBO-less hotel); now seeing how long I can remain oblivious of whatever Apple just announced.
I can't imagine a better time to try to convince people that an always-online device that watches them is a must-have item #XboxE3 #PRISM
Ages ago, I wanted to make a B3ta-ish quiz called "Startup or Stasi", to highlight dodgy privacy statements & T&Cs of FB etc. Too late now.
Told by an official at LHR that using my new e-passport will always require a human to check it due to my name :-( #isntthatdiscrimination
Maybe I'm massively behind the times, but I've only just been made aware of this "The Internship" thing - oh dear....
Wondered why "Google Wallet Service" was showing up after recently updating to a recent Chromium nightly build: http://t.co/I9XFAdTvOO
Just had an accident claim nuisance caller accuse me of wasting *his* time - didn't believe I'd had an accident in a car with reg P155OFF...
Good to see the serious business press highlighting the most important aspect of the latest tech company acquisition http://t.co/9vh9pLurGV
"with foo as bar" clauses having the values the opposite way round in SQL vs Python irritates me far more than it probably should
RT @firefox: "I want companies to be able to track every move I make online and sell that information to make money." - No one ever.
Not sure whether to laugh or cry whenever The Guardian run an article about protecting your privacy online.... http://t.co/42xRZRmyTV
Twitter meltdown in 5.. 4... 3... 2... 1...
So Richard Branson (born 1950) says he "[learned] how to code and program computers when I was a kid" ?!? Hmmm.... http://t.co/krdGaUlSu0