Appengine and handle_entity
Jun 17 2009, 12:52AM
Just spent a frustrating evening with Google AppEngine - and I want to document the problem I had in case it helps others.
I wanted to upload data to AppEngine from a CSV file - however the values in said file needed some adjustments (they don't directly correspond to the fields in the table I have, and some are ReferenceProperty fields). The AppEngine documentation on uploading data says nothing about overriding handle_entity to deal with these requirements.
I found a couple of pages that appeared to be what I needed. They document how to override HandleEntity() in your loader class. I spent ages futzing with this. It wasn't until I looked in the SDK source code that I realized the function to override is called handle_entity()! There is an alias_old_names() in there you can call - but who knew?
For the record, I'm using Release 1.2.2 of the AppEngine SDK
Here is what my loader class looks like now:
import datetime, os, sys # 'sources' and 'models' down there are not on the python path curr_path = os.path.abspath(os.path.dirname(os.path.realpath(__file__))) sys.path = ['/'.join(curr_path.split('/')[:-1])] + sys.path from google.appengine.ext import db from google.appengine.tools import bulkloader from sources import cct from models import Label, cctRegion, cctVariable, Record class RecordLoader(bulkloader.Loader): def __init__(self): bulkloader.Loader.__init__(self, 'Record', [('variable', str), ('var_type', str), ('region', str), ('year', int), ('month', int), ('quantity', float)]) def handle_entity(self, entity): kname = ''.join([entity.region, entity.variable, str(entity.year), str(entity.month)]) if Record.get_by_key_name(kname): return cct.support_classes() # needed classes exist in db q = entity.quantity dt = datetime.datetime(entity.year, entity.month, 1) label = Label.all().filter('name = ', 'cct').get() variable = cctVariable.all().filter('short_name = ', entity.variable).get() region = cctRegion.all().filter('short_name = ', entity.region).get() # reset the entity here to avoid # unneeded attributes, and set keyname entity = Record(key_name=kname) entity.label = label.key() entity.variable = variable.key() entity.region = region.key() entity.time = dt entity.quantity = q return entity loaders = [RecordLoader]
blog comments powered by Disqus