Topp Engineering

Last week I attended the Chicago Google App Engine Hack-A-Thon, which was a small day-long event to hang out at the Chicago Google offices and work on things related to App Engine.

I was hoping for something structured more like a coding sprint, where people would work together on small projects. I think some other people were also expecting something like this, but the day kind of wandered by without any clear “let’s start doing stuff together” moment, and the opportunity was lost. Some people followed a tutorial given by Marzia Niccolai, some worked on their own projects, and there was a bit of chatting.

I started working on a simple project I’d started to make a very (very) simple CMS/Wiki. After playing with Javascript for a little while I was chatting about wikis and MoinMoin came up. After a quick look at the code I realized it was very tightly bound to the filesystem, and lacked even the most basic abstraction layer for storage. MoinMoin should have an abstraction layer, but I wasn’t really inclined to work on that. Instead I thought: why not try a fake writable filesystem?

I knew from the start that the idea was a bit absurd. The App Engine data store isn’t much like a filesystem. Files are quick to read and write, okay for scanning, hard to query. The data store is okay to read and write, very slow to scan, and easy to query. But I figured there was some hack value to it.

I don’t know that I really accomplished my goal, but I did get some code in appengine-monkey. It kind of worked, but my strategy was probably wrong: I simulated an entire filesystem, except those places that were mingled with code (where templates would typically be stored). Instead I should have required explicit locations that would be handled by the data store (e.g., /wiki-data). Python doesn’t expect file operations to lead to a lot of Python routines, and there were some circular situations deep in the Python code as a result. I seemed to mostly work those out, but didn’t actually get MoinMoin to render anything, I only got it to work very hard on the slow process of setting up its files.

I’ve been a little annoyed with the App Engine environment from the start, because it left out lots of routines that are present in every other environment (e.g., os.utime). You can’t use these routines in App Engine because there is no file writing. But it would all be more sensible if every call to these routines just raised a permission error. This is the kind of error that existing code understands. Instead you can’t even import the routines, making App Engine incompatible with lots of existing code. That would be fine except the incompatibility is so trivial. appengine-monkey seeks to relieve some of this, but it would be much simpler if it was just there in the platform to start with.

­Conclusion? Porting code is hard, but porting old, organically developed code like MoinMoin is really hard. And App Engine still needs a good wiki.