Ghosts of Frameworks Past

By anders pearson 25 Nov 2017

Recently, Github added the ability to archive repositories. That prompted me to dig up some code that I wrote long ago. Stuff that really shouldn’t be used anymore but that I’m still proud of. In general, this got me reminiscing about old projects and I thought I’d take a moment to talk about a couple of them here, which are now archived on Github. Both are web frameworks that I wrote in Python and represent some key points in my personal programming history.

I started writing Python in about 2003, after spending the previous five years or so working mostly in Perl. Perl was an important language for me and served me well, but around that point in time, it felt like every interesting new project I saw was written in Python. I started using Python on some non web-based applications and immediately liked it.

Back then, many smaller dynamic sites on the web were still using CGI. Big sites used Java but it was heavy-weight and slow to develop in. MS shops used ASP. PHP was starting to gain some popularity and there were a ton of other options like Cold Fusion that had their own niches. Perl CGI scripts running on shared Linux hosts were still super popular. With Perl, if you needed a little more performance than CGI scripts offered, if you ran your own Apache instance, you could install mod_perl and see a pretty nice boost (along with some other benefits). Once you outgrew simple guestbooks and form submission CGI scripts, you needed a bit more structure to your app. Perl had a number of templating libraries, rudimentary ORMs, and routing libraries. Everyone generally picked their favorites and put together a basic framework. Personally, I used CGI::Application, Class::DBI, and HTML::Template. It was nowhere near as nice as frameworks that would come later like Ruby on Rails or Django, but at the time, it felt pretty slick. I could develop quickly and keep the code pretty well structured with cleanly separated models, views, and templates.

Python wasn’t really big in the web world yet. There was Zope, which had actually been around for quite a while and had proven itself to be quite capable. Zope was… different though. Like, alien technology different. It included an object database and basically took a completely different approach to solving pretty much every problem. Later, I would spend quite a bit of time with Zope and Plone, and I have a great deal of respect for it, but as a newcomer to Python, it was about ten steps too far.

So, in the meantime, I did the really predictable programmer thing and just ported the stuff I missed from Perl to Python so I could write applications the same way, but in a language that I had grown to prefer.

Someone else had already made a port of HTML::Template, htmltmpl. It worked, but I did have to make some fixes to get it working the way I liked.

Similarly, Ian Bicking had written SQLObject, which was a very capable ORM for the time.

My major undertaking was cgi_app, which was a port of CGI::Application, routing HTTP requests to methods on a Python object, handling the details of rendering templates, and providing a reasonable interface to HTTP parameters and headers.

It looks pretty basic compared to anything modern, and there are some clear PEP8 violations, but I remember it actually being pretty straightforward and productive to work with. I could write something like

#!/usr/bin/python

from cgi_app import CGI_Application

class Example_App(CGI_Application):
    def setup(self):
        self.start_mode = 'welcome'

    def welcome(self):
        return self.template("welcome.tmpl",{})

    def param_example_form(self):
        return self.template("param_example_form.tmpl",{})

    def param_example(self):
        name = self.param('name')
        return self.template("param_example.tmpl",{'name' : name})

if __name__ == "__main__":
    e = Example_App()
    e.run()

drop that file in a cgi-app directory along with some templates and have a working app. It was good enough that I could use it for work and I built more than a few sites with it (including at least one version of this site).

I quickly added support for other template libraries and mod_python, the Python equivalent to mod_perl.

I don’t remember ever really publicising it beyond an announcement here and I didn’t think anyone else was really using it. Shockingly recently though, I got an email offering me some consulting work for a company that had built their product on it, were still using it, and wanted to make some changes. That was a weird combination of pride and horror…

By around 2006 or 2007, the landscape had changed dramatically. Ruby on Rails 1.0 was released in 2005 and turned everything on its head. Other Python developers like me had their own little frameworks and were writing code but there wasn’t really any consensus. Django, TurboGears, web.py, and others all appeared and had their fans, but none were immediately superior to the others and developer resources and mindshare were split between them. In the meantime, RoR was getting all the press and developer attention. It was a period of frustration for the Python community.

Out of that confusion and frustration came something pretty neat though. A few years earlier, PJE had come up with WSGI, the Web Server Gateway Interface, but it hadn’t really taken off. WSGI was a very simple standard for allowing different web servers and applications to all communicate through a common interface. At some point the Python community figured out that standardizing on WSGI would allow for better integration, more portability, simpler deployment, and allowed for chaining “middleware” components together in clever ways. Most of the competing frameworks quickly added WSGI support. Django took a little longer but eventually came around. The idea was so good that it eventually was adopted by other languages (eg, Rack for Ruby).

JSON had mostly pushed out XML for data serialization and web developers were beginning to appreciate RESTful architecture. I began thinking about my own applications in terms of small, extremely focused REST components that could be stitched together into a full application. I called this approach “microapps” and was fairly evangelical about it, even buying ‘microapps.org’ and setting it up as a site full of resources (I eventually let that domain lapse and it was bought by spammers so don’t try going there now). Other smart folks like Ian Bicking were thinking in similar ways it seemed like we were really onto something. Maybe we were. I don’t really want to take any credit for the whole “microservices” thing that’s popular now, but I do feel like I see echoes of the same conversations happening again (and we were just rehashing SOA and its predecessors; everything comes back around).

Eventually, after writing quite a few of these (mostly in TurboGears), almost always backed by a fairly simple model in a relational database, I began to tire of implementing the same sort of glue logic, mapping HTTP verbs to basic CRUD operations in the database. So I wrote a little framework. This was Bourbon (WSGI can be pronounced “whiskey”).

Bourbon let me essentially define a single database table (using SQLALchemy, which was the new cool Python ORM) and expose a REST+JSON endpoint for it (GET to SELECT, POST or PUT to INSERT, PUT to UPDATE, and DELETE to DELETE) with absolutely minimal boilerplate. There were hooks where you could add additional functionality, but you got all of that pretty much out of the box by defining your schema and URL patterns.

For a number of reasons, I mostly switched to Django shortly after that, so I never went too far with it, but it was simple and clear and super useful for prototyping.

In recent years, I’ve built a few apps using Django Rest Framework and it has a similar feel (just way more complete).

I don’t have a real point here. I just felt like reminiscing about some old code. I’ve managed to resist writing a web framework for the last decade, so at least I’m improving.

Tags: perl python rest microapps frameworks