Blog of a freelance software developer

A developers approach to documentation

From Kees van den Broek. May 22, 2010, 1 Comment

The traditional approach to documentation in a technical company is setting up a wiki on the intranet.

In my experience, it’s not a good enough solution, because of these issues:

1. A pain to maintain properly, i.e. extra ACL administration.
2. A wiki comes with a default markup language, and it needs to appeal to all who work with it.
3. If it ain’t time efficient for programmers, they’ll not use it.
4. Content should be easy to migrate when a wiki sytem is no longer maintained or fitting the organisation.
5. Attachments (i.e. flowchart images) are rarely in sync with reality, it’s too much effort to update.

Let’s take a step back and look at the fundamental question: where should documentation go?

If possible, ‘documentation’ should be the code itself. Whenever there’s more to explain, it should be in comments around the actual code. Meta-documentation for a specific project should go in the projects README (i.e. setting up the test environment).

Documentation on setting up an IDE that’s applicable to multiple projects should go in a more generic place outside any specific project.
‘Getting started’ documentation for new interns (codingstyle, traps to avoid, etc.) also needs a place.

There is a ’simple’ solution to the problem, although it requires a little bit of scripting. The benefits:

1. Integrates well into the workflow of programmers
   - put project-related documentation close to the actual code.
   - use any editor, any markup language. (example script supports only markdown)
   - documentation is in textfiles, in your regular VCS. (in our case Subversion)
   - create a link to a specific source file or diagram by using a VCS relative path. No more uploading of binaries to the wiki.
2. No extra system administration needed
   - ACLs and older revisions are provided by the VCS

This is the ‘architecture’ overview, compared to a traditional wiki. Two very small shell scripts is all there is to it.

This is how the ‘Getting started with Android’ page for new interns looks:

Scripts can be downloaded below.

What’s your ideal solution?

(more…)

Tokens and Buckets in the App Engine Task Queue

From Kees van den Broek. April 16, 2010, No Comments

Here’s my understanding of the Task Queue processes on the Google App Engine. A summary, so I won’t forget.

According to the documentation, there are 2 settings to a task queue: rate and bucket_size.

On the left side of the picture, the ‘token generator’ adds tokens to the bucket a a certain rate. For example 2 per minute to slowly fill up the bucket, or 5 per second to speed it up. The generator stops adding tokens to the bucket when the bucket is full (defined by bucket_size).

On the right side, there’s the Task Queue, holding tasks the user enqueued from the App.

In the middle, another process kicks in when there’s at least one token in the bucket and at least one task waiting in the Task Queue. If that’s the case, one token is taken out of the bucket (and thrown away), and one task is taken out of the Task Queue for immediate processing. If the task ended successfully, the task will be thrown away as well, and we’re done. If the task failed to run, it gets re-added to the Task Queue for trying again later.

When there are at least 2 tokens in the bucket and 2 tasks in the Task Queue, they process in the middle may run in parallel.

Mail uncaught errors with Google App Engine

From Kees van den Broek. February 28, 2010, 1 Comment

A simple script to start off any new GAE project.
All exceptions raised in production will show up in the GAE Logs dashboard, but I prefer to be notified by email.
To prevent email flooding, the same error will only be reported once a day.

class SomeHandler(EmailRequestHandler):
    def get(self):
        # Any unhandled exception here will be mailed.

(more…)