Richard Jones' Log

Thu, 15 Jan 2009
Sane Python application packaging: conclusion (for now?)

This is part four of my continuing adventure in packaging my Python application.

In part three I was close to a solution. In the end I have decided to:

  1. Use the "build_apps" distutils command I wrote to generate zip application bundles for the three target platforms. The zip file contains all libraries (pure python, thankfully) and a launcher script to run the application.
  2. Split out the examples from the rest so there's just one examples zip file and the other zip files are about half as heavy.
  3. Generate a standard distutils source distribution which is intended for Linux system package maintainers.
  4. Upload all those files to Google Code using a modified googlecode_upload.py script.
  5. Still register with PyPI but set the download_url to point to Google Code.

The upload script was modified in two ways:

  1. First, it's broken with the svn module available under Ubuntu (and others). I modified the code to directly parse the svn passwords. It's a gross hack and probably not widely useful so I won't submit a patch. For the curious the code is below.
  2. Secondly I hard-coded it to upload the five files I want to send, just to make my life easier.

I'll await feedback from actual users but I believe that this solution will be good enough.

By the way, I just released a new version of the application in question. It's Bruce, the Presentation Tool (who puts reStructuredText in your projector). The 3.2 release is pretty nice with support for external styles, recording, automated playback of various types, gradual exposure of lists and some other stuff. It's pretty cool.

On to the change to googlecode_upload.py, which replaces get_svn_auth() with:

def get_svn_auth(project_name, config_dir):
  """Return (username, password) for project_name in config_dir."""
  realm = (' Google Code Subversion Repository'
           % project_name)
  authdir = os.path.join(config_dir, 'auth', 'svn.simple')
  for fname in os.listdir(authdir):
      info = {}
      key = None
      for line in open(os.path.join(authdir, fname)):
          line = line.strip()
          if line[0] in 'KV':
              continue
          if line == 'END':
              break
          if line in 'password username svn:realmstring'.split():
              key = line
          else:
              info[key] = line
      if info['svn:realmstring'] == realm:
          return (info['username'], info['password'])
  return (None, None)
Fri, 02 Jan 2009
Sane Python application packaging: initial solution

This is part three of my continuing adventure in packaging my Python application.

Thanks again for the advice given in response to my last post on Linux-specific application packaging issues. I've decided to leave packaging up to the experts and help them as much as possible. To this end I've:

  1. split the installation instructions out of README.txt into INSTALL.txt and put in a notice that the INSTALL.txt is not for regular users,
  2. looked into how to best support more organised system package maintainers (the INSTALL.txt is my start at that), and
  3. generated stand-alone application downloads for Linux, Windows and OS X.

The approach I've taken for the stand-alone packages is as follows:

  1. bundle all of bruce, pyglet, cocos, docutils (including roman.py) and pygments into a library zip file,
  2. create a simple script for each O/S to run bruce using that zip file, and
  3. create the distribution zip files with those contents alongside the README.txt, HOWTO.txt and PKG-INFO.

Since the examples directory is pretty big I've also created a separate "examples" zip file.

This is implemented as a new setup.py command "build_apps" (as BuildApps.) I didn't use zipfile's PyZipFile but rather implemented my own. PyZipFile only includes compiled modules (.pyc or .pyo). The problem I have with this is that the compiled modules are not compatible across Python minor releases (modules compiled for Python 2.5 are not compatible with 2.6 and vice versa.) I would have to include separate library zip files for each system interpreter I wished to support. After some very simple checks I determined that the performance difference is negligible if I just bundle the original .py files instead.

I wrote "build_apps" so it'd fake being an "sdist" command enabling me to upload the resultant file to the Python Package Index (PyPI). This part is really quite inelegant but I'd like to keep the downloads all in one place. Unfortunately the file listing in PyPI lists the files as "Source" which isn't good. PyPI has no concept of an "application" though.

The Windows and OS X application bundles are completely untested.

I intend to investigate py2app and py2exe in the longer term to produce more system-friendly programs (no need to install Python, have icons, etc) but for now I believe this solution is good enough.