Subversion - Roundup Integration

Author: Richard Jones
Date: August 27, 2007

Contents

Overview

The integration takes the form of two components:

  1. Communicating from Subversion (messages to Roundup)
  2. Data handling in Roundup to support references to Subversion

The work producing this document and its supporting files was sponsored by Chris Ryland @emsoftware.com for the purposes of integrating Roundup and Subversion. The project description for that work is available as svn-project.html. A change history also appears at the end of that document.

Communicating from Subversion

Information needs to be passed to Roundup from Subversion when a commit takes place. This will be handled by the hook script notify-roundup.py which uses information embedded in change log messages.

Configuring Roundup

You will need to configure Roundup using either the svn-tracker template (see provided files) or by following the instructions in the data handling in roundup section.

Once you've done that you need to add a repository to your tracker's svn_repo class. Use the admin web interface or roundup-admin command-line to do this.

Setting up the hook script

The notify-roundup.py script is implemented as a hook script. It is intended to be added to the post-commit script in your Subversion repository's "hooks" directory. For example:

PYTHON=/usr/bin/python
NOTIFY=/home/richard/src/roundup/svn-roundup/notify-roundup.py
CONFIG=/home/richard/src/roundup/svn-roundup/notify-roundup.ini
"$PYTHON" "$NOTIFY" "$CONFIG" "$REPOS" "$REV"

The script implements both a "local" and "email" mode of operation, controlled by its configuration file (see notify-roundup.ini for a sample configuration). The configuration variables are:

main :: mode -- email or local.

Indicate which mode of operation notify-roundup.py will use. You may notify either a local or emailed tracker. Examples:

mode = email
mode = local
main :: item-class -- a valid tracker Class name

Change this to detect other issue types. Multiple issue classes are possible (use regular expression "either" syntax). Examples:

item-class = issue
item-class = system
item-class = dev|system|network
main :: host -- a valid hostname

Only set this if socket.gethostname() doesn't return the host's name as registered with your tracker. Examples:

host = host.name.example
host = svn.python.org
local :: tracker-home -- your tracker home directory

If notifying a local tracker, configure this variable. Example:

tracker-home = /path/to/your/tracker-home
email :: smtp-host -- SMTP host name

If notifying a tracker by email, set this to the SMTP host you will use to send the mail to the tracker. Example:

smtp-host = smtp-host.example
email :: tracker-address -- your tracker's email address

Set this to the email address your tracker listens to for incoming email. Example:

tracker-address = issues@host.example
email :: default-domain -- the default author email domain

"email-domain" is used in conjuntion with the Subversion author name or the address mappings below. Example:

default-domain = @host.example
address mappings -- one line per mapping

Use this to map Subversion author names to email addresses that the tracker will recognise. The "email :: default-domain" var will be appended if the address doesn't specify a domain. Examples:

richard = rjones
richard = ni@spam.example

If no mapping is defined for a particular author, we either:

  1. use the "<Subversion author name>@<default-domain>" address or,
  2. if a "*" entry is defined under address mappings, then we use that address as the from address.

Example:

* = subversion@host.example

So for all Subversion author names that don't specify an address mapping, the email from Subversion will appear to have been sent from "subversion@host.example".

Adding information to log messages

The trigger for talking to Roundup will be a token in the change log message formatted on a new line as:

<issue designator> [<new status>]

for example:

issue3
issue123 resolved
issue12 done-cbb

Data Handling In Roundup

Note

This section assumes you've not used the svn-tracker template and need to modify an existing tracker.

Information needs to be handled, stored and presented to the user. A single Link to a Subversion revision will appear on each message, which is obviously optional. It is automatically set by the Subversion hook script notify-roundup.py

We will want to formally store Subversion revision numbers relevant to an issue. These will then be used to format links from the issue to a Subversion web interface which may view information about the revision.

The specific classic tracker changes are:

  1. Adding a new class, "svn_repo", which will have four properties:

    svn_repo = Class(db, "svn_repo",
        name=String(),
        host=String(),
        path=String(),
        viewcvs_url=String())
    svn_repo.setkey('name')
    

    "name" is a symbolic name to identify the repository

    "host" is the name of the host holding the repository

    "path" is the filesystem path on the host machine

    "viewcvs_url" is the url which may be used to view a revision in the repository - it should have "%(revision)d" embedded. Optional.

  2. Adding a new class, "svn_rev", which will have two properties:

    svn_rev = Class(db, "svn_rev",
        repository=Link('svn_repo'),
        revision=Number())
    

    "revision" is the revision supplied by the checkin script.

  3. Make sure your relevant roles have permission to create 'svn_rev', and view 'svn_repo', e.g. (in schema.py):

    for role in ('User',):
        db.security.addPermissionToRole(role, 'Create', 'svn_rev')
        db.security.addPermissionToRole(role, 'View', 'svn_repo')
    
  4. Adding a Link on the "msg" Class to the "svn" class:

    msg = FileClass(db, "msg",
                    author=Link("user", do_journal='no'),
                    recipients=Multilink("user", do_journal='no'),
                    date=Date(),
                    summary=String(),
                    files=Multilink("file"),
                    messageid=String(),
                    inreplyto=String(),
                    revision=Link("svn_rev"))
    
  5. Adding a property "svn_name" mapping the Subversion author names to Roundup users:

    user = Class(db, "user",
                    username=String(),
                    password=Password(),
                    address=String(),
                    realname=String(),
                    phone=String(),
                    organisation=String(),
                    alternate_addresses=String(),
                    queries=Multilink('query'),
                    roles=String(),     # comma-separated string of Role names
                    timezone=String(),
                    svn_name=String())
    user.setkey("username")
    

    We will attempt to look up the "svn_name" first, and if there's no match we'll attempt "username".

  6. Modifying the standard issue.item.html page such that the message listing displays the Subversion Link information if present. You can either use the external web svn view link generation:

    <tr tal:condition="msg/revision">
     <th tal:define="r msg/revision" colspan="4">
      <a tal:condition="r/repository/viewcvs_url"
         tal:attributes="href python:str(r['repository']['viewcvs_url'])%r"
         tal:content="python:'Subversion revision %d'%r['revision']" />
      <span tal:condition="not:r/repository/viewcvs_url"
            tal:content="python:'Subversion revision %d'%r['revision']" />
     </th>
    </tr>
    

    or an internal tracker page (svn_rev.item.html) for displaying the revision information:

    <tr tal:condition="msg/revision">
     <th tal:define="r msg/revision" colspan="4">
      <a tal:attributes="href string:svn_rev${r/id}"
         tal:content="python:'Subversion revision %d'%r['revision']" />
     </th>
    </tr>
    

    A good place for this table row is between the standard message information row and the message content row.

  7. Modifying the standard user.item.html page such that the Subversion login details for users may be edited:

    <tr>
     <th i18n:translate="">Subversion login</th>
     <td tal:content="structure context/svn_name/field">svn_name</td>
    </tr>
    
  8. If using the "email" mode of notify-roundup.py, you will want to install the svnauditor.py detector module in your tracker's "detectors" directory.

Provided files

svn-tracker-1.5.zip
Ready to go template to create a Subversion-linked Roundup tracker. Run roundup-admin install in the directory created by the zip file.
notify-roundup.py
Subversion hook script. Is placed in the repository's "hooks" directory.
notify-roundup.ini
Sample configuration file for notify-roundup.py. May be placed anywhere, as the location of this file is the first argument to the notify-roundup.py script.
svnauditor.py
Auditor handling incoming notification messages. Goes in the tracker "detectors" director, and is only required if the email mode of notify-roundup.py is used.
svn_rev.item.html
Tracker HTML template allowing display of revision information. Goes in the tracker "html" directory.
revision_info.py
Support code for the revision information display - goes in the tracker "extensions" directory.

Notes

Installation Requirements

You will need to have installed the Subversion modules for Python from the Subversion project (not the modules from the "pysvn" project).

Building Subversion on OS X

When building on OS X, I needed to use the following configure line:

GXX=yes ./configure --without-apxs --without-berkeley-db --with-zlib \
  --enable-shared=yes --with-ssl --with-swig=/usr/local/swig/bin/swig

And then:

echo "/usr/local/lib/svn-python" > /Library/Frameworks/Python.framework/Versions/2.3/lib/python2.3/site-packages/svn_python.pth

(note that your swig location may vary)

Building the Python Subversion bindings

When building Subversion, you will need to configure && make && make install and make swig-py && make install-swig-py. You will then need to add /usr/local/lib/svn-python/ to your PYTHONPATH (using a .pth file in site-packages). subversion/bindings/swig/INSTALL in the Subversion source has notes.

Testing the script

Ensure that the REPOS argument to the script (ie. the repository dir) does not have a trailing '/' or you'll get an error like:

subversion/libsvn_subr/path.c:113: failed assertion `is_canonical (base, blen)'

(useful, that)