Richard's stuff Jul 2003

Wed, 30 Jul 2003

The Python Package Index was hiding Mac OS X packages (amongst some other more obscure sub-categories) in the browsing mode. It's been fixed now :)

path: /python | permanent link |

Wed, 23 Jul 2003

It was TRUE?!?

Violating national security for petty fun...

I really hope things don't get that bad here... I don't see Howard's office as being this petty though, so maybe our intelligence agencies can feel a little more safe than their collegues in the US...

path: /stuff | permanent link |

Mon, 21 Jul 2003

PyPI has had some much-needed attention

The Python Package Index (PyPI) uses the very cool sqlite engine to store the database of package information. Sqlite is cool because it's so simple to use and self-contained. Unfortunately, it's not a multi-user database. This means it locks the database when anyone accesses it. This caused PyPI some problems because ... well, PyPI is much more popular than I'd anticipated :)

After some brief analysis, I found:

The RSS feed gets hit about every 30 seconds or so (on average)
Some other PyPI page is hit at around the same frequency
About every third of those other hits is to the browse code, and the browse code was slow - taking up to 30 seconds to complete a request

Of course, this is all using averages, so during times of peak requests (ie. lunchtime in the US ;) then the rates are higher. And the combination of many requests and slow code result in users seeing "sorry, the database is locked".

To remedy this, I've:

Cached the RSS feed, so it only rarely has to hit the database
Significantly improved the speed (and accuracy while I was at it) of the browsing code

So hopefully things will run much smoother. Please, go kick the tyres and let me know if I've broken anything :)

path: /python | permanent link |

Fri, 18 Jul 2003

New version of pytaskplan, 1.1.0

Tasks are now allocated to people based on when they will finish the task, not when they will start it, resulting in much better use of people's time (no more gaps when people go on leave :)

Latest version over here.

path: /python | permanent link |

Mon, 14 Jul 2003

Melbourne Zope3 Sprint, 2003 ... all done

Wow. That was my first Sprint and I have to say it was a lot of fun :)

Jan (without whom I doubt the Sprint would ever have happened) has put up a summary of what the various sprinters did. There's photos of the pairs too. I'm the only one who decided to not look like he was working, but rather to grin and wave like a loon. The second photo from the bottom shows me actually deep in thought, proving that the first photo has not captured me in my natural state :)

path: /python | permanent link |

Mon, 07 Jul 2003

Listing TODO items in your source

This afternoon is TODO time (since my main job of the day - generating nice little help tooltips - was solved by the very cool overlib.) So, in usual fashion I avoid the job for a while and write a script to help me. The following is the result, finding all TODO items in source trees and printing them out with some context:

#! /usr/bin/env python2.3

import sys, os

def walker(args, path, filenames):
    for filename in filenames:
        joined = os.path.join(path, filename)
        if os.path.isdir(joined):
            continue

        # find lines to display, at least 2 either side of a TODO marker
        lines = open(joined).readlines()
        display = {}
        for i, line in enumerate(lines):
            if 'TODO' not in line:
                continue
            for n in range(i-2, i+3):
                display[n] = 1
        if not display:
            continue

        # organise the lines to display
        display = [n for n in display.keys() if n >= 0 and n < len(lines)]
        display.sort()

        # print them
        print '**', joined
        last = None
        for n in display:
            if last is not None and n > last+1:
                print '---'
            last = n
            sys.stdout.write(lines[n])
        print '*'*75

if __name__=='__main__':
    for arg in sys.argv[1:]:
        os.path.walk(arg, walker, None)

Yes, I took the time to have a look at some new python 2.3 features too (see if you can spot them :)

path: /python | permanent link |