Richard Jones' Log

Here's another nice couple of photos stolen from Ping and Jeremy - I hope they don't mind.

I swear that Barry wasn't threatening me. Not that I recall anyway.

A number of us (including myself) think that PyPI needs a better name. There's a couple of reasons for this:

1. PyPy. Oh, the confusion during the sprints. Yes, PyPI was first, but that hardly matters. Pronunciation isn't going to solve this one.
2. "Index" doesn't really capture the entire nature of what we've got now, as we now include an archive.

I don't like PyPAN or anything similar. I'd actually prefer to go with something Monty-Python-esque which isn't an acronym. Someone just suggested "spam" which would be so cool. Perhaps spam's been around for long enough that we can try to reclaim it? Probably not. Other suggestions are "shrub", "shrubbery", "lumberjack", "wafer", "creosote", "grail", "holy grail", "brian", ...

So, suggestions?

Well, I finally emerged from the airport to see Rachel and Abbey at 9am Melbourne time yesterday. The trip is really quite arduous but unexciting. The only excitement came right at the start when there was a security scare in the middle concourse (the important one for me) at the D.C. National airport. Closed it down for about half an hour, which meant lines of passengers reaching to either end of the terminal (an impressive sight very poorly captured by my mobile camera, so I won't post the image). It cleared up though, and my flight wasn't delayed, which was an enormous relief (I only had about 30 minutes to change planes in Texas, but the 5 hour change in LA was a little more comforting).

I'm going to be doing all I can to make it next year. Hopefully I get stupidly rich and can afford business class. I'm not holding my breath though.

The last day of the conference was spent winding down. I went to Greg Stein's keynote, which other people blogged about (Google should use Roundup :). I was to follow that up with Fred Drake's XML presentation, but there were technical difficulties so I went to the registration desk early. I then spent then next 2 hours or so manning the desk, mostly talking to people about random things (Greg about Google, Grigory about mod_python, various people about GPG keysigning and some other less-well-remember conversations).

I made it to the last half of the lightning talks for the day, which included some fun stuff. Of note:

1. The "http://www.python.org/pypi/[package name]/[version]" URLs we generate in PyPI are being indexed already, and they're ranking really high. Kudos to Martin v L�wis for implementing that.
2. I need to talk to Daniel Chudnov about OpenURL and related topics. I think this could be a very close conversation for Common Ground to have with him.

I then kinda drifted around, caught Steve Holden's closing presentation which included some amazing limerics. I didn't copy down the best one, as I was enjoying it too much :) I'm sure others will have copies of it online. It's a shame that no-one at the conference appeared to know that the call-for-limerics was on.

We had an attempt at a pydotorg BOF (Sean Reifschneider, Ping, AMK, Barry, Grigory and myself, IIRC). I've decided to look into a PyLucene (good noises about it came from a number of presentations) hook into Roundup, which should help some things. Barry and I have some work to do, it seems :) At the end, I mentioned I would've liked to have a boardgame session or two during the conference, and just about everyone spoke up and said that'd be a great idea. Sigh. Next time I'll be more organised, I'm sure. Sean did mention a game I'd not heard of, Metro, which is apparently good for two players.

Finally, the conference wound down. I met a number of people I'd not met during the conference, including Moshe Zadka. We talked very briefly about favourite shows, and I need to check out Veronica Mars. Available *cough* online *cough*.

Then was the final dinner with AMK, his better half Barb, Andrew Dalke, Mike Fletcher (those were the people sitting around me) and about 20 other people :) The discussion centered mostly around sci-fi novels to read (old sf books are tradeable on sf-books.com, if I could bear to part with them) and graphic novels.

I then said goodbye to people I really, really want to see again next year. I guess I'd better start saving...

On the other hand, I get to go home tomorrow (Saturday - I actually arrive home on Monday) and see Rachel and Abbey again. Yay!

Wow, what a day. I got a decent night's sleep, including a bit of a sleep-in (alarm didn't go off) so I was alert for the whole day. First up was Guido's state of Python, which was interesting. Nothing really new (except that he's a little hesitant about static typing - though that might not be new for some). After that was a mad rush for the doors as the PSF Q&A started - a shame really, as there's a bit of a problem with the PSF's image.

I spent the next talk mostly just tweaking my alternative presentation. I had a well-rehearsed 20 minute Introduction to Roundup presentation already written, but I'd gotten the impression that I should have an alternative one with more emphasis on Roundup customisation.

Next up was a talk about profiling and visualising Python execution. The Python "top" tool was very cool, and I want to give it a whirl some time. The graphical visualisation of the Python core was also neat, though I'm not sure I've any immediate uses for it.

Then it's my turn. A quick poll of the audience before I started showed the opposite though, so I went with the intro one. It was well-received, and the questions afterward were quite good. Someone tried to push an advert for Trac, which I thought was a little rude, but he came up afterwards and apologised. Maybe I need to write that trivial bit of code that does "CVS integration" some day.

During lunch, Barry Warsaw and I talked a bit about possible cross-pollentation of Roundup, SQLObject and Mailman. I think we clarified that SQLObject and Roundup's Hyperdatabase occupy different spaces. The mail spooling aspect of mailman is definitely something I'd be interested in for Roundup though.

After lunch were lightning talks, which were a total blast. Ping presented a couple of really nice tools. One was his older Zesty idea but with added flavour (really hard to describe, so you'll have to wait for Ping to put info up on a website). The other was a very cool tool for poking around HTML pages. It has a beautiful API for loading up and poking into the pages, and the neat thing is that it parses as it needs to. So if you ask for a list of anchor tags - it uses a regexp to locate just the anchor tags and their contents. The result is a sequence of objects that you can then ask other queries on, like what their containing tags are. Since it's parsing only when it needs to, the HTML can be totally broken, and we just don't care. BeautifulSoup might parse broken HTML well, but this tool has a significantly better API.

I gave two talks. The first was me running a "python2.3 setup.py bdist_wininst upload --sign" in a shell. Unfortunately, a rare bug was tickled and it broke. It was quickly fixed though, and I managed to get appropriate applause from the audience when I showed the uploaded file online. My second talk was a rehash of the LD48 talk I gave at the December OSDC in Melbourne. It was also well-received, with due laughter at my apalling graphical skills. Unfortunately the appropriate URLs dropped off the end of the presentation :(.

One of the last things to be shown was a collaborative Python interpreter called SHPY. Kinda like SubEthaEdit, but without the Rendezvous and with code execution. In a PyGame window. Quite cool. Quite wierd.

After all that I went off to learn about metaclasses, descriptors and decorators. I think they make sense now, but I still have to use them. I did immediately start thinking about how Zope 2 could be cleaned up a fair bit with the use of metaclasses and decorators.

A bunch of us then got together and signed GPG keys. Quite useful.

Then there was the PSF member's meeting, and finally we adjourned for dinner. Which I just got back from. Sleep now, in preparation for the final day...

Wow, what a day. I started the day off tired as I only slept about two hours last night (I guess the jetlag finally overcame the adrenaline). I managed to stay awake though, and got to meet a whole lot more people face-to-face. It's all a bit of a blur, but one of the highlights was meeting Ping, even though I was a bit addled and didn't really make much sense ;)

Highlight talks included some of the "web" track including the PyWebOff and dynamic web GUI talks. It was good to hear the Zope roadmap in Jim's presentation too - it cleared up a lot about where Zope 2, 3 and the ZODB are going. Holger's talk about the "py" library was enlightening, though I missed the start of the py.test section. By the afternoon my brain was just too fried for going to the serious OO programming talks, so I just kinda drifted around talking to people.

I had an open-space discussion with some Roundup users (including Ping :). We discussed some of the warts, and one of the ones I've known about for a long time - templates are hard to edit and fall out of sync with schema changes - might have a solution.

A number of people stopped me to thank me for Roundup, which was nice. A few other people were interested in the PyPI development - though there's still a huge number (most likely a comfortable majority) of people at the conference who don't even know it exists. I've lined up another Lightning Talk entitled "PyPI: Python's CPAN" in which I'll give a demo. The five minutes is plenty of time to show the basic submission interfaces. Unfortunately, at one point someone (I forget who, it's all a blur) started asking questions about the python.org machine and services and I started to talk about the need for someone full-time to look after it and implement some of the services we'd all like -- and I think the discussion ended up going a little sour, so I apologise to whoever it was as I was not trying to be offensive, I was just plain exhausted.

PyPI has had its last feature update for the Sprints. New features:

1. ReStructuredText formatting of descriptions - eg. Roundup 0.8.2 (no existing descriptions have been processed - to force a reformatting, just edit the release description online).
2. Files may have accompanying OpenPGP signatures. These may be generated automatically using the "--sign" (or "-s") switch on the setup.py "upload" command. Note that the "upload" command requires Python CVS (to be Python 2.5). You can still submit files via the web interface.
3. The web interface now uses Zope Page Templates for the HTML and XML rendering. A whole lot of interface cleaning up was done during this process.
4. PyPI is much more sane in its handling of Unicode and UTF-8 (unfortunately, this means existing setup.py files with latin-1 fields will not submit - if they're changed to UTF-8 they should submit OK).
5. There's a nicer way to link to releases: http://www.python.org/pypi/roundup/0.8.2.
6. Selenium tests for the web interface.

A big thanks to my PyPI / distutils co-conspirators: Fred Drake, Sean Reifschneider, Martin v. L�wis, Mick Twomey, John Camara, Andy Harrington, Andrew Kuchling, David Goodger and Ian Bicking (with Barry Warsaw in a supporting role).

I'm not off for an afternoon at the Air & Space Museum (I hope).

Here's the flickr PyCon2005 tag page. Mostly just stuff from my camera phone at the moment. Others are running around with better quality cameras though.

Another good day of sprinting, mostly. Martin v. L�wis got the distutils upload command checked into Python 2.5 CVS. He's working on an OpenPGP signing facility now too. Yay Martin!

I met Mick Twomey today, who I know from Roundup, and he's a great guy. We worked on migrating the PyPI web interface to ZPT. The ChiPy people did some work on this, which we built on. Unfortunately, we couldn't find a ZopePageTemplates download (a recent one, that is) so we switched to SimpleTAL. Later in the day, realising that we were losing significant development time to chasing down errors that were hidden by SimpleTAL, we switched over to the Zope 3 zope.pagetemplates implementation. There were some teething problems which Mick was still working on when we finished up at 9:30. I spend the evening working on Unicode issues (with Mick's help). PyPI now handles unicode much better, and the web interface and database work only in utf-8.

David Goodger supplied us with docutils patches allowing disabling of the "raw" and "include" directives, allowing John Camara to work more on the ReST-ifying of the PyPI "description" fields. John also spent some time working on general bugs in the distutils code lodged in the sf.net tracker.

The package archive had two packages in it by the end of the day - my roundup upload and Fred Drake's ZConfig one. More will come soon, I suppose :)

Another successful day of sprinting today. I got the postgres switchover done first thing this morning - and anecdotally it's running better (but then python.org's load is about 1/10th of what it's been recently).

Martin von Loewis joined us and worked with Fred Drake on getting the PEP 314 (metadata 1.1) spec implemented (requires, provides and obsoletes predicates). I paired with Fred to work out the web interface for managing the fields. That all went live this afternoon too.

I spent the day implementing a package upload mechanism. I spent some time adding some paranoia checks on the uploaded files, with calls from around the room for ideas of things to check for. After Barry Warsaw tweaked some permissions for me, it all went live.

Yes, we now have a package archive on www.python.org! There's many of us who are very excited about this :)

Tomorrow I'll spend some more time working on the user interface side of things.

Day one of the PyCon sprints was a mixed bag for the distutils group. I had good success migrating PyPI over from sqlite to postgresql. I expect that I'll actually switch www.python.org over tomorrow and move on to file uploads.

AMK spent a frustrating day trying to get a better web application environment going, in the end with no real improvement, but a possibility of one. John Camara spent most of his day trying to get a PyPI development environment going on his Windows machine. Fred Drake had a good ol' bash at the distutils code to introduce the PEP 314 metadata. Additionally to the requires, provides and obsoletes terms in the PEP, he'll be looking at adding requires-feature and provides-feature. These spell out more abstract concepts than the strictly-python-module-or-package-name other terms. They will allow "requires-feature: python (2.4)" for example. We had Andy Harrington and Darrell Gallion helping out Fred near the end of the day too.

ps. no photos except the one I've uploaded to flickr (tag: pycon) - my camera is flat and my charger doesn't work (silly me, I didn't check it - the powerbook charger works OK in 110V though, which is good ;) Hopefully I can get a charger that works ...

Well, I made it to PyCon, woohoo! I've had about 6 hours' sleep since I left home on Friday morning (aussie time).

Sitting here at the distutils sprint with AMK, Fred Drake and John Camara. We've got an abitious list of things we could get done (partly inspired by the recent ChiPy sprint), and I'm being good and bossy in organising them. Fred and John are going to knock over PEP 314 (Fred has CVS commit privs, making that easier). I'm going to migrate PyPI to postgresql and start the package submission work. AMK's looking at making www.python.org work better - assuming he can stop making pained noises about the wiki for long enough ;)

I've also met a bunch of people for the first time, which is way cool, and also part of the reason I'm here.

A number of us are running IRC clients in the irc.freenode.net channel #pycon.

Oh, and I've added a new Lightning Talk to clear up the confusion about the pronunciation of PyPI. It's a particular problem given that the PyPy people are in the same sprinting room as us. Yes, PyPI was named first, but it was always intended to be pronounced "pipi" with the short "ih" sound.

I finally finished all the editing of the presentation (cutting 10 minutes from the format I presented last December). Just rehearsed it and it ran for 20 minutes on the dot. The companion "paper" has been submitted too. I can now focus on the sprint with a clear conscience :)

I have a camera phone now ;)

I'm supposed to be finishing off my presentation and the accompanying "paper" for publication. Instead I've been fiddling with printing out the schedule for the conference ;)

The following marks the "printable" version of the Wiki table with some highlighting to make it a little easier to read:

import re
r = []
n = 0
spesh = re.compile('\s(break|lunch|Social Event|Member Meeting|TBA)\s', re.I)
for line in open('/tmp/Schedule.html').readlines():
 if line.startswith('<tr>'):
  if n%4 > 1: r.append('<tr class="odd">\n')
  else: r.append('<tr class="even">%s\n')
  n += 1
 else: r.append(line)
 if spesh.search(line): n += 1
open('/tmp/mod-schedule.html', 'w').writelines(r)

With the above wiki page saved as "/tmp/Schedule.html" with the following style info inserted near the top:

<style>
    td {vertical-align: top; }
    td > p { margin: 1px 2px 1px 2px; }
    tr.even td { background: #ddf; }
    tr.odd td { background: #eee; }
    td {border: none; }
    @media print {
        h1 { display: none }
        div > p {display: none}
    }
</style>

It generates "/tmp/mod-schedule.html" which I think is much clearer ;)

... until I get on the plane. Whee!

Having done a bunch more research, I've started to settle on the following currently-available components:

• Shuttle SN85G4 - small box, potentially quiet with a simple fan change, optical digital audio in/out, SATA drives. Good Linux support for all components.
• AMD Sempron 3100+ - quite capable, and cooler than the regular Athlon64s.
• AverTV DVB-T 771 - digital TV reception at a very reasonable price, and AverMedia have Linux installation instructions (Word doc).
• Western Digital Caviar 200Gb Serial ATA (no review) - as far as I can tell, it's the coolest, quietest available drive for under $1 per GB.
• Antec 80mm SmartCool fan - replaces standard fan in the Shuttle case; thermally-controlled to lower fan speed when cool enough.
• ATI RF Remote Wonder - seems to have good Linux support.
• Existing Matrox G400 MAX video card.
• ... and some other components like the DVD-RW for the Shuttle case and some memory.

Just ordered myself a new phone (old one was getting crappier and crappier, and new one can take photos). Guy signing me up asked if I'd like a new SIM sent out for the old phone. I don't. I countered by asking whether they offer a recycling service. He was quite surprised - he said he'd not been asked that question in 2 years. Yes, they do have a service. What does it involve? I hand my old phone over to the courier delivering the new one. That's it! I asked him why they don't mention the service. He actually sounded puzzled. Maybe he might ask someone. There's always hope.

I'm not sure we'll ever see the VIA EPIA SP motherboard for sale in Australia, so I'm starting to look into other options. I just priced up a Shuttle-based Athlon 64 system (after reading a nice HOWTO on the subject) and it came in at just over $1200. That's definitely in the running then.

According to Sumea:

"Attorney-General Rob Hulls said yesterday all states, territories and the Commonwealth had, for the first time, agreed to a uniform classification system for films and computer games.

Under the changes, computer games will now be rated G, PG, M, MA 15+ or RC (refused classification).

"Computer games will adopt the system used by films to make it easier for consumers, especially parents, to make informed choices about computer games and films their children are viewing," Mr Hulls said."

So where's the R rating? Or is that being dropped for film? Or have I just imagined the R rated films I've seen?

Ryan Tomayko compares the ZODB to RDBMSes, but I think he's got a few things wrong:

1. A ZEO server can serve up multiple databases. My Zope systems serve up a main and a sessions database (one of these days I'll find the right non-journalling backend to put in the sessions store too).
2. ZEO packing is trivial using the zeopack.py script (utilities/ZODBTools in the Zope source). As an aside, I also use repozo.py to back up my ZEO databases (same place in the source). I launch both from cron regularly.

I strongly agree with his overall point though - if you use the ZODB to store a large dataset that wants to be relational, or needs to be queried / accessed like a relational table, you'll find that your data is pretty much duplicated (or triplicated if index and meta-data) inside ZCatalogs to have any sort of speed. Just use an RDBMS in this instance. Well, unless your data schema is highly mutable (as ours is) in which case just accept that half your ZODB will be ZCatalogs.

Ryan sums up well when he says "making the decision between ZODB and relational has been less about what's better for the code and more about what's better for the data."