Richard Jones' Log: PyCon Sprints, day 3

Tue, 22 Mar 2005

Another good day of sprinting, mostly. Martin v. L�wis got the distutils upload command checked into Python 2.5 CVS. He's working on an OpenPGP signing facility now too. Yay Martin!

I met Mick Twomey today, who I know from Roundup, and he's a great guy. We worked on migrating the PyPI web interface to ZPT. The ChiPy people did some work on this, which we built on. Unfortunately, we couldn't find a ZopePageTemplates download (a recent one, that is) so we switched to SimpleTAL. Later in the day, realising that we were losing significant development time to chasing down errors that were hidden by SimpleTAL, we switched over to the Zope 3 zope.pagetemplates implementation. There were some teething problems which Mick was still working on when we finished up at 9:30. I spend the evening working on Unicode issues (with Mick's help). PyPI now handles unicode much better, and the web interface and database work only in utf-8.

David Goodger supplied us with docutils patches allowing disabling of the "raw" and "include" directives, allowing John Camara to work more on the ReST-ifying of the PyPI "description" fields. John also spent some time working on general bugs in the distutils code lodged in the sf.net tracker.

The package archive had two packages in it by the end of the day - my roundup upload and Fred Drake's ZConfig one. More will come soon, I suppose :)

Comment by Ian Bicking on Tue, 22 Mar 2005

Whoops -- I used the zpt.sf.net code, but the zope3 code is probably better. Hope that didn't cause too much trouble -- I was next door in the other room (but I wasn't on IRC)

Comment by Fredrik on Tue, 22 Mar 2005

Now that even things you post yourself are messed up ("Löwis"), perhaps it's time to fix the Unicode problems in pyblagg ;-)

Comment by Richard on Tue, 22 Mar 2005

Argh! Unicode! :)

Comment by Fredrik on Tue, 22 Mar 2005

Want me to take a look? Where's the code?

Comment by Richard on Tue, 22 Mar 2005

Sure, the code's in fetch.py and scrape_wiki.py.

You'll need sqlite and Mark Pilgrim's feedparser module.

Comment by Fredrik on Tue, 22 Mar 2005

And a script to initialize the database, perhaps?

Comment by Fredrik on Tue, 22 Mar 2005

Nevermind. Here's a minimal patch (this assumes that you're using 2.3 or newer):

Index: fetch.py
===================================================================
--- fetch.py    (revision 333)
+++ fetch.py    (working copy)
@@ -206,7 +206,7 @@
 </div>
 %(items)s
 </html></body>'''%contents
-    f.write(s.encode('utf-8'))
+    f.write(s.encode('ascii', 'xmlcharrefreplace'))
     f.close()

     # format RSS

Comment by Richard on Tue, 22 Mar 2005

Thanks, I've updated the code and am re-generating the page right now.