Mon, 03 Feb 2003
Python versions of WWW:Mechanize

Simon Willison wants a Python version of Perl's WWW::Mechanize. I believe I've already written it. It appears in the form of two projects I've used to test the eKit website:

  1. The more recent web performance tester called PyWebPerf. PyWebPerf is a performance measuring tool written in Python which simulates a web browser fetching a page (handles cookies, multiple threads, image and css download). Command-line and cgi interfaces are provided. I wrote this as a replacement for apachebench so I could have my testing behave more like a web browser.
  2. The earlier web unit testing framework.

The code (and API) is designed to let your script look like a regular web browser. Recently I used a slightly modified version of PyWebPerf to do the following:

    fetcher = WebFetcher()
    login = fetcher.fetch('https://www.xxxxxxxx.com/')
    response = login.postForm(0, login.POST, {'UserID': 'xxxxxxx',
        'Password': 'xxxxxxxx'})
    r2 = response.fetch('/account/msg_status.asp',
        {'FromDate': '1/Jan/2003', 'ToDate': '3/Feb/2003',
        'MsgStatus': 'Failed', 'NumRec': '20', 'GenerateCSV': 'blah'})
    r3 = r2.fetch('/account/preview/xxxxxxx.csv')
    print r3.body

Note that each successive fetch uses the previous fetch result object - thus carrying over any cookies sent back from the server. The WebFetcher code can be (and indeed has been in the above application) trivially changed to turn off the automatic image/css loading and timing marks.

Aside: yes, this is a real example of code I have to use to automatically get failure reports from one of our service providers. Yecch.

path: /python | permanent link |
No Indie movies on DVD for me...

Sadly I don't live in the U.S.A, so I don't qualify for Film Movement membership. They only ship to the U.S. :(

path: /stuff | permanent link |
Indie movies to the masses via DVD

Film Movement runs a DVD-a-month club that distributes first-run indie movies to its subscribers for US$19.95 (about $34 Aussie). Such films have included "El Bola," a Spanish film about child abuse that won four Goya awards, the highest honors in Spain and "He Died with a Felafel in His Hand," a very cool Australian indie movie that had an incredibly short run even in its home land. Each monthly feature film also comes with a first-run short film.

Maybe the "masses" bit won't really happen, but this sort of effort really will help ward off the MPAA-taking-over-the-world feeling that I sometimes get. In the words of the director of "Marion Bridge," a Canadian film about a dysfunctional family in Nova Scotia:

"They are guaranteeing us a five- to 10-city release, which is really great for a Canadian film about three women," said "Marion Bridge's" director, Wiebke von Carolsfeld. "It's funny and it's sad but it doesn't have explosions."

from cnn.com via Neil Schemenauer's Web Log

path: /stuff | permanent link |