Richard Jones' Log

The 17th Python Game Programming Challenge (PyWeek) is coming. It'll run from the 1st to the 8th of September.

The PyWeek challenge:

1. Invites entrants to write a game in one week from scratch either as an individual or in a team,
2. Is intended to be challenging and fun,
3. Will increase the public body of game tools, code and expertise,
4. Will let a lot of people actually finish a game, and
5. May inspire new projects (with ready made teams!)

Check out the help page for how to compete and the growing resources message board post.

So PyCon Australia is done for another year and it was a corker. I'm somewhat overwhelmed and Graeme Cross has an great summary of the event, but I really must repeat what an amazing job the organisers did in running a quality, smooth event packed with Python. See also Katie Miller's writeup about the 10 things that PyCon AU does particularly well.

Personally, I pleased as punch that my talk Don't Do This went as well as it did - I had a blast giving it and it was well-received.

Also, Ben Finney talked a bunch about PyCon AU in Wednesday's Byte Into It on 3RRR.

PyCon was awesome.

It started with the Young Coders tutorial where I had the privilege of helping out as an assistant. The kids were amazing. Barbara Shaurette and Katie Cunningham were amazing. It was awesome.

The next day I popped in to help set up that room again but then spent the rest of the day focusing on my afternoon tutorial teaching people to write games in Python. It was packed - 35 or so attendees had signed up but then a bunch of the kids from the first day of Young Coders also came along to watch and learn. Including an 8 year old girl who told her dad she wanted to write video games. Awesome.

Then the conference proper started with an inspiring opening speech by the conference chair Jesse Noller about his vision of Change the future - education, outreach, politeness, respect, tenacity and vision. A broad, ambitious and awesome vision, and one I can fully get behind. Then Eben Upton took the stage to tell us about his journey with the Raspberry Pi project and a little of his vision. Lots of similarities. And then everyone got a Raspberry Pi. Awesome.

People wanting help figuring out how to teach kids to program their Raspberry Pi could try the free Raspberry Pi Education Manual PDF.

I spent a lot of time during the conference split between talking through various things in the hallway (I'm the Cheeseshop BDFL and there's many things afoot there and PyCon is a great time to move things forward very quickly), and spending time in the Raspberry Pi lab, chairing talk sessions for amazing speakers and generally having my mind awesomely expanded.

I taught many people the basics of pygame. I ran into the Kivy developers, and talked to them about the Pi. During the sprints they ported Kivy to the Pi and even developed some simple games using Kivy, the Pi and some simple hardware sensors (I'm playing the game with a tilt sensor).

I had an inspiring conversation with James Bennett during which a light went off in my head and the result is Python Enhancement Proposal (PEP) 439 for "Inclusion of pip bootstrap in Python installation". (apologies if you get a badly-formatted version; it should be fixed shortly.)

I'm exhausted. The people here - the community - is amazing and vibrant and brilliant and huge. 2500 people at the conference, 20% women. Awesome.

Seriously, look how many people were there...

The 16th Python Game Programming Challenge (PyWeek) is coming. It'll run from the 14th to the 21st of April.

The PyWeek challenge:

1. Invites entrants to write a game in one week from scratch either as an individual or in a team,
2. Is intended to be challenging and fun,
3. Will increase the public body of game tools, code and expertise,
4. Will let a lot of people actually finish a game, and
5. May inspire new projects (with ready made teams!)

Check out the help page for how to compete and the growing resources message board post.

Recently we have been auditing and improving security of the Python Package Index (PyPI) and other python.org hosts.

You may be aware that the wiki.python.org host was compromised. Since we must assume that all passwords stored in that system are also compromised, and we also assume that some users share passwords between python.org systems, we are performing a password reset of all PyPI accounts in one week's time, at 2013-02-22 00:00 UTC.

If you log in before that deadline and change your password then you'll be fine, otherwise you'll need to use the password recovery form after the reset has occurred.

Additionally, we would ask you to begin to access PyPI using HTTPS through the web. We're in the process of installing a new SSL certificate so the current Big Red Certificate Warning should go away very soon.

We are in the process of updating the Python packaging toolset to use HTTPS.

These steps are but a couple of those we're intending to take to better secure PyPI. If you are interested in these matters I encourage you to participate in the discussion on the catalog SIG.

Finally, we apologise for any inconvenience these changes have caused.

We've just added a new domain to the python.org stable, pythonhosted.org, to replace the existing packages.python.org domain. All accesses to the old domain are redirected to the new - all existing references will work. Google et al should catch up eventually.

This was done as part of an overall review of security of the PyPI website. Warning, that thread is longish :-)

Occasionally I need to compile cx_Oracle on OS X and the 32- and 64-bit worlds collide head-on in a mess of "implicit conversion shortens 64-bit value into a 32-bit value" and "file was built for unsupported file format which is not the architecture being linked (x86_64)" errors.

I keep forgetting the various steps needed to make this work correctly so here they are:

1. Create a virtualenv with a python interpreter stripped to 32-bit only. In the virtualenv bin directory ($WORKON_HOME/name_of_virtualenv/bin):

   % mv python python.fat
   % lipo python.fat -remove x86_64 -output python

   Doing this saves all the hassles of that "arch", "VERSIONER_PYTHON_PREFER_32_BIT" and "defaults write blah blah" guff and doesn't affect any other virtualenvs.
2. Now that we have a "thin" Python compatible with the Oracle library, we can build cx_Oracle. Unfortunately even though Python is thin its configuration is not so we need to force things using an obscure environment variable that distutils will pick up:

   ARCHFLAGS="-arch i386" pip install cx_Oracle

There, that was easy, wasn't it?

The 15th Python Game Programming Challenge (PyWeek) is coming. It'll run from the 9th to the 16th of September.

The PyWeek challenge:

1. Invites entrants to write a game in one week from scratch either as an individual or in a team,
2. Is intended to be challenging and fun,
3. Will increase the public body of game tools, code and expertise,
4. Will let a lot of people actually finish a game, and
5. May inspire new projects (with ready made teams!)

Check out the help page for how to compete and the growing resources message board post.

The next PyWeek will be in May from the 6th to 13th. Not April.

The 14th Python Game Programming Challenge (PyWeek) is coming. It'll run from the 6th to the 13th of May.

The PyWeek challenge:

1. Invites entrants to write a game in one week from scratch either as an individual or in a team,
2. Is intended to be challenging and fun,
3. Will increase the public body of game tools, code and expertise,
4. Will let a lot of people actually finish a game, and
5. May inspire new projects (with ready made teams!)

If you're in the US and can make it I'm co-presenting a 3 hour pygame tutorial at PyCon in March.

PyPI is now an OpenID provider.

To use this OpenID provider, enter pypi.python.org into any form that expects an OpenID*. Should the service not support OpenID 2, you will have to enter pypi.python.org/id/username instead (using your PyPI username.) Log into PyPI and visit your details page if you'd like to cut-n-paste the URL.

We follow the emerging approach that you have to sign into PyPI before signing into the actual services. This is intended to prevent phishing, as otherwise the relying party may fake PyPI's login page and collect your PyPI password (which they can still do if you fall for it.) It also avoids "nested" logins (i.e. where you need to log into PyPI with an OpenID while trying to login elsewhere with the PyPI id.)

If you find any problems with this service, please report them to the PyPI bug tracker.

*: of course for sites that extend PyPI this can be simplified to a simple button saying "link to my PyPI account".

This post contains some tips for testing Twisted applications. Note that I'm not using trial, the official Twisted test runner, for a variety of reasons.

@deferred(1)

defer.setDebugging(1)
observer = log.PythonLoggingObserver()
observer.start()

--- original/nose/twistedtools.py
+++ installed/nose/twistedtools.py
@@ -39,12 +39,19 @@

 _twisted_thread = None

+_deferreds = []
+
 def threaded_reactor():
     """
     Start the Twisted reactor in a separate thread, if not already done.
     Returns the reactor.
     The thread will automatically be destroyed when all the tests are done.
     """
+    from twisted.internet import defer
+    def __init__(self):
+        _deferreds.append(self)
+    defer.DebugInfo.__init__ = __init__
+
     global _twisted_thread
     try:
         from twisted.internet import reactor
@@ -135,6 +142,8 @@
             def errback(failure):
                 # Retrieve and save full exception info
                 try:
+                    if issubclass(failure.type, StandardError):
+                        print failure
                     failure.raiseException()
                 except:
                     q.put(sys.exc_info())
@@ -156,6 +165,9 @@
             try:
                 error = q.get(timeout=timeout)
             except Empty:
+                for d in _deferreds:
+                    if hasattr(d, 'creator') and not hasattr(d, 'invoker'):
+                        print d._getDebugTracebacks()
                 raise TimeExpired("timeout expired before end of test (%f s.)"
                                   % timeout)
             # Re-raise all exceptions

@then('the message should be sent')
@deferred(1)
@inlineCallbacks
def step(context):
    # wait until we're all done
    context.browser.implicitly_wait(1)
    yield threads.deferToThread(lambda: get_element(context.browser, id="done"))

    # there should be a result container
    results = yield threads.deferToThread(lambda: find_elements(context.browser, id="result"))
    result = results[-1]
    status = yield threads.deferToThread(lambda: get_element(result, id="status"))

    # the status could be a number of values depending on synchronisiation of
    # the tests / runner
    assert status.text in 'created new accepted done'.split()

    message = context.db.messages.values()[0]
    if message.status != message.STATUS_DONE:
        yield esme._wait_until_done(context)

More tips as I happen to think of them or discover them :-)

Thanks, Tarek, for this fun idea.

1. What's the coolest Python application, framework or library you have discovered in 2011?
I was pretty happy I discovered the awesomeness of bottle when researching my web micro-framework battle (video.)

I've started using Python 2.7 and 3.2 which is pretty cool (having been stuck in 2.3, gasp!)

I've been working with Twisted again after a couple of years' break and have discovered txpostgres and Twisted's own inlineCallbacks. Both are pretty cool. inlineCallbacks make Twisted programming quite bearable to think about :-)

2. What new programming technique did you learn in 2011?
I've been honing my testing skills and learned about Behaviour Driven Development. I've also learned to use both Mercurial and Git (and the latter still drives me insane sometimes) and their related websites bitbucket and github.

I also started using virtualenv and Fabric way more this year.

3. What's the name of the open source project you contributed the most in 2011? What did you do?
This year I created parse, overload and ooch and contributed to behave. I also helped run PyCon Australia.

4. What was the Python blog or website you read the most in 2011?
Planet Python and the New Packages feed from PyPI.

5. What are the three top things you want to learn in 2012?
I can never predict what I'll end up learning. Or what software I'll be writing.

I know I'll be learning a lot more about Mercuruial (and Git, I suppose) and the vagaries of modern software deployment.

I plan on working more on my test- and behaviour-driven development practices.

I also want to learn how to bake more things thanks to the Great British Bake Off :-)

6. What are the top software, app or lib you wish someone would write in 2012?
I'm hoping that someone will do something cool with the new PyPI OpenID provider ;-)

Want to do your own list? here's how:

1. copy-paste the questions and answer to them in your blog
2. tweet it with the #2012pythonmeme hashtag

We've had a fun couple of days sprinting on the Cheese Shop at PyCon AU where a number of contributors have fixed bugs and improved or added features (though always with the goal of keeping the service simple of course.)

In particular:

1. Andy Todd helped clean up some aspects of the underlying database and fix up some of the sql.
2. Capel Brunker added some more XML-RPC functionality, performed some tracker triage and also addressed some bugs and security issues.
3. Kaleb Ufton, in his first contribution to Open Source development, added a bug tracker URL field to packages (which persists across releases and you must enter by editing through the website.) He also helped me sort out some twisted Apache configuration issues.
4. I finally got around to writing the "newest packages" RSS feed.

There's another secret project we kicked off that will hopefully appear in the next couple of days, and some additional work that will hopefully come to fruition within a week or so. Stay tuned :-)

Thanks to everyone who contributed!

I needed support for "priv" instead of Fabric's built-in "sudo" support. I went through a number of (sometimes quite horrific) iterations before I settled on this relatively simple solution:

import contextlib

@contextlib.contextmanager
def priv(user):
    '''Context manager to cause all run()'ed operations to be
    priv('user')'ed.
    
    Replaces env.shell with the priv command for the duration of the
    context.
    '''
    save_shell = env.shell
    env.shell = 'priv su - %s -c' % user
    yield
    env.shell = save_shell

This is then used in a fabfile like so:

    with priv('remote_user'):
        run('do some remote command as remote_user')
        run('another remote command as remote_user')

The 13th Python Game Programming Challenge (PyWeek) is coming. It'll run from the 11th to the 18th of September.

The PyWeek challenge:

1. Invites entrants to write a game in one week from scratch either as an individual or in a team,
2. Is intended to be challenging and fun,
3. Will hopefully increase the public body of game tools, code and expertise,
4. Will let a lot of people actually finish a game, and
5. May inspire new projects (with ready made teams!)

If you've never written a game before and would like to try things out then perhaps you could try either:

1. The tutorial I presented at LCA 2010, Introduction to Game Programming, or
2. The book Invent Your Own Computer Games With Python

A reminder that the Call for Proposals for PyCon Australia 2011 will be closing soon. We've had some great proposals so far, but there is still time left and program to fill.

PyCon Australia is Australia's only conference dedicated exclusively to the Python programming language, and will be held at the Sydney Masonic Center over the weekend of August 20 and 21. See below for more information and updates on:

1. Call For Proposals
2. Registration is Open
3. More Sponsors Announced

Please share this message on to those you feel may be interested.

Call For Proposals

The deadline for proposal submission is the 2nd of May. That's only a few days away!

We are looking for proposals for talks on all aspects of Python programming from novice to advanced levels; applications and frameworks, or how you have been involved in introducing Python into your organisation. We're especially interested in short presentations that will teach conference-goers something new and useful. Can you show attendees how to use a module? Explore a Python language feature? Package an application?

We welcome first-time speakers; we are a community conference and we are eager to hear about your experience. If you have friends or colleagues who have something valuable to contribute, twist their arms to tell us about it! Please also forward this Call for Proposals to anyone that you feel may be interested.

The earlier you submit your proposal, the more time we will have to review and give you feedback before the program is finalised.

Speakers receive free registration for the conference, including a seat at the conference dinner. Don't miss out, submit your proposal today!

Registration is Open

We offer three levels of registration for PyCon Australia 2011:

Corporate - $440

If your company is paying for you to attend PyCon, please register at the corporate rate. You'll be helping to keep the conference affordable for all, especially for students and those needing financial aid. Government employees should also register at the corporate rate.

Full (Early Bird) - $165

This is the registration rate for regular attendees. We are offering a limited Early Bird rate for the first 50 registrations until the end of May. Once the Early Bird period ends, or when all Early Bird slots are filled, registration will increase to $198. Full registration includes one seat at the conference dinner on Saturday night.

Student - $44

For students able to present a valid student card we're offering this reduced rate. Student registrations do not include a seat at the conference dinner.

Additional seats at the conference dinner may be purchased for $77 each.

All prices include GST.

Information about the registration process is on the PyCon Australia website.

More Sponsors Announced

We are delighted to announce that ComOps has joined as a Gold Sponsor. Thank you to the following companies for their continuing support of Python and for helping to make PyCon Australia 2011 a reality:

Gold: Google
Gold: Microsoft
Gold: ComOps

Silver: Anchor
Silver: Enthought
Silver: Python Software Foundation

Thanks also to Linux Australia, who provide the overarching legal and organisational structure for PyCon Australia.

Lennart Regebro has provided me with a free copy of his self-published book "Porting to Python 3" so that I may review it. He has considerable experience in porting code from Python 2 to Python 3, and it shows in much of his advice and examples. Regebro has assembled an excellent cast of helpers: the technical reviewer Martin von L�wis also has much expertise with Python 3 (he implemented the first port of the popular Django web framework). Brett Canon's introduction also provides a good starting point for someone new to the story of Python 3.

The book's structure is well thought-out. The first chapter immediately invites the reader to make sure they really need to port their code, and what considerations might be taken into account when deciding to do so. The book even has advice, backed up with useful information, for those who are currently unable to port their code but may start preparing for doing so in the future.

The second chapter discusses strategies for moving to Python 3 (view it online). These are presented clearly with one section per strategy. Within each section there are clear references to the other, more detailed chapters of the book which may be used to implement each strategy.

Once you've decided which strategy to apply you can focus on the pertinent remaining chapters. These follow a pretty logical progression:

• firstly preparing the way by making your code more modern,
• introducing the 2to3 tool for automatically converting code to Python 3 and the various options for how to use it,
• talking through some common migration problems and some simple solutions to them,
• presenting some modern Python (2.6+) idioms that your code may use in Python 2 and 3, and
• supporting Python 2 and 3 without using the 2to3 tool.

The final two chapters are for more limited audiences and cover migrating C extensions and writing custom 2to3 tool fixers. Even though these are topics which most readers won't need to worry about they are still covered in good detail with minimum fuss. It is in this final chapter that Regebro's experience with porting gives weight to the advice and examples he presents.

Finally there's a couple of pretty comprehensive appendices covering over the main language incompatibilities and library changes which makes it a good reference.

As previously mentioned, the book is very well-organised giving the reader an easy path through the material depending on their situation. Throughout the book there are many pertinent and clear references to software written by others. The book suffers from some typesetting and grammatical errors, though these are minor and none could cause any confusion regarding the content.

CONCLUSION
There's a lot of information out there for porting from Python 2 to 3, but Regebro has produced a concise, well-organised and complete reference for doing so. It's not skimped on any detail though; the book covers all areas related to moving to Python 3. It's not just about porting code; it's also a handy book for any programmer who's grown up with Python 2 (or 1!) and is looking to move to Python 3.

The 12th Python Game Programming Challenge (PyWeek) is almost upon us. It'll run from the 3rd to the 10th of April. Registration for teams and individuals is now open on the website.

The PyWeek challenge:

1. Invites entrants to write a game in one week from scratch either as an individual or in a team,
2. Is intended to be challenging and fun,
3. Will hopefully increase the public body of game tools, code and expertise,
4. Will let a lot of people actually finish a game, and
5. May inspire new projects (with ready made teams!)

If you've never written a game before and would like to try things out then perhaps you could try either:

1. The tutorial I presented at LCA 2010, Introduction to Game Programming, or
2. The book Invent Your Own Computer Games With Python

The second PyCon AU will be held in Sydney on the weekend of the 20th and 21st of August at the Sydney Masonic Center.

We are looking for proposals for talks on all aspects of Python programming from novice to advanced levels; applications and frameworks, or how you have been involved in introducing Python into your organisation. We're especially interested in short presentations that will teach conference-goers something new and useful. Can you show attendees how to use a module? Explore a Python language feature? Package an application?

We welcome first-time speakers; we are a community conference and we are eager to hear about your experience. If you have friends or colleagues who have something valuable to contribute, twist their arms to tell us about it! Please also forward this Call for Proposals to anyone that you feel may be interested.

To find out more go to the official Call for Proposals page.

The deadline for proposal submission is the 2nd of May.

See you in Sydney in August!

Richard Jones
http://pycon-au.org/
PyCon AU Program Chair