Mon, 23 Sep 2002
I was avoiding this until I could do some decent profiling and figure hotspots and so forth, but the current profiling tool produces... difficult to use results. Maybe some day I'll write an interface to it ;) I also fixed a bug in the LRU caching for the SQL backends (I'm a doofus ;)
Anyway, the new numbers on the Sun box are:
Test name fetch journl jprops lookup filter filtml TOTAL
anydbm: 10 0.02 0.10 0.02 0.04 3.03 2.82 6.03
bsddb: 10 0.02 0.08 0.02 0.03 2.77 2.83 5.74
bsddb3: 10 0.02 0.09 0.02 0.03 2.83 2.79 5.78
sqlite: 10 0.09 0.35 0.09 0.02 0.38 0.54 1.46
anydbm: 20 0.03 0.25 0.03 0.09 5.28 5.35 11.04
bsddb: 20 0.03 0.17 0.03 0.06 5.28 5.26 10.83
bsddb3: 20 0.03 0.18 0.04 0.07 5.94 5.40 11.65
sqlite: 20 0.17 0.73 0.17 0.03 0.53 0.98 2.61
anydbm: 100 0.15 0.94 0.17 0.77 57.30 57.10 116.43
bsddb: 100 0.14 0.85 0.17 0.62 64.23 62.95 128.97
bsddb3: 100 0.31 1.74 0.38 2.06 68.36 57.70 130.56
sqlite: 100 0.92 4.31 0.94 0.20 1.92 11.38 19.68
Neat, huh? I'm a bit concerned by the change in the sqlite time, since there really shouldn't be much going on in python-land...
Why has no-one written a tool to analyse python profiling data? Something as simple as the Zope call profiler I wrote would be nice. I noticed an
article
which at first glance has a solution: turn the profiling data into an XML call tree, and XSLT the result. Only problem is, the module supplied is obscurely written (the article doesn't really help) and doesn't have even the most basic command-line interface.
Time to go see what I can produce I suppose...
Well, the numbers are in for the sun box:
Test name fetch journl jprops lookup filter filtml TOTAL
anydbm: 10 0.04 0.18 0.05 0.09 6.79 9.65 16.81
bsddb: 10 0.04 0.21 0.05 0.09 5.01 4.94 10.34
bsddb3: 10 0.04 0.16 0.05 0.08 4.88 4.89 10.11
sqlite: 10 0.11 0.43 0.33 0.02 0.54 0.70 2.13
anydbm: 20 0.08 0.35 0.11 0.22 9.96 9.75 20.47
bsddb: 20 0.08 0.32 0.11 0.18 9.64 9.88 20.19
bsddb3: 20 0.20 0.64 0.27 0.28 11.03 16.23 28.65
sqlite: 20 0.23 0.94 0.96 0.05 0.66 1.33 4.17
anydbm: 100 0.89 3.33 0.97 5.88 122.97 119.40 253.44
bsddb: 100 0.39 1.63 0.53 2.92 121.49 103.47 230.42
bsddb3: 100 0.41 1.67 0.54 3.04 103.09 110.23 218.99
sqlite: 100 1.21 5.24 3.83 0.28 1.98 21.06 33.60
... ouch. The time for the join in the filtml select really comes out in the sqlite 100 test...
Or rather, it was raised, late last year ;)
I've decided that Grand Theft Auto 3 is a most amazing piece of work, regardless of what everyone else says :)
The amount of freedom that the game's developers have given the player is truly astounding. The bar is definitely raised.
The game has a distinct moral imbalance, and the developers have documented "issues with women". Both of these could be reasonably addressed through:
- Greater freedom to choose a career with the Good Guys (beyond vigilante justice, putting out car fires and saving people with the ambulance). Most of the missions that appear in the game could be played from the perspective of a Good Guy, and with a lighter emphasis on the violence. Just for an alternative... and
- Having women in the game that aren't hookers, bag ladies or sex-fantasy-lesbians.
Well, first he analyses the hole that Lucas has dug himself, but then
proposes a way out of it. (ta Cam via Rachel)
I've made a few more optimisation changes to Roundup, but the index page of our dev tracker (about 600 entries) takes a good 8 seconds to render on our sparc. The last set of benchmark results I posted were run on the laptop.
- Laptop
- This machine benchmarks at 14285.7 pystones/second
- Sun box
- This machine benchmarks at 3021.15 pystones/second
Bit of a difference there - I've got to keep remembering that my development environment is blindingly fast ;)
Just running the benchmark on the target system now, will be interesting to see the difference...
