Richard Jones' Log: (Update) Roundup benchmark time again
Update: have added timings for N=100,000 (or thereabouts), which equates to 87,000 or so open issues.
Roundup 0.7 includes some new optimisations, specifically automatic generation of a number of table indexes in the RDBMS backends.
I've added a script "tools/load_tracker.py" which loads a tracker with N issues and N/100 users. Random distribution of priorities, statuses and assignetos. Approximately 85% of issues are open (ie. 17 in 20, 170 in 200, etc).
Against a stock classic tracker, loading the standard index page (sort by activity, group by priority, showing batches of 50 issues at a time):
| Backend | N | Index Page Time |
|---|---|---|
| anydbm | 200 | 3.5s |
| sqlite | 20,000 | 1.1s |
| 100,000 | 4.5s (9.4s) | |
| metakit | 20,000 | 1.9s |
| 100,000 | 8.4s (18s) | |
| mysql | 20,000 | 1.3s |
| 45,000 | 2.1s | |
| 65,000 | 3.3s (5.7s) | |
| postgresql | 20,000 | 1.1s |
| 75,000 | 2.7s | |
| 100,000 | 4.1s (5.7s) |
Note:The second (bracketed) values indicate the time for a concurrant index page view (I press reload in a second browser window shortly after requesting the first). The sqlite and metakit backends serialise accesses, so their second access time is understandably doubled.
Any particular reason why you do not use the same n's for all dbs/except maybe anydbm) ?
Martin
The intention was to do that, but the load time for mysql was just too long (it took over 12 hours to load 65,000). I do intend to finish loading mysql with the 100,000 issues, and I'll update the list then.
Indeed! Our bsddb3 roundup tracker has slowed to a crawl with only ~400 issues(quickly growing). The main index page with about 50 open issues takes about 10s to load. The server is a P3-933 with enough RAM, running nothing else at the moment, so it looks like we'd greatly benefit from switching backends.
bsddb3 appears to have the worst performance of all the backends. I'm not sure why yet, but it really, really sucks.
Would be nice if there was a unit time displayed no? IPT / N, so we could get a sense of what was truly fastest?
Also, is there an existing benchmark for 0.6.xx somewhere kicking around, so we can see the speed differences?
I was thinking to myself "anydbm was much slower than that for me" until I noticed the two-orders-of-magnitude difference in test size between the anydmb hash and the RDBMS tests.
Nice. And I'm gladder still that I migrated to sqlite a while ago...