Richard Jones' Log: Sane Python application packaging: Linux
Just focusing on the Linux side for a moment.
Thanks to everyone who responded to yesterday's post about Python application packaging. My efforts will now be focused on creating files suitable for Linux package managers. This would initially be Debian software package (.deb) and Red Hat Package Manager (.rpm) - I considered Gentoo Linux's ebuild but really the audience for that is too narrow and they have tools to convert rpm and deb files to their own format.
Tasks for my application:
- I'll have to organise my install and generate meta-data files according to the two formats: Debian package basics and maintainer's guide (which may differ from Ubuntu's) and RPM package basics,
- Generate the package files: .deb via stdeb (there's also easy-deb) and .rpm using the built-in distutils bdist_rpm), and
- Host the files somewhere.
If you don't want to use stdeb or easy-deb there's a nice ShowMeDo screencast of creating a .deb for a Python program by Horst Jens.
One of the commenters mentioned above, Daniel, has already produced .deb files using stdeb. I say files because he produced one each for pyglet, cocos2d, and bruce. The docutils and pygments libraries are already available in Ubuntu's package repository.
Already I have a problem: Ubuntu also packages a version of pyglet. The pyglet version packaged is 1.0 which is quite out of date compared to the version I'd like to package, 1.1 (or even 1.2 pre-release). If I install the Ubuntu version and then try to upgrade to Daniel's version I get the rather unfriendly error:
richard@shiny:/tmp$ sudo dpkg -i python-pyglet_1.1.2-1_all.deb (Reading database ... 165323 files and directories currently installed.) Preparing to replace python-pyglet 1.0.dfsg-1 (using .../python-pyglet_1.1.2-1_all.deb) ... Unpacking replacement python-pyglet ... Setting up python-pyglet (1.1.2-1) ... pycentral: pycentral pkginstall: not overwriting local files pycentral pkginstall: not overwriting local files dpkg: error processing python-pyglet (--install): subprocess post-installation script returned error exit status 1 Processing triggers for python-support ... Errors were encountered while processing: python-pyglet
Removing the existing python-pyglet package resolves this issue of course. I don't know what the state is for other Linux distributions.
So, there's still some issues:
- There's a potential issue when a Linux distribution starts packaging bruce, pyglet and cocos2d itself. As mentioned above there's already one conflict on Ubuntu.
- I've still got an issue with compatibility. While I control the package files that's not a big deal but once the distributions start doing so I no longer have any control, and
- Is it better to require users to download .deb files or would it be better to be able to just add "deb http://pypi.python.org/debian/ pypi" to Debian's apt configuration, and whatever the similar configuration is for rpm-based systems (yum or urpmi)?
I also noticed getdeb and CNR (click n run) while poking around. They could be good application promotion tools. There's probably others.
(Updated to include link to Ubuntu packaging guide)
Thanks, I did try to find it :)
I prefer an apt source. If I'm going to be tracking an upstream build, I am probably going to want to keep tracking it...
Adding custom apt lines is now best done in /etc/apt/sources.list.d, ie, provide people not just with a line for their /etc/apt/sources.list file, but with an entire file which they then stick in /etc/apt/sources.list.d
Hi again :)
I've just found that pyglet 1.1.2 is packaged for Ubuntu Jaunty:
http://packages.ubuntu.com/jaunty/python-pyglet
The package installs fine over mine, on my Ubuntu Hardy, so it's pretty cross-version.
Jaunty also has newer docutils and pygments, which I'll check after some sleep :)
The reason you get a failure when installing the Debian package is that you have an install of this package that you did outside of the Debian packager, with files where the Debian packager wants to install its own files. The Debian packager is by default refusing to overwrite the install that you did.
On a side note, I have always be very surprised that distutils/setuptools defaulted to installing in "/usr" under Linux. This is absolutely against basic system administration guidelines under almost all distributions. If you want the system to work well, you have to play well with the system: the right prefix to install to when installing packages not managed by the package manager is "/usr/local".
Wow.
This and yesterday's post---and their comments---have surely taught me this: Don't bother packaging your code for fancy installers and/or package management systems. Just create a tar.gz of it, include a good INSTALL.txt file, and leave the rest to the users. If your software is useful, someone will no doubt come along and package it up as a deb/rpm/tgz/whatever package for their respective OS-of-choice.
"Simple is better than complex."
@Ex: In my experience, pushing out a .deb will basically double or treble your users overnight. But if you don't really care about adoption rates...
Thanks to Richard for some very good links anyway.
There's a Python Application Packaging Team which accepts people outside Debian (we have f.e. few upstream authors and few Ubuntu developers with us)
Our packages can be found here (QA page)
join #debian-python @ OFTC if you want help or if you want your package to be uploaded to Debian
(for Python modules/extensions there's a Debian Python Modules Team, same channel, almost the same people)
BTW: I agree with Ex, INSTALL.txt file is enough. I've seen too many badly packaged debs outside Debian. To be honest, I never accepted a package from non-DD without some major fixes first.
I didn't bother to create RPM for my own app. - people from RPM based distros know better how it should look like. I only try to make their lives easier by adding notes about new dependencies or other important changes in NEWS file (besides some notes in INSTALL or README files)
Just to be clear, I'm sure we're all talking about packaging applications here.
For making distributions of packages and modules, looks like distutils is still the standard solution for that.
you may also want to take a look at epm for building packages for multiple targets from one installation description/configuration file.
There is a packaging guide for ubuntu: https://wiki.ubuntu.com/PackagingGuide