Python Packaging Build-Depends

As a follow-up to my last post where I discussed common Python packaging related errors, I thought it would be worth to have a separate post on how to decide on build-depends for Python (and Python3) packages.

The python ecosystem has a lot of packages built around supporting multiple versions of python (really python3 now) in parallel.  I’m going to limit this post to packages you might need to build-depend on directly.

Python (2)

Since Jessie (Debian 8), python2.7 has been the only supported python version.  For development of Stretch and backports to Jessie there is no need to worry about multiple python versions.  As a result, several ‘all’ packages are (and will continue to be) equivalent to their non-‘all’ counterparts.  We well continue to provide the ‘all’ packages for backward compatibility, but they aren’t really needed any more.

python (or python-all)

This is the package to build-depend on if your package is pure Python (no C extensions) and does not for some other reason need access to the Python header files (there are a handful of packages this latter caveat applies to, if you don’t know if it applies to your package, it almost certainly doesn’t).

You should also build-depend on dh-python.  It was originally shipped as part of the python package (and there is still an old version provided), but to get the most current code with new bug fixes and features, build-depend on dh-python.

python-dev (or python-all-dev)

If your package contains compiled C or C++ extensions, this package either provides or depends on the packages that provide all the header files you need.

Do not also build-depend on python.  python-dev depends on it and it is just an unneeded redundancy.

python-dbg (or python-all-dbg)

Add this if you build a -dbg package (not needed for -dbgsym).

Other python packages

There is not, AFAICT, any reason to build-dep on any of the other packages provided (e.g. libpython-dev).  It is common to see things like python-all, python, python-dev, libpython-dev in build-depends.  This could be simplified just to python-all-dev since it will pull the rest in.

Python3

Build-depends selection for Python 3 is generally similar, except that we continue to want to be able to support multiple python3 versions (as we currently support python3.4 and python3.5).  There are a few differences:

All or not -all

Python3 transitions are much easier when C extensions are compiled for all supported versions.  In many cases all that’s needed if you use pybuild is to build-depend on python3-all-dev.  While this is preferred, in many cases this would be technically challenging and not worth the trouble.  This is mostly true for python3 based applications.

Python3-all is mostly useful for running test suites against all supported python3 versions.

Transitions

As mentioned in the python section above, build-depends on python3-{all-}dev is generally only needed for compiled C extensions.  For python3 these are also the packages that need to be rebuilt for a transition.  Please avoid -dev build-depends whenever possible for non-compiled packages.  Please keep your packages that do need rebuilding binNMU safe.

Transitions happen in three stages:

  1. A new python3 version is added to supported python3 versions and packages that need rebuilding due to compiled code and that support multiple versions are binNMUed to add support for the new version.
  2. The default python3 is changed to be the new version and packages that only support a single python3 version are rebuilt.
  3. The old python3 version is dropped from supported versions and packages will multiple-version support are binNMUed to remove support for the dropped version.

This may seem complex (OK, it is a bit), but it enables a seamless transition for packages with multi-version support since they always support the default version.  For packages that only support a single version there is an inevitable period when they go uninstallable once the default version has changed and until they can be rebuilt with the new default.

Specific version requirements

Please don’t build-depend against specific python3 versions.  Those don’t show up in the transition tracker.  Use X-Python3-Version (see python policy for details) to specify the version you need.

Summary

Please check your packages and only build-depend on the -dev packages when you need it.  Check for redundancy and remove it.  Try and build for all python3 versions.  Don’t build-depend on specific python3 versions.

Python3.5 is default python3 in sid

As of today, python3 -> python3.5.  There’s a bit of a transition, but fortunately because most extensions are packaged to build for all supported python3 versions, we started this transition at about 80% done.  Thank you do the maintainers that have done that.  It makes these transitions much smoother.

As part of getting ready for this transition, I reviewed all the packages that needed to be rebuilt for this stage of the transition to python3.5 and a few common errors stood out:

  1. For python3 it’s {python3:Depends} not {python:Depends}.
  2. Do not use {python3:Provides}.  This has never been used for python3 (go read the policy if you doubt me [1]).
  3. Almost for sure do not use {python:Provides}.  The only time it should still be used is if some package depends on python2.7-$PACKAGE. It would surprise me if any of these are left in the archive.  If so, since python2.7 is the last python2, then they should be adjusted.  Work with the maintainer of such an rdepend and once it’s removed, then drop the provides.
  4. Do not use XB-Python-Version.  We no longer use this to manage transitions (there won’t be any more python transitions).
  5. Do not use XB-Python3-Version.  This was never used.

Now that we have robust transition trackers [2], the purpose for which XB-Python-Version is obsolete.

In other news, pysupport was recently removed from the archive.  This means that, following the previous removal of pycentral, we finally have one and only one python packaging helper (dh-python) that supports both python and python3.  Thanks to everyone who made that possible.

 

[1] https://www.debian.org/doc/packaging-manuals/python-policy/

[2] https://release.debian.org/transitions/html/python3.5.html

Debian LTS Work December 2015

This was my eighth month as a Freexian sponsored LTS contributor. I was assigned 8 hours for the month of December.  It’s also the month in which I (re)learned an important lesson.

I decided to take another run at backporting the security fixes for Quassel.  Unlike the first time, I was successful at getting the fixes backported.  Then I ran into another problem: the changes took advantage of new features in c++11 such as std::function.

I made an attempt to change things away from c++11 with my limited c++ foo and after running head first into a brick wall several times finally consulted with the upstream author of the original fixes.   He let me know that while the problematic code is in fact present in the quassel versions in squeeze and wheezy, it’s not actually possible to trigger the security issue and that the CVEs should not actually apply to those versions.

That’s my report of a singularly unproductive and unpleasant 8 hours.  Next time I ask upstream first if there’s any doubt.  I shouldn’t assume they only care about current/recent releases.

Debian LTS Work November 2015

This was my seventh month as a Freexian sponsored LTS contributor. I was assigned 8 hours for the month of November.

As I did last month, I worked on review and testing of the proposed MySQL 5.5 packages for squeeze-lts and did a bit more work on Quassel.  It has been suggested that maybe we ought to just EOL Quassel since backporting the necessary fixes is so complicated.  I think they may be right, but I haven’t quite given up yet.

I reviewed CVE-2015-6360 for SRTP and my assessment was that squeeze-lts was not affected (same for the other Debian releases while I was at it).

I published one security update, it was for libphp-snoopy.  This resolves the outstanding security issues by updating to the newest version as was done for all other Debian releases.

Finally, in the interest of getting better support in tools for Debian LTS, I came up with a patch for the pull-debian-source[1] script in ubuntu-dev-tools so that it will download Debian LTS packages correctly.  Although it took a bit of investigating, the patch turned out to be very simple.  I filed bug #806749.  I also started looking at the distro-info package (thinking I’d need it updated to fix pull-debian-source, which turned out not to be the case), but didn’t finish it yet.  I plan to work on that this month.

[1] Even though this is in ubuntu-dev-tools and not devscripts, there’s really nothing Ubuntu specific about it.

Debian LTS Work October 2015

This was my sixth month as a Freexian sponsored LTS contributor. I was assigned 4 hours for the month of October and I had 4 unused hours from September for a total of 8.

With that time I started working on backporting security fixes for Quassel, but it’s turned into a major project. The commit message for one of the commits between what’s in squeeze-lts and what I was trying to backport is “Reformat ALL the source!”. That’s never a good sign.

I set that aside and focused instead on reviewing the MySQL 5.5 packages that the LTS team is working on. They are getting there, but we need to make sure we have it all right as we don’t want to break existing installations.

This month I hope to continue the work on both these packages.

Resolving Tension …

I just noticed this post with the same title. At least in my case, I feel the tension with the community council is resolved.

In my case I resolved it by resigning from the Kubuntu Council, stopping work on Ubuntu development, and starting to migrate more of my systems to Debian.

For me, the tension is resolved because it’s not my problem any more.

Debian LTS Work August 2015

This was my fourth month as a Freexian sponsored LTS contributor. I was assigned 4 hours which was enough for me to release a fix for screen and review CVEs for libvpx and determine that they did not apply to squeeze-lts. The screen update is covered under DLA 305-1.



Follow

Get every new post delivered to your Inbox.