Extending Debian with customizable packages

In my previous post I drafted a rough idea how to add a feature to Debian that would bridge the gap between Debian as a Binary Distribution and any Source Distribution. The feature in question is giving users the chance to build customized Debian packages for specific feature sets. The possible use cases are simple:

Lets say Anthony User needs super-duper-tool with postgresql support. However this is a not so common use case and so the archive only has postgresql support. If he now wants to build a custom package, the process is „as simple“ as:

apt-get source super-duper-tool
cd super-duper-tool-*
apt-get build-dep super-duper-tool OR sudo mk-build-deps -r -i
(possibly try out which ./configure options are supported by super-duper-tool)
vi debian/rules
edit debian/rules
dpkg-buildpackage -us -uc
sudo debi

That is, to be honest lot of work. And the biggest problem about it is that you need to know all this, which is not really user knowledge.

The basic idea is that Anthony can do something like
export DEB_BUILD_OPTIONS=nomysql,postgresql
deb-build-tool build super-duper-tool

and is done.
So the question is how could this be achieved?

  1. Define a set of common options (options should be consistent through all source packages, otherwise it makes not that much sense) that every package should support, if it is a supported feature by the package in question. This includes flags like mysql, postgresql, ldap and its no-equivalents (nomysql, nopostgresql) to negate it.
  2. Enable use of this common options in the package build process and that is really the hardest part. Defining a lot of ifneq..findstring constructs for 2-10 options per source package in any available source package is…. not…. a good approach. Luckily a lot of packages uses autoconf where enabling and disabling features is as easy as adding a –with-foo or –enable-foo option to the ./configure parameters. So we could write a wrapper that would handle these options. The debhelper scripts already have dh_auto_configure: Maybe this could get enhanced.
  3. If it does not exist: Write a tool to auto-build packages with DEB_BUILD_OPTIONS set by the user. Should automatically get the source, (optional: setup/update a pbuilder environment) and build it and optional install/upload it (to a user repository for example) .

Still, the idea is only a rough draft. There are still a lot of open questions/issues, for example:

  • How to handle these feature flags in packages that don’t use autoconf
  • How to enable these feature flags for packages that don’t use debhelpers
  • Shall a central place for setting DEB_BUILD_OPTIONS exist (like it does for Gentoo, FreeBSD etc.) and if so: Where?
  • How should packages define new build options, if those defined Debian-wide, aren’t enough?
  • What about support? Do we support such custom builds of packages or will this be a support-on-good-will thing

Is anybody interested in pushing this idea forward? I’m happily interested in it and I know people who like the idea, too. But I can’t push this forward on my own all alone. Certainly the work won’t affect Lenny anymore, but considerable the time frame that is planned for Lenny+1 by the release team, this could be a feature for Lenny+1.

Where to find help for commands?

Today an discussion rised up in #debian-devel on the OFTC IRC Network. It started because someone noted the bug report #501318 which is, to summarize it, just a user mistake, because someone obviously read the man page (time(1)) for the /usr/bin/time command, while time is also a shell builtin (which does not accept the same arguments as the time command. Certainly this bug reports appears to be funny at the first sight, but on the other hand, there is a suboptimal situation that leads to this.

  1. There are some builtin commands that also have binary equivalents (like time, printf, echo). Its quiet easy to tell weither the one or the other is used, by using the which command, but thats not really a realistic workflow, so its better to know this. The difference between these commands is often causing problems, which we have to cope with. For example bashs builtin echo behaves different as /bin/echo and people who use the bash as their default shell tend to use what bash provides, which in turn causes problems if other people who use a different default shell try to work with these scripts. But this is another problem, because…
  2. … every program, utility and function in Debian has to provide a manpage, as said by our policy: „Each program, utility, and function should have an associated manual page included in the same package.“ I guess that the rationale behind that is that ‚man‘ is a very common tool in the Un*x world, which is wide-spread and which usage is much recommended to find out how specific tools behave. It is always referred to in Documentation, weither it is Debian specific or not, e.g. in books.

    The time package, which is of priority Standard and which includes /usr/bin/time, does conform to that policy by providing a manpage for the time command. The various shells don’t do that, because they usually don’t have a man page for each and every command (usually they have a more generic manpage which includes the builtins or in some seldom cases a special manpage for such and similar things (as for zsh, which has zshmisc(1)) and because its not that easy, because the man command cannot (AFAIK) distinguish between the user joey_average calling ‚man time‘ in a bash, while the user schoenfeld calls the command ‚man time‘ in a zsh, or if joey_foobar calls ‚man read‘ in a shell which does not have a builtin time command and uses /usr/bin/time instead.

So whats wrong about this situation? Would you say that users that expect ‚man time‘ (or similar examples) to do the right thing are making wrong assumptions? I disagree. Its what they’ve always been told to do. And if it does not show anything eventually run info, or look at HTML documentation or what else. But they haven’t been prepared for the case where the manpages does show something, something wrong. The good thing is that the manpage for time includes a sentence:

„Users of the bash shell need to use an explicit path in order to run the external time command and not the shell builtin variant.“

The bad thing about it is, that it is at approx. 70-80% of the manpage.

Clint, the maintainer of zsh, mentioned run-help which seems to be a part of the zsh, but not of any other shell, and does more or less the right thing (at least for the builtins) but not for external commands and not even for itself (it opens the code of the function instead of something user-readable like a manpage). I guess „one tool for a specific need“ is a good maxime, but is it really a good maxime for finding documentation?

But how could the situation be bettered? I could think of a wrapper for man, similar to run-help but as a more generic solution. Any ideas for it? Is it the right way at all to better the users experience? Any other ideas? Other opinions?