On Debian discussions

In my article „Last time I’ve used network-manager“ I made a claim for which I’ve been criticized by some people, including Stefano, our current (and just re-elected) DPL.  I said that a certain pattern, which showed up in a certain thread, were a prototype for discussions in the Debian surroundings.

Actually I have to commit, that this was a very generalizing statement, making my own point against the discussion point back directly at myself.
Because as Stefano said correctly there has been some progress in the Debian discussion cult.
Indeed, there are examples of threads, were discussions followed another scheme.
But to my own advocacy I have to say that such changes are like little plants (in the botanical sense). They take their time to grow and as long as they are so very new, they are very vulnerable to all small interruptions. Regardless of how tiny those interruptions may seem.

I’ve been following Debian discussions for 6 or 7 years. That scheme I was describing was that which had the most visibility of all Debian discussions. Almost every discussion which were important for a broader audience followed that scheme. It has a reason that Debian is famous for flamewars.
In a way its quiet similar to the network-manager perception, some people have. Negative impressions manifest themselves. Especially if they have years of time.
Positive impressions does not have a chance to manifest themselves as long as the progress is not visible enough to survive small interruptions.

I hope that I didn’t cause to much damage with my comment, which got cited (context-less) on other sites. Hopefully the Debian discussion cult will improve further to a point where there is no difference between the examples of very good, constructive discussions we already have in some parts of the project and the project-wide decision-making-discussions which affect a broad audience and often led to flamewars.

password-gorilla 1.5.3.4 ACCEPTED into unstable

The password-gorilla package has lacked some love since a while and at some point in time I orphaned it.
That happened due to the fact, that the upstream author was pretty unresponsive and inactive and my own TCL skills are very limited. As a result password-gorilla package was in a bad state, at least from a user point of view, with several (apparently) random happening error message and alike, stalling feature development etc.

But in the meanwhile there was a promising event arising. A guy, named Zbigniew Diaczyszyn, wrote me a mail that he intended to continue upstream development. Well, meanwhile is kind of an understatement. That first mail already happened in December 2009. And he asked me, if I’d like to continue maintaining password-gorilla in Debian. I agreed to that, but as promising as it sounded to have a new upstream, I was not sure if that would work out. However: My doubt were not justified.

In the time between 2009 and now Zbigniew managed to become the official upstream (with the accreditation of the previous upstream), create a github project for it and make several releases.


I know there are several people out there who tested password-gorilla. I know there were magazine reviews including the old version, which were a bit buggy with recent tcl/tk versions. It made a quiet good multi-platform password manager, with support for very common password file formats, stand in a bad light.
I recommend previous users of password-gorilla to try the new version, which recently has been
uploaded to unstable.

Last time I’ve used network-manager..

Theres an ongoing thread on the Debian mailing lists about making network-manager installed by default on new Debian installations. I won’t say much about the thread. Its just a prototype example for Debian project discussions: Discuss everything to death and if its dead discuss a little more. And – very important – always restate the same arguments as often as you can. Or if its not your own argument you restate, restate the arguments of others. Ending with 100 times stated the same argument. Even if its already disproved.

I don’t have a strong opinion about the topic in itself. However there is something I find kinda funny. A statement brought up by the people who strongly oppose network-manager as a default.
A statetement I’ve heard so often that I can’t count it anymore.

The last time I’ve tried network-manager it sucked.

It often comes in different masquerades, like:

  • network-manager is crap.
  • network-manager is totally unusable
  • network-manager does not even manage to keep the network connection during upgrades
But it basically boils down to that basic essence of the sentence I’ve written above. Sometimes I ask people who express this opinion a simple question:

When did you test network-manager the last time?

The answers are different but again the basic essence of the answers is mostly the same (even if people would never say it that way):

A long time ago. Must have been around Etch.

And guess what: There was a time when I had a similar opinion. Must have been around Etch.
During the life cycle of network-manager between Etch and now a lot has happened. I restarted using network-manager at some point of the Lenny development.
My daily driver for the management of my network connections on my notebook. Yes, together with ifupdown because, yes, network-manager does not support every possible network-setup with all of the special cases possible. But it supports auto-configuring of wired and wireless devices. Connecting to a new encrypted network, either in a WLAN or in a 802.1x LAN, using UMTS devices, using tethering with a smart phone. And everything: on a few mouse-clicks.
Yes, it had some rough edges in that life cycle. Yes, it had that nasty upgrade bug, which was very annoying.
But face it: It developed a lot. Here are some numbers:
Diffstat between the etch version and the lenny version:
 362 files changed, 36589 insertions(+), 36684 deletions(-)
Diffstat between the Lenny version and the current version in sid:
 763 files changed, 112713 insertions(+), 56361 deletions(-)
The upgrade bug has been solved recently. Late. But better late then never.
So what does that mean? It means that, if your last network-manager experience was made with Lenny or even worse around Etch, you should better give it another try, if you are interested in knowing what you talk about. For now it seems that a lot of people do not know. Not even in a distance.

Let me introduce DPKG::Log and dpkg-report

We have customers, which require a report about what we’ve done during maintenance windows. Usually this includes a report about upgrades, newly installed packages etc. and obviously everything we’ve done apart from that.
Till now we’ve prepared them manually. For a greater bunch of systems this is a big PITA, because to be somehow useful you have to collect the data for all systems and after that, prepare a report where you have:

  • The upgrades done on all systems (e.g. a libc or kernel update)

    seperated from

  • the updates specific to a certain system or system class

Its also error-prone because humans make mistakes.

Perl to the rescue!
At least the part about generating a report about installed/upgraded/removed packages could be automated, because dpkg writes a well-formed logfile in /var/log/dpkg.log. But I noticed that there appearently is no library specialised at parsing that file. Its not a big deal, because the format of that file is really simple, but a proper library would be nice anyway.
And so I wrote such a library.
It basically takes a logfile, reads it line by line and stores each line parameterized into a generic object.

Its features include:

  • Parsing the logfile (obviously)
  • Storing each line in a logfile as a DPKG::Log::Entry object that holds informations about the entry type (e.g. action or status update), the package associated (if any), timestamp, verbatim line, line number etc.
  • Limiting the number of lines parsed to a time range specified by DateTime objects

Based on that, I wrote another library DPKG::Log::Analyse, which takes a log, parses it with DPKG::Log and then extracts the more relevant information such as installed packages, upgraded packages, removed packages etc.

This, in turn, features:
– Info about newly installed packages
– Info about removed packages
– Info about upgraded packages
– Info about packages which got installed and removed again
– Info about packages which stayed halfinstalled or halfconfigured at the end of the logfile (or defined reporting period)

These libraries are already uploaded to CPAN and packaged for Debian.
They passed the NEW queue very quickly and are therefore available for Sid:

http://packages.debian.org/sid/libdpkg-log-perl

As an example use (and for my own use case, as stated above), I wrote dpkg-report, which uses the module and a Template::Toolkit based template to generate a report about what happened in the given logfile.
It currently misses some documentation, but it works somehow like this:

Report for a single host over the full log:

dpkg-report

Report for a single host for the last two days:

dpkg-report –last-two-days

Report for multiple hosts (and logfiles):
The script expects that each log file has the name .dpkg.log so that it can guess the hostname from the system and can grab all such log files from a directory if a directory is specified as log-file arguments:

dpkg-report –log-file /path/to/logs

This will generate a report about all systems without any merging.

Report for multiple hosts with merging:

dpkg-report –log-file /path/to/logs –merge

This will do the following:

  • Report packages which where installed/removed/etc. on all systems seperate from the upgrades only done on the specific systems
  • If systems start with a common name and end on a number (e.g. mail1, mail2, mail3 etc.), report packages which where installed on all systems with that common name seperate from the upgrades only done on the specific systems
  • For a specific system list only the changes on that particular system and nothing else.

A (fictive) report could look some what like this:

dpkg-Report for all:
————————
Newly Installed:
abc
Removed:
foo
Upgraded:
bar (0.99-12 – 1.00-1)
dpkg-Report for test*:
————————
Newly Installed:
blub
dpkg-Report for test1:
————————
Upgraded:
baz (1.0.0-1 -> 1.0.0-2)
dpkg-Report for test2:
————————
Upgraded:
zab (0.9.7 -> 0.9.8)

Currently this report generator is only included in the source package or in the git(hub) repository of the library. I wonder if it makes sense to let the source package build another binary package for it.
But its only a 238 lines perl script with a dependency on the perl library so I’m unsure if it warrants a new binary package. What do others think?

Ubuntu considering critical bugs an „invalid“ bug?

I just discovered that bug report over at Ubuntu.
Short summary:
They have a script in upstart which is not meant to be run manually and if you do it will erase your whole file system. Additionally it seems that the fact that you shall not run that script is not communicated anywhere.

That alone isn’t the most spectacular about it. Bugs happen. Whats spectacular about it is how a Canonical employee and member of the TechBoard (for people who don’t know it: The people who decide about the technical direction Ubuntu takes) handles that bug. One quote of him to reflect it all:

Sorry, the only response here is „Don’t Do That Then“

So what we have here is a classical case of bad programming. The problem in question is that the script expects a certain environment variable to be set. Fair enough. However it does not check if its set at all and instead of failing or using a sensible default it simply sticks to undefined behaviour. What we have here is a classical programming mistake every beginner tends to do. People who start programming often forget (or don’t know) that every external value we rely on must be considered untrustworthy. Therefore a good practice is to check those values.

In this case someone decided that this is useless because they suffer from the wrong assumption that nobody ever calls it manually and the other wrong assumption that caller of the scriptwill always set the environment variable correctly. This is a double-fail.

Now the developer in question does not accept that (someone else indicated why the behaviour of the script is dangerous), he simply says that the bug is invalid. Thats really a pity.

Debian translations – EPIC LOL

As a german I’m used to strange translations in computer context.
I saw it back when I was using Microsoft products and I regulary stumble upon it on Debian systems. But whats actually kind of funny:Debian is outstanding in that regard.
An example:
Debian has that „wonderful“ package manpages-de which provides a german manpage for ps.
It contains a formulation that is so inaccurate and not really funny:

„–cumulative Daten von toten Kindern einbeziehen (als Summe zusammen mit den Eltern)“

For the English only speakers: That can be roughly translated to „Collect data from dead children
(summed with their parents)“. For me this somehow sounds like we are a butcher OS and so I reported #495441 but nobody cared about it yet.

Today I had another WTF on german translations because when I wanted to upgrade my system I read the following sentence:

„Sind Sie sich sicher, dass Sie die oben genannten Pakete installieren bzw. aufrüsten wollen?“

Cold war, anyone?

Well its a classical case of using a word-by-word translation instead of a meaning-orientied translation. Its a fact that ‚upgrade‘ can be translated to ‚aufrüsten‘ (as done above) but in that case its just not appropriate. As a matter of fact most german people I’m aware of would actually confuse the meaning with a meaning such as in a military context (therefore the „Cold war, anyone?“ text above). And just btw. if we would rely on computing power to translate it back into English we would get:

Are you sure itself that you want to install and/or rig the packages specified above?

So it seems that Babelfish has a similar pereception.

Re: Vimoutliner… finally!

Hi Matthew,

you don’t have comments in your blog, so I feel the urge to reply to your post this way.
I recently took over the vim-vimoutliner package, unfortunately too late to fix it for Lenny.

> However, the Debian vimoutliner package (in combination with the awesomely undocumented
> vim-addons mess) is an absolute pain to get going, requiring far more swearing and prodding
> than seems necessary. Since I don’t do it often enough, too, I keep forgetting how, and today
> things came to an absolute head when I very nearly flung my netbook across the room as it
> singularly refused to do my outliner bidding.

I think the situation of the plugin in Debian has improved. It still needs vim-addons to be enabled, but that is because it is simply better to have an opt-in/opt-out system for each user on a system instead of simply enabling it systemwide. I don’t think that calling vim-addons ‚awesomly undocumented‘, because the manpage is quiet good with respect to the use case of the application. What really is wrong about the Lenny package (except some technical flaws, like modifying the upstream tarball and not documenting it) is the fact, that it misses the documentation for using vim-addons. That has changed in the squeeze version and shouldn’t be a problem there.

Indeed in the current package, there is eventually one piece of documentation missing (which I got alread y prodded about by a user):

How to NOT improve relationships to your peer projects

I don’t know if Aaron is an Ubuntu Developer. I hope not. And I hope that no one of the commenters is one. But this post and especially the comments to it (because the article may have some justification) is certainly not a way to improve the relationship between projects like Ubuntu and the projects its based on.

Yes, it is true, that we have a rough tone in our mailing lists and yes it is sad that every now and then topics pop up that need to be discussed till somebody is hurt and leaves (the project). I certainly hope that this can be improved over the time, but most likely this is not something which breaks the project. Who knows what its worth. We have a large community, very much different opinions and therefore conflicts. Possibly it enhances the quality of our distribution, because things that certainly would make our product worse (e.g. lowering quality standards) won’t get done, because they’ll get a strong opposition.

But whats the point with the innovations, Aaron? Is it the job of a distribution to serve with innovations? A distribution that serves as a base for more then 10 other distributions? A distribution that is known to care a lot about freedom and is known (and appreciated) for a high quality standard. I don’t think we need to serve innovations. We served innovations back in a time where it was needed. Where things were missing. Like a good package management, like conf.d directories. Apart from this I think there were enough innovations in the past years. Possibly not that user-visible and trendy as a graphical virtualization frontend. What about xen-tools for example?
Now its all about reinventing the wheel or developing frontends. Launchpad for example is just reinventing the wheel. Savannah exists, GForge exists. Bazaar is just reinventing the wheel. Git exists. So does Mercurial.

But whats most sad about your posts and the comments is that you (and your commentors) seem to share the opinion that all Debian does is packaging software and nothing more. According to you Debian developers do not fix bugs, forward patches, file (important) wishlist bugs, encourage upstreams to remove horribly broken licensed software, improve their own software and tools (dpkg, aptitude, apt-get, the devscripts, ….), do integration work (alternatives, $EDITOR usage, …), documentation work (writing manpages). The reality is that there are many small and greater things driven by Debian Developers that serve the whole community. Additional to the ground work that your favorite operating system is based on and without which it could not exist.
Its just not beeing doped like it is done by companies with their marketing departments that actually earn money with what they do. Take the projects you named. They all have one. They all have paid developers.

In any way such a ranting (with no real constructiveness) will not serve the relationships between other projects and Debian.

Re: How many bugs have you fixed today?

Bastian Venthur notices how the „How many bugs have you fixed today“ question is used as a killer-argument against critism at the release-process.

I must confess that I also don’t like this attitude of people to use such arguments as a form to silence others. It is something that only people should do, who enjoyed bad manners.
But OTOH it is really sad to see that always the same people are fixing rc bugs. I can only guess, but it seems like out of over 1000 developers eventually 100 are actively involved in the squashing of release-critical-bugs. That is a shame. And people who try to blame the release team for the consequences happening because of this only aren’t really helpful, too. Its not like having a release team is meant to be a „5 people to rule/rescue the world“ approach.

Bastian, the problem with your attitude is, that you adduct such arguments with it. If its not visible that you do anything more to get Lenny out, except pointing with the finger at the release team, saying „Ha! Ha!“, and speculating how we will fail to release Lenny timely, then you shouldn’t wonder. You should remember that you are a part of the „We“, which is failing, when we as a project miss yet another release date.

And yes, I do agree that our release process is not really ideal and I think we should further push changes to optimize it (which already happened, given that the unblock-policy was way less strict as the years before), but where is your constructive contribution to that?