Planet Exgentoo

Ciaran McCreesh: UIs for Parallelism

Mon, 27 Oct 2008 14:56:43 +0000

Getting the UI right for parallel execution is tricky.

Make, for example, doesn’t even try. It just displays output from all the children all mixed together, and relies upon commands prefixing their output with a filename or similar identifier. This generally works for what make needs to do, but it’s a bit too crude for some things.

For the secret “not supposed to use it” parallelism in Paludis, we’re currently using something slightly more sophisticated: each line of output from each child job is prefixed by the job’s name. So you’ll see things like:

sync unavailable> wget -T 30 -t 1 -O /var/tmp/paludis-tarsync-cvh9gX/exherbo_repositories.tar.bz2 http://git.exherbo.org/exherbo_repositories.tar.bz2
sync unavailable-unofficial> wget -T 30 -t 1 -O /var/tmp/paludis-tarsync-kvEzRZ/exherbo_unofficial_repositories.tar.bz2
sync unavailable-unofficial> --14:24:06--  http://git.exherbo.org/exherbo_unofficial_repositories.tar.bz2
sync unavailable-unofficial>            => `/var/tmp/paludis-tarsync-kvEzRZ/exherbo_unofficial_repositories.tar.bz2'
sync unavailable> --14:24:06--  http://git.exherbo.org/exherbo_repositories.tar.bz2

Something like this is necessary because the commands involved don’t do their own prefixing. But whilst it’s an improvement, the output’s rather cluttered and it’s hard to work out what’s going on. I’ve been experimenting with something more like this (slowdown induced artificially to make the point):

Starting sync

Repository                    Status                Pending Active  Done
-> alsa                       starting              16      1       0
-> ciaranm                    starting              15      2       0
-> ferdy                      starting              14      3       0
-> compnerd                   starting              13      4       0
-> arbor                      starting              12      5       0
-> compnerd                   success               12      4       1
-> gnome                      starting              11      5       1
-> alsa                       failed                11      4       2
    ... fatal: The remote end hung up unexpectedly
-> hardware                   starting              10      5       2
-> ferdy                      success               10      4       3
-> media                      starting              9       5       3
-> gnome                      success               9       4       4
-> mozilla                    starting              8       5       4
-> hardware                   success               8       4       5
-> pioto-exheres              starting              7       5       5
-> media                      success               7       4       6
-> python                     starting              6       5       6
-> pioto-exheres              success               6       4       7
-> rbrown                     starting              5       5       7
-> mozilla                    success               5       4       8
-> texlive                    starting              4       5       8
-> ciaranm                    active                4       5       8
-> python                     success               4       4       9
-> unavailable                starting              3       5       9
-> texlive                    success               3       4       10
-> unavailable-unofficial     starting              2       5       10
-> rbrown                     success               2       4       11
-> unwritten                  starting              1       5       11
-> arbor                      active                1       5       11
-> unwritten                  success               1       4       12
-> x11                        starting              0       5       12
-> ciaranm                    active                0       5       12
    ... Initialized empty Git repository in /var/db/paludis/repositories/ciaranm/.git/
-> unavailable                active                0       5       12
    ... Literal data: 0 bytes
    ... Matched data: 0 bytes
    ... File list size: 749
    ... File list generation time: 0.001 seconds
    ... File list transfer time: 0.000 seconds
    ... Total bytes sent: 771
    ... Total bytes received: 26
    ...
    ... sent 771 bytes  received 26 bytes  1594.00 bytes/sec
    ... total size is 57557  speedup is 72.22
-> unavailable-unofficial     active                0       5       12
    ... Literal data: 0 bytes
    ... Matched data: 0 bytes
    ... File list size: 110
    ... File list generation time: 0.001 seconds
    ... File list transfer time: 0.000 seconds
    ... Total bytes sent: 132
    ... Total bytes received: 26
    ...
    ... sent 132 bytes  received 26 bytes  316.00 bytes/sec
    ... total size is 4013  speedup is 25.40
-> arbor                      active                0       5       12
-> unavailable                success               0       4       13
-> unavailable-unofficial     success               0       3       14
-> x11                        active                0       3       14
-> ciaranm                    active                0       3       14
-> ciaranm                    success               0       2       15
-> x11                        success               0       1       16
-> arbor                      success               0       0       17

 * Cleaning write cache for ebuild format repositories...
 * Done cleaning write cache for ebuild format repositories

 * No unread news items found

Sync results

* alsa:                       sync of '/var/db/paludis/repositories/alsa' from 'git://git.exherbo.org/demonstrate/failure/alsa.git' failed (paludis::SyncFailedError)
    Log file:                 /var/log/sync-alsa.1225118654.log
* arbor:                      success
* ciaranm:                    success
* compnerd:                   success
* ferdy:                      success
* gnome:                      success
* hardware:                   success
* media:                      success
* mozilla:                    success
* pioto-exheres:              success
* python:                     success
* rbrown:                     success
* texlive:                    success
* unavailable:                success
* unavailable-unofficial:     success
* unwritten:                  success
* x11:                        success

In particular:

Output is automatically logged to a file, rather than dumping everything to the screen. That file can be automatically removed if the job succeeds.
A summary of what’s going on is displayed every time a job starts or finishes.
Every ten seconds (for some value of ten), if a job hasn’t finished, we automatically display the last ten (for some other value of ten) lines of its output (but not any output we’ve already displayed) to the screen, along with a note that it’s still active. We also tail the log if a job fails.

This seems to be a comfortable balance between not showing anything except job statuses (which for jobs that can take an hour to run might well leave you wondering whether it’s hung) and just dumping everything to the screen. Now it’s just a case of finding appropriate values of ten…

Tagged: paludis

Ciaran McCreesh: xorg-x11 hackery for a Logitech RX1000 Mouse

Fri, 24 Oct 2008 09:23:50 +0000

The Logitech RX1000 mouse has a scroll wheel that can tilt left or right. This generates HWHEEL events by default, which are pretty much useless. I find it more useful to have the tilt generate WHEEL events (scroll up and down), so I can press and hold left or right rather than repeatedly spinning the mouse wheel.

The wheel can also be pressed to get a middle click, but if I don’t concentrate I end up scrolling as well as middle clicking. There’s another button with a magnifying glass icon on it just below the scroll wheel which is more useful, which usually shows up as button 8. By making this button 2 we can use that for X11 paste, open-in-new-tab in Firefox and so on.

So I don’t forget how to get this working:

Section "InputDevice"
    Identifier  "Mouse1"
    Driver      "evdev"
    Option      "Name" "Logitech USB Optical Mouse"
    Option      "HWHEELRelativeAxisButtons" "4 5"
EndSection

Unfortunately, evdev doesn’t seem to have a ButtonMapping equivalent, so we have to fall back to xmodmap to do the button remapping. Doubly unfortunately, this makes pressing the middle mouse button useless.

pointer = 1 6 3 4 5 2 7 8 9 10

Thanks Nicolas.

Tagged: hardware, linux, logitech, mouse, rx1000, xorg-x11

Ciaran McCreesh: Paludis is about Choices

Fri, 24 Oct 2008 09:23:05 +0000

Paludis 0.32 will include a new mechanism called ‘choices’. Examples of choices on Gentoo include use flags, the various USE_EXPAND settings (linguas, video_cards and so on) and a shiny new setting called build_options. For use flags and the USE_EXPAND settings, there’s no change from a user perspective, and you can carry on using use.conf as normal — choices are merely a more general mechanism for handling package settings.

On Exherbo, things do change a bit, and if you’re one of those naughty people who ignored the “Exherbo has no users” warning you’d better have been paying attention.

So what’s this build_options thing? You’ll see it cropping up for most packages:

* net-print/cups::gentoo [U 1.3.8-r1 -> 1.3.8-r2] <target>
    Reasons: app-text/ghostscript-gpl-8.62:0::installed, net-print/foomatic-filters-3.0.20080507:0::installed, 2 more
    X acl -avahi -dbus -gnutls -java jpeg -kerberos -ldap pam perl -php png ppds python -samba
    -slp ssl -static -tiff -xinetd -zeroconf LINGUAS: -de en -es -et -fr -he -id -it -ja -pl -sv
    -zh_TW build_options: -optional_tests split strip

Packages using 0-based EAPIs will usually have three build options:

optional_tests. This handles what --checks used to do. If this choice is set, src_test will be run.
split. This used to be --debug-build split.
strip. This used to be --debug-build internal.

Packages using kdebuild-1 or exheres-0 will have recommended_tests instead of optional_tests, which defaults to being enabled rather than disabled.

Despite what you might expect from older Paludis versions, these are not use flags. You can set them in use.conf, though — this means you can select whether to run tests or whether to keep debug information on a per-package basis, which was tricky to do with the old commandline options.

Tagged: choices, exherbo, gentoo, paludis

Ciaran McCreesh: Path and installable-to dependencies

Thu, 16 Oct 2008 14:55:44 +0000

A while ago, Paludis got from-repository dependencies. 0.32 will extend this further with path and installable-to dependencies.

Path dependencies are fairly simple: cat/pkg::/ will match cat/pkg installed at root /. A full path can be used too, so cat/pkg::/var/mychroot will work (as well as anything involving non-/ roots works, anyway…).

Installable-to dependencies take a bit more explaining. There are two kinds of installable-to dependency: repository and path. An installable-to-repository dependency looks like cat/pkg::repo? and is matched by any cat/pkg that could be installed to the repository named repo. An installable-to-path dependency looks like cat/pkg::/? and is matched by any cat/pkg that could be installed to any repository with root /.

Installable-to dependencies will not match masked packages. To specify “packages that would be installable, ignoring masks”, use a double question mark, like cat/pkg::/??. At present the double question mark ignores all masks, even those that cannot be overridden (unsupported EAPI, unavailable or unwritten). This may change to only include overridable masks; I’m not sure which is better behaviour yet.

Tagged: paludis

Ciaran McCreesh: Paludis 0.30.3 Released

Fri, 10 Oct 2008 18:28:57 +0000

Paludis 0.30.3 has been released:

Tests for 0.30.2 would fail on some systems. This is now fixed.

Tagged: paludis, release

Ciaran McCreesh: Paludis 0.30.2 Released

Thu, 09 Oct 2008 22:24:12 +0000

Paludis 0.30.2 has been released:

Work around ebuilds that treat cwd as secure.

Tagged: paludis, release

Ciaran McCreesh: Dealing with Unwritten Packages

Mon, 06 Oct 2008 21:57:14 +0000

A while ago Exherbo came up with a solution for the “large number of repositories” problem. It turns out this solution works rather well, and I’m now convinced that anyone rushing off and assuming that Git on its own will magically solve anything other than CVS sucking is going in the wrong direction.

Paludis also has ways of tracking packages installed by hand. Whilst useful on Gentoo, this really comes into play on Exherbo, where we don’t want to end up with a massive unmaintainable tree full of packages used by only two people.

The logical next step is tracking packages that don’t exist yet.

An unimaginative person might think that the way to solve this would be to have a bug for each requested package. But that gets messy — it’s not integrated with the package manager, and requires considerable extra effort from the user. A while ago Bernd suggested something more interesting: tracking unwritten packages in a repository in a similar way to how we do unavailable packages.

The implications:

Most obviously, we can keep track of things we want that haven’t been written yet in a way that doesn’t involve leaving the package manager to look things up.
We can also use it to track version bumps that we know will take a while. Doing paludis --query '>=hugescarypackage-4', say, would show that it’s being worked upon.
We can also depend upon things that don’t exist, rather than leaving incomplete dependency strings around. This is fine in at least two cases — if a dependency is conditional upon an option that should probably be implemented but isn’t yet, we can add the option and make it unusable. And we can handle obscure suggested dependencies (e.g. git has lots of optional dependencies upon weird perl modules, so we can say “if you want support for git-send-email with SSL, you need to write such and such a package”).
Bored developers could simply paludis --query '*/*::unwritten' and get ideas for what to do.

Adding support to Paludis for this only took a couple of hours. So now Exherbo developers can use this:

format = unwritten
location = /var/db/paludis/repositories/unwritten
sync = git://git.exherbo.org/unwritten.git
importance = -100

And see this:

$ paludis -q genesis
* sys-apps/genesis
    unwritten:               (1.0)X* {:0}
    Description:             The daddy of all init systems
    Homepage:                http://www.exherbo.org/
    Comment:                 We need an init system.
    Masked by unwritten:     Package has not been written yet

As with unavailable, installing behaves sensibly:

$ paludis -pi genesis
Building target list...
Building dependency list...
Query error:
  * In program paludis -pi genesis:
  * When performing install action from command line:
  * When executing install task:
  * When building dependency list:
  * When adding PackageDepSpec 'sys-apps/genesis':
  * All versions of 'sys-apps/genesis' are masked. Candidates are:
    * sys-apps/genesis-1.0:0::unwritten: Masked by unwritten (Package has not been written yet)

Those interested in the repository format can browse the tree. The observant might notice that the file format is quite similar to the one used by unavailable repositories, and wonder whether I was feeling lazy and swiped a load of code rather than implementing a new repository from scratch.

In the spirit of silly buzzwords, one could argue that this increases distributivity and democratisation of the repository, since there’s nothing to stop motivated users from creating their own wishlist repositories. It wouldn’t be hard to make a simple web interface that lets users request and vote on packages and version bumps, automatically generating a repository that anyone can use. But fortunately Exherbo is a dictatorship and has no users, so we don’t have to put up with that kind of nonsense.

Tagged: exherbo, gentoo, paludis, unavailable, unpackaged

Alexander Færøy: Hello from Open Source Days

Sat, 04 Oct 2008 08:10:05 +0000

First picture from Open Source Days:

Ciaran McCreesh: EAPI 2: doman language support

Wed, 01 Oct 2008 11:44:35 +0000

This is the final post in a series on EAPI 2.

The doman helper is one of those pesky little beasts that makes specifying EAPI behaviour formally such a nuisance (although it is nowhere near as horrible as dohtml). EAPI 2 makes it even peskier.

I’ll try that again.

The doman helper makes writing ebuilds substantially easier by automagically doing the right thing when installing manual pages, freeing the developer from having to care about manual sections. EAPI 2 makes doman even more useful by making it aware of language codes as well as sections.

The painful details are available in PMS, but basically this will now ‘do the right thing’:

doman foo.1 foo.en.1 foo.en_GB.1

Previously only the first of the items would go to the right place.

This one’s a Gentoo innovation; see bug 222439 for its history. It was shamelessly stolen for exheres-0, but was too late for kdebuild-1.

Tagged: eapi, eapi 2, ebuild, gentoo, paludis

Ciaran McCreesh: EAPI 2: default_ phase functions and the default function

Mon, 29 Sep 2008 12:06:36 +0000

This post is part of a series on EAPI 2.

With EAPIs 0 and 1, if you want to add something to, say, src_unpack, you have to manually write out the default implementation and then add your code. This is easy to screw up — developers are highly prone to getting the quoting wrong and forgetting which functions do and do not want a || die on the end.

EAPI 2 makes the default implementation of phase functions available as functions themselves. These functions are named default_src_unpack, default_src_configure and so on.

Typing out default_src_compile in full is pointless, though (especially since it’s illegal to call phase functions or default phase functions from other phase functions). So we also introduce the special default function, which calls whichever default_ phase function is appropriate for the phase we’re in. Thus:

src_compile() {
    default
    if useq extras ; then
        emake extras || die "splat"
    fi
}

Both features first appeared in exheres-0.

An alternative proposal (I think it came from the Pkgcore camp) was to make all EAPI default implementations available through functions named like eapi0_src_compile, eapi1_src_compile and eapi2_src_compile. This proposal was rejected because various Paludis people moaned about it not making sense or having any legitimate use cases (the ‘obvious’ use cases don’t work if you think them through), and no-one stood up to defend it.

Tagged: eapi, eapi 2, ebuild, gentoo, paludis

Ciaran McCreesh: EAPI 2: src_configure and src_compile

Mon, 29 Sep 2008 12:06:28 +0000

This is post five in a series describing EAPI 2.

EAPI 2 splits src_compile into src_configure and src_compile. Like src_prepare, it’s mostly a convenience thing to reduce copying default implementations, although in this case it also makes it easier to hook in code in between configure and make being run.

The default src_configure implementation behaves like this:

src_configure() {
    if [[ -x ${ECONF_SOURCE:-.}/configure ]]; then
        econf
    fi
}

This is the first half of EAPI 1’s src_compile, not the non-ECONF_SOURCE-aware EAPI 0 version.

The default src_compile implementation is reduced accordingly:

src_compile() {
    if [[ -f Makefile ]] || [[ -f GNUmakefile ]] || [[ -f makefile ]]; then
        emake || die "emake failed"
    fi
}

The split configure / compile setup was first used in exheres-0, which uses more elaborate default implementations. Like src_prepare, it was considered but rejected for kdebuild-1 because of eclass difficulties.

Tagged: eapi, eapi 2, ebuild, gentoo, paludis

Ciaran McCreesh: EAPI 2: src_prepare

Sun, 28 Sep 2008 20:45:51 +0000

This is post four in a series describing EAPI 2.

EAPI 2 has a new phase function called src_prepare. It is called after src_unpack, and can be used to apply patches, do sed voodoo and so on. The default implementation does nothing.

This function is purely for convenience. It gets rather tedious copying out the default implementation of src_unpack just to add a patch in somewhere.

src_prepare was first introduced in exheres-0 (which has a more elaborate default implementation). It was considered but rejected for kdebuild-1 because making best use of it requires eclass awareness, and the packages using kdebuild-1 had to share eclasses with the main Gentoo tree.

Tagged: eapi, eapi 2, ebuild, gentoo, paludis

Ciaran McCreesh: EAPI 2: !! Blockers

Sun, 28 Sep 2008 20:45:21 +0000

This is part three of a series of posts describing EAPI 2.

Blockers are a nuisance for end users. It’s rarely obvious how to fix them or what they mean, and getting it wrong can leave systems unusable.

There have been various proposals on how to fix this. For exheres-0, we’re going to go with something like this:

DEPENDENCIES="
    !app-misc/superfrozbinator [[
        description = [ Can only have one frozbinator installed at once ]
        resolution = uninstall-blocked-after
        url = [ http://explain.example.org/?only-one-frozbinator ]
    ]]
    !dev-libs/icky [[
        description = [ Having icky installed breaks the build process ]
        resolution = [ manual ]
        url = [ http://explain.example.org/?myfroz-hates-icky ]
    ]]"

The user can then be presented with a list of things that would need to be uninstalled to resolve blockers, along with clear descriptions of why they need to do so. Once the user has explicitly accepted the uninstalls, the package manager could then safely perform the installs.

Unfortunately, annotations aren’t something that can be implemented for Portage any time soon. Instead, Portage has gone with a fairly horrible and dangerous semi-automatic block resolution system that sometimes removes blocked packages automatically (often screwing up the user’s system in the process). Whilst doing so, Portage changed the meaning of EAPI 0 / 1 blockers from “this must not be installed when we do the build” to “this must be uninstalled after we do the build”.

EAPI 2 introduces a new kind of blocker using double exclamation marks, like !!app-misc/other. This goes back to the old meaning of “this must not be installed when we do the build”, keeping !app-misc/other for “this must be uninstalled after we do the build”.

This does not, unfortunately, make the user any safer, but it does allow packages that really can’t have something installed at build time to say so.

Tagged: eapi, eapi 2, ebuild, gentoo, paludis

Ciaran McCreesh: EAPI 2: Use Dependencies

Sun, 28 Sep 2008 19:59:03 +0000

This is the second post in a series of posts describing EAPI 2.

Use dependencies have been needed for a very long time. They eliminate most of the built_with_use errors you see during pkg_setup, replacing them with an error that is seen at pretend-install time.

The first two real world trials of use dependencies were with Exherbo’s exheres-0 and Gentoo’s kdebuild-1. It became apparent that an awful lot of packages would end up with dependencies like:

blah? ( app-misc/foo[blah] ) !blah? ( app-misc/foo )
monkey? ( app-misc/foo[monkey] ) !monkey? ( app-misc/foo[-monkey] )
fnord? ( app-misc/foo ) !fnord? ( app-misc/foo[-fnord] )

Syntactically, that’s rather inconvenient. For exheres-0 and kdebuild-1, we added the following syntax:

[opt]: The flag must be enabled.
[opt=]: The flag must be enabled if the flag is enabled for the package with the dependency, or disabled otherwise.
[opt!=]: The flag must be disabled if the flag is enabled for the package with the dependency, or enabled otherwise.
[opt?]: The flag must be enabled if the flag is enabled for the package with the dependency.
[opt!?]: The flag must be enabled if the use flag is disabled for the package with the dependency.
[-opt]: The flag must be disabled.
[-opt?]: The flag must be disabled if the flag is enabled for the package with the dependency.
[-opt!?]: The flag must be disabled if the flag is disabled for the package with the dependency.

Dependencies could be combined by specifying multiple blocks, as in foo/bar[baz][monkey?].

For EAPI 2, Zac decided to go with an arbitrarily different syntax:

[opt]: The flag must be enabled.
[opt=]: The flag must be enabled if the flag is enabled for the package with the dependency, or disabled otherwise.
[!opt=]: The flag must be disabled if the flag is enabled for the package with the dependency, or enabled otherwise.
[opt?]: The flag must be enabled if the flag is enabled for the package with the dependency.
[!opt?]: The flag must be disabled if the use flag is disabled for the package with the dependency.
[-opt]: The flag must be disabled.

And to combine use dependencies, one uses a comma, as in foo/bar[baz,monkey?].

In both cases, the slot dependency must go before the dependency, so foo/bar:1[baz], not foo/bar[baz]:1. The use dependency goes after any version restrictions, so >=foo/bar-2.1:2[baz].

In both cases, it is illegal to reference a use flag that does not exist (including USE_EXPAND flags that are not explicitly listed in IUSE). So foo/bar[opt] when any version of foo/bar does not have opt in IUSE is illegal and has undefined behaviour, as is foo/baz[opt?] if either the owning package or foo/baz has no opt. For cases where only some versions of a package have a flag, use dependencies can be combined with version or slot restrictions.

From an implementation perspective: the package manager should not try to automatically solve unmet use dependencies. The package manager doesn’t know the impact of changing a use flag (changing some flags makes a system unbootable), so it can’t simply override the user’s choice. (Paludis will suggest an automatic reinstall if and only if the user has already modified their use.conf, so you don’t need to manually reinstall a dependency if you’re ok with altering the flags with which it is built.)

Tagged: eapi, eapi 2, ebuild, gentoo, paludis

Ciaran McCreesh: Paludis 0.30.1 Released

Sun, 28 Sep 2008 19:44:55 +0000

Paludis 0.30.1 has been released:

EAPI 2 support.

Tagged: paludis, paludis releases

Bryan Østergaard: Software Freedom Day + Planet Larry

Fri, 19 Sep 2008 18:51:05 +0000

Tomorrow is Software Freedom Day - a yearly event where people all over the world get together to celebrate free software, enjoy talks related to free software and just as importantly get to meet lots of people.

If you happen to be in Copenhagen tomorrow you can meet myself and several other people from SSLUG at Copenhagen Business school. SSLUG's SFD program includes talks on Free Software, Linux, Open Office and GIMP. Everybody else can look up their local Software Freedom Day events - there's more than 500 teams registered all over the world so there's probably going to be an event nearby.

And regarding Planet Larry.. Steve Dibb just announced that he's setting up a feed for retired Gentoo developers which is very good news in my opinion. Lots of retired developers blog and they often have interesting comments on things related to Gentoo or tips that other people can benefit from. And this way people can know whether the blog posts they're reading comes from a normal user or a retired developer. I would probably have prefered marking retired developers another way instead of having multiple feeds but I can see why some people wants to know who's who and I'd much rather have a seperate feed than nothing at all. Oops, I was a bit too quick - Exdevs are now going in the main feed instead and will be marked using colour or some other way instead of a seperate feed.

And since I've been having this discussion with Steve on and off for quite some time: Thank you Steve :)

Bryan Østergaard: Meet Exherbo at Open Source Days!

Wed, 03 Sep 2008 13:12:32 +0000

That's right! Exherbo will be at the Open Source Days conference october 3-4 and ticket sale opened a few days ago.

Open Source Days expects 2000+ visitors and have lots of talks ranging from open source project management to more technical networking talks as well as talks large scale open source deployment. I'll be talking about the Exherbo project of course - how it all began, the current ideas defining Exherbo and some of the technical features support those ideas. Besides my talk Exherbo will also have a booth where you can meet several of our developers.

I hope to meet lots of people at OSD and get to discuss different aspects of Exherbo - it's an exciting project with lots of interesting features and above all room for innovative new features and improvements.

Bryan Østergaard: Credit where credit is due

Tue, 19 Aug 2008 08:44:13 +0000

All open source developers get for their hard work most of the time is credit - other people knowing they did some hard work and sometimes even other people acknowledging and thanking them for it.

Anybody who's been doing open source development for a few years knows this which is why I'm extremely surprised to see Diego "flameeyes" Pettenò trying to hide other developers work.

In one of his many blog posts Diego talks about ABI dependencies and how that's important for cross compiling among other things. As it turns out I agree this is important and more to the point, David Leverton (a Paludis and Exherbo developer) pointed out quite a bit of prior work being done on this exact problem in Exherbo.

David did so in the following comment:

You might want to talk to spb about that, he did some design work for Exherbo a while back (he's been too lazy to implement it so far, though). See http://git.exherbo.org/?p=exherbo.git;a=tree;f=scratch/multilib and http://dev.exherbo.org/~spb/labels.txt

This comment is nothing but helpful and friendly even if Diego might not want to talk to spb as they have some history - the comment still points directly at prior work being done in this exact area that Gentoo could easily benefit from and hiding it (by not allowing the comment) is dishonest at best.

I hope Diego is going to be a lot more cooperative in the future and actually follow the open source spirit instead of trying to mimic behaviour that's primarily seen from closed source companies.

Bryan Østergaard: Goals for Exherbo

Sun, 17 Aug 2008 23:08:26 +0000

Just a quick note that I wrote up a long list of things Exherbo still needs on http://lists.exherbo.org/pipermail/exherbo-dev/2008-August/000254.html.

Reading that list and any follow-up discussion is probably a good way to follow Exherbo development for the foreseeable future and might give some insights into what goes on behind the doors for those of you with no real experience in distribution development.

Bryan Østergaard: FOSS Aalborg videos released!

Fri, 15 Aug 2008 20:06:39 +0000

Some of you might remember me doing a presentation on Exherbo and primarily our planned initsystem a few months ago at the FOSS Aalborg conference in Denmark.

Most of the videos are now available including the one of my presentation. The presentation isn't highly technical but I talk a lot about why I don't think other initsystems solve the problems very well as well as what I expect from an initsystem. The video should be fairly interesting to all the people who wasn't at the FOSS Aalborg conference.

Hope you like the presentation(s) and thanks to the FOSS Aalborg organisers for getting the videos online and for having a great conference.

Alexander Færøy: Inkpot theme for Mutt

Sun, 03 Aug 2008 21:36:43 +0000

I am using Mutt as my e-mail client and I have done that ever since I started using Unix. I have for many months been annoyed by my very ugly theme that I created when I was around 13, but I have never actually spend a few minutes on making something shiny that didn’t piss me off.

Inspired from Ciaran McCreesh’s Vim theme called Inkpot, I decided to create an inkpot’ish theme for Mutt and here is the result that you can put in your .muttrc:

color   attachment  color30 color80
color   header      color10 color80 "^(From|Subject|cc|date|To|X-Spam-Level|User-Agent|X-Mailer):"
color   signature   color39 color80
color   tree        color26 color80
color   message     color26 color80
color   status      color85 color81
color   normal      color78 color80
color   error       color79 color64
color   indicator   color80 color73
color   markers     color26 color80
color   index       color64 color80 ~D
color   index       color64 color80 ~F
color   index       color30 color80 ~T
color   tilde       color80 color80

color   body        color10 color80 "(http|https|ftp|news|telnet|finger|irc)://[^ \">\t\r\n]*"
color   body        color10 color80 "mailto:[-a-z_0-9.]+@[-a-z_0-9.]+"
color   body        color26 color80 "[;:=][-][)/(|]"

color   quoted      color52 color80
color   quoted1     color22 color80
color   quoted2     color71 color80

Note: This probably only works in urxvt since that’s what I am using

There is, of course, a screenshot:

Mutt

Ciaran McCreesh: My Exherbo Supplemental Repository

Wed, 30 Jul 2008 20:06:33 +0000

I’ve tidied up my Exherbo supplemental repository (which is what Gentoo people confusingly call an ‘overlay’) and published it for the foolish.

Current goodness is:

hddtemp. This is mostly as an experiment in handling regularly updated data files.
detox
An updated imlib2 nicked from rbrown, who needs to lrn2src_uri
Fluxbox SCM, based upon (and I feel very very dirty for doing this) ahf’s 1.0.0 packages.

Expect these to be neglected and / or broken irregularly.

Ciaran McCreesh: Paludis 0.28.1 Released

Thu, 24 Jul 2008 13:27:26 +0000

Paludis 0.28.1 has been released:

Work around compiler issues that eventually lead to unpack dying when it shouldn’t on certain EAPIs on certain systems (ticket:622).

Ciaran McCreesh: GNU make Target Specific Variables are Dumb

Tue, 22 Jul 2008 19:41:56 +0000

Let’s say we have a Makefile using some target-specific variables:

$ cat Makefile
foo = global
b : foo = for-b

all : d
        @cat d

a :
        @echo a:$(foo) > a

b : a
        @{ cat a ; echo b:$(foo) ; } > b

c : a
        @{ cat a ; echo c:$(foo) ; } > c

d : b c
        @cat b c > d

Then we can get different results depending upon how we make the ‘all’ target:

$ rm a b c d ; make
a:for-b
b:for-b
a:for-b
c:global
$ rm a b c d ; make a b c d all
a:global
b:for-b
a:global
c:global

This is considered a feature:

There is one more special feature of target-specific variables: when you define a target-specific variable that variable value is also in effect for all prerequisites of this target, and all their prerequisites, etc. (unless those prerequisites override that variable with their own target-specific variable value).

In practice, it’s a pain in the ass. Using Quagmire, you’re supposed to be able to do this:

noinst_SHARED_LIBRARIES = libone.so libtwo.so libthree.so

libone.so_SOURCES = one.c
libone.so : LDFLAGS = -Wl,-soname,libone.so
libone.so_LIBS = libthree.so

libtwo.so_SOURCES = two.c
libtwo.so : LDFLAGS = -Wl,-soname,libtwo.so
libtwo.so_LIBS = libthree.so

libthree.so_SOURCES = three.c

Which leads to some rather bizarre behaviour:

$ make clean all &>/dev/null ; scanelf -S libthree.so
 TYPE   SONAME FILE
ET_DYN libone.so libthree.so 

$ make clean libone.so &>/dev/null ; scanelf -S libthree.so
 TYPE   SONAME FILE
ET_DYN libone.so libthree.so 

$ make clean libtwo.so &>/dev/null ; scanelf -S libthree.so
 TYPE   SONAME FILE
ET_DYN libtwo.so libthree.so 

$ make clean libthree.so &>/dev/null ; scanelf -S libthree.so
 TYPE   SONAME FILE
ET_DYN  libthree.so

I can think of a few not very nice ways of getting around this. Hopefully someone will come up with something better…

Ciaran McCreesh: Hot Fresh Shiny Config Files

Wed, 16 Jul 2008 19:31:17 +0000

I’ve moved my configuration files repository from Subversion to Git, since I’m sick of trying to decide whether my laptop or desktop should hold the ‘master’ copy. For the lulz, I shall also push to GitHub at irregular intervals, so you shall no longer need to pester me to upload newer versions of my bashrc / vimrc / whatever to webspace somewhere. Instead, you can just:

git clone git://github.com/ciaranm/dotfiles-ciaranm.git

Or browse them online.

We’ll see whether this lasts…

Ciaran McCreesh: Git and Subversion information in the Bash Prompt

Tue, 15 Jul 2008 23:41:50 +0000

Here’s my variation on the “Git and Subversion status in the bash prompt” thing:

ps_scm_f() {
    local s=
    if [[ -d ".svn" ]] ; then
        local r=$(svn info | sed -n -e '/^Revision: \([0-9]*\).*$/s//\1/p' )
        s="(r$r$(svn status | grep -q -v '^?' && echo -n "*" ))"
    else
        local d=$(git rev-parse --git-dir 2>/dev/null ) b= r= a=
        if [[ -n "${d}" ]] ; then
            if [[ -d "${d}/../.dotest" ]] ; then
                if [[ -f "${d}/../.dotest/rebase" ]] ; then
                    r="rebase"
                elif [[ -f "${d}/../.dotest/applying" ]] ; then
                    r="am"
                else
                    r="???"
                fi
                b=$(git symbolic-ref HEAD 2>/dev/null )
            elif [[ -f "${d}/.dotest-merge/interactive" ]] ; then
                r="rebase-i"
                b=$(<${d}/.dotest-merge/head-name)
            elif [[ -d "${d}/../.dotest-merge" ]] ; then
                r="rebase-m"
                b=$(<${d}/.dotest-merge/head-name)
            elif [[ -f "${d}/MERGE_HEAD" ]] ; then
                r="merge"
                b=$(git symbolic-ref HEAD 2>/dev/null )
            elif [[ -f "${d}/BISECT_LOG" ]] ; then
                r="bisect"
                b=$(git symbolic-ref HEAD 2>/dev/null )"???"
            else
                r=""
                b=$(git symbolic-ref HEAD 2>/dev/null )
            fi

            if git status | grep -q '^# Changed but not updated:' ; then
                a="${a}*"
            fi

            if git status | grep -q '^# Changes to be committed:' ; then
                a="${a}+"
            fi

            if git status | grep -q '^# Untracked files:' ; then
                a="${a}?"
            fi

            b=${b#refs/heads/}
            b=${b// }
            [[ -n "${r}${b}${a}" ]] && s="(${r:+${r}:}${b}${a:+ ${a}})"
        fi
    fi
    s="${s}${ACTIVE_COMPILER}"
    s="${s:+${s} }"
    echo -n "$s"
}

This is added to PS1 as normal.

For Subversion, it gives us the revision, and a * if anything’s been modified.

For Git, we get quite a bit more. If we’re in the middle of a merge, interactive rebase or am, we get told so. If we’re in the middle of a bisect, we get question marks, since I’ve not had to bisect anything since I wrote that code and I haven’t implemented anything fancy for it. We also get the branch, and a combination of * for changed but not updated files, + for changed and committed files and ? for untracked files.

It’s generally sufficiently fast to not be annoying, although the initial cd into a large project can take a few seconds. It’s probably possible to drop to a single ‘git status’ call if performance matters, although the kernel does a pretty good job of speeding up subsequent runs.

Ciaran McCreesh: Edimax EW-7728In 802.11n (RaLink rt2860) with Linux 2.6.26

Mon, 14 Jul 2008 16:11:54 +0000

Being sick of wireless disconnecting every time it rained, I got my hands upon a cheap Edimax EW-7728In PCI 802.11n card. This uses a RaLink rt2860 (PDF) chip, which has vendor-supplied open source drivers available. Getting it to work with Linux 2.6.26 is slightly non-trivial, however.

First, we need the drivers. At the time of writing, that means 2008_0522_RT2860_Linux_STA_v1.6.1.0.tar.bz2.

Next, we need to fix a couple of things. First, the Makefile is dumb, and tries to install into /tftpboot:

From 82dc3a8737e4f97311f4f4fccd79ea55a319f1ea Mon Sep 17 00:00:00 2001
From: Ciaran McCreesh <ciaran.mccreesh@googlemail.com>
Date: Mon, 14 Jul 2008 16:47:00 +0100
Subject: [PATCH] Get your filthy paws off my /tftpboot

---
 Makefile |    2 --
 1 files changed, 0 insertions(+), 2 deletions(-)

diff --git a/Makefile b/Makefile
index 72bc933..aed3b00 100644
--- a/Makefile
+++ b/Makefile
@@ -80,11 +80,9 @@ LINUX:
 ifneq (,$(findstring 2.4,$(LINUX_SRC)))
        cp -f os/linux/Makefile.4 $(RT28xx_DIR)/os/linux/Makefile
        make -C $(RT28xx_DIR)/os/linux/
-       cp -f $(RT28xx_DIR)/os/linux/rt$(CHIPSET)sta.o /tftpboot
 else
        cp -f os/linux/Makefile.6 $(RT28xx_DIR)/os/linux/Makefile
        make  -C  $(LINUX_SRC) SUBDIRS=$(RT28xx_DIR)/os/linux modules
-       cp -f $(RT28xx_DIR)/os/linux/rt$(CHIPSET)sta.ko /tftpboot
 endif

 clean:
--
1.5.6.2

Next, dev->nd_net should now be dev_net(dev):

From 0878b37a40e2a7f466a74938920ff3751917eec3 Mon Sep 17 00:00:00 2001
From: Ciaran McCreesh <ciaran.mccreesh@googlemail.com>
Date: Mon, 14 Jul 2008 16:48:50 +0100
Subject: [PATCH] dev->nd_net is now dev_net(dev)

---
 os/linux/rt_main_dev.c |    2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)

diff --git a/os/linux/rt_main_dev.c b/os/linux/rt_main_dev.c
index 24604da..6a3471d 100644
--- a/os/linux/rt_main_dev.c
+++ b/os/linux/rt_main_dev.c
@@ -669,7 +669,7 @@ static NDIS_STATUS rt_ieee80211_if_setup(struct net_device *dev, PRTMP_ADAPTER p

 #if LINUX_VERSION_CODE >= KERNEL_VERSION(2,5,0)
 #if LINUX_VERSION_CODE >= KERNEL_VERSION(2,6,24)
-        device = dev_get_by_name(dev->nd_net, slot_name);
+        device = dev_get_by_name(dev_net(dev), slot_name);
 #else
                device = dev_get_by_name(slot_name);
 #endif
--
1.5.6.2

Finally, the driver is unusably noisy. Unless you want fifty-odd lines of kernel debug informatione every five seconds:

From 19c7a6895333624566775541fbc836e0c9208225 Mon Sep 17 00:00:00 2001
From: Ciaran McCreesh <ciaran.mccreesh@googlemail.com>
Date: Mon, 14 Jul 2008 16:47:43 +0100
Subject: [PATCH] You shut your dirty whore mouth

---
 include/rt_linux.h |    4 ----
 1 files changed, 0 insertions(+), 4 deletions(-)

diff --git a/include/rt_linux.h b/include/rt_linux.h
index dfe4ab5..079d256 100644
--- a/include/rt_linux.h
+++ b/include/rt_linux.h
@@ -325,10 +325,6 @@ extern ULONG    RTDebugLevel;

 #define DBGPRINT_RAW(Level, Fmt)    \
 {                                   \
-    if (Level <= RTDebugLevel)      \
-    {                               \
-        printk Fmt;               \
-    }                               \
 }

 #define DBGPRINT(Level, Fmt)    DBGPRINT_RAW(Level, Fmt)
--
1.5.6.2

There’s no Makefile install target, so you’ll need to sudo cp os/linux/*.ko /lib/modules/`uname -r`/kernel/ then sudo depmod -a before you can sudo modprobe rt2860sta. You’ll also need to sudo mkdir -p /etc/Wireless/RT2860STA/ then cp RT2860STA.dat /etc/Wireless/RT2860STA/, even though we’re not using that file for any configuration.

Gentoo’s init scripts, if you’re using them, try to be overly clever, and won’t be able to bring up the interface. So use something like this as your /etc/init.d/net.ra0:

#!/sbin/runscript

depend() {
    need localmount
    after bootmisc hostname net.lo net.lo0
    use isapnp isdn pcmcia usb wlan
}

start() {
    bash -x -c 'ifconfig ra0 up'
    bash -x -c 'iwconfig ra0 essid giant-space-monkey key "aa:bb:cc:dd:ee:ff:00:11:22:33:44:55:66" freq 2.417G'
    bash -x -c 'dhcpcd ra0'
}

stop() {
    :
}

restart() {
    :
}

And that appears to be sufficient.

Bryan Østergaard: From the department of wasting time

Thu, 10 Jul 2008 10:31:47 +0000

This is shiny but pretty much useless if you ignore the geek factor.. Not one to ignore the geek factor I've made a quick movie of arbor commits :)

There's movies of the Apache, Postgres, Eclipse and Python projects available on the authors homepage.

Alexander Færøy: Logo

Wed, 09 Jul 2008 16:41:16 +0000

Exherbo has finally got a logo thanks to Ida Jensen. Here it is:

You can find the svg file in the exherbo git repository.

Update: I completely forgot to thank the person who actually draw the logo. It was Christel from the Exherbo team who draw the initial cartoon and Ida then later turned it into a digital version! Sorry Christel, I owe you a cookie

Ciaran McCreesh: Paludis 0.28.0 Released

Mon, 07 Jul 2008 15:31:02 +0000

Paludis 0.28.0 has been released:

Compilers without tr1 memory, type traits and functional support are no longer supported. This means you, gcc-3.
Support for the Exherbo distribution.
Better detection of SCM versions.
Bug fixes: pkg_pretend now behaves correctly with resumes (ticket:604). Better support for Portage config files (ticket:597). Package dep spec parsing no longer has trailing hyphen issues (ticket:590).
New Selection + Filter + Generator interface using Environment, replacing the old PackageDatabase Query (ticket:559).
importare can now change file ownership to root (ticket:599).

Ciaran McCreesh: Hardware Vendor Hate List

Sat, 05 Jul 2008 08:41:19 +0000

Logitech. You helpfully claim to be able to provide replacement feet for keyboards free of charge — presumably, breakages are not exactly uncommon. Then you claim not to keep track of whether or not you have any spares for my year-old G15 keyboard, so you can’t give me any. And no, you won’t go and have a look to find out whether you do have any.
Whoever makes the push pins found on LGA 775 CPUs. Especially the kind that require so much force to lock that it’s a race between the motherboard and my finger to see which will break first.
Abit. You should be ashamed of the F-I90HD motherboard for all sorts of reasons, but today’s gripe is about the heatsink on the integrated graphics chip. I shouldn’t have to install a cooling fan onto the heatsink to stop the screen from going black when the CPU is under load.

Ciaran McCreesh: Bedtime Reading III

Sun, 29 Jun 2008 23:06:50 +0000

An interview with Bjarne Stroustrup
Google C++ Style Guide
Sustainable Energy - Without the Hot Air, David J.C. MacKay