Updated E17 packages

Now that I'm on my brand new cable connection, I can upload packages with 1Mbit/s - a vast improvement which allows me to keep the E17 Debian repository up to date more frequently than during the last four months.
The new packages include the E17 NetworkManager client (exalt) and its control module (emodule-exalt). Thanks to exchange the E17 user can now download and install themes right from the desktop shell.

At this point I'd like to thank Daniel (aka Quaker) for updating and adding Debian files in the E17 svn tree.

New E17 packages

After many weeks I finally updated the i386 and amd64 packages in the E17 Debian repository. The reason behind the delay is first and foremost my slow internet connection on which it took more than 15 hours (at 6Kb/s) to upload the new deb's. With that in mind I made sure that the new packages work prior to uploading them in order to avoid the pain of uploading fixes.

Nonetheless, due to many changes to the E17 source code since the last update, some issues were brought to my attention most of which are caused by outdated configuration directories. The previous packages are too old and its settings are not compatible. Even though purging the config dir is usually not strictly necessary, I recommend it for future updates if errors occur. Furthermore it might be necessary to remove old packages from this repository before installing the new ones because some libraries (i.e. evas) were reduced to less packages.

In general, I recommend the following upgrade procedure:

1. Backup your configuration directory (~/.e).
2. Upgrade to new packages.
3. Check if the configuration works with the new packages. If yes, skip to 13.
4. Remove the (previously backed-up) configuration directory.
5. Check if E17 works. If yes, skip to 13.
6. Remove all packages from this repository.
7. Run 'apt-get clean' or equivalent.
   
8. Reinstall E17 (i.e. by running 'apt-get install e17').
9. Check if E17 works. If yes, skip to 13.
10. Please make sure to set up pinning as described in the wiki.
11. Check if E17 works. If yes, skip to 13.
    
12. Please contact me.
13. Done.

Any feedback is welcome.

KDE 4.2

I'm usually changing desktop environments on a weekly to monthly basis, depending on their releases. On Gentoo that's a hell lot of fun and could take several hours whereas in Ubuntu it's done in a couple of minutes.
For me the three most usable and interesting desktop environments are KDE, Gnome and - of course - Enlightenment. There are other alternatives, most notably XFCE which offers it's own set of utilities, like file manager, widget set, etc. but I haven't used it intensely in years mostly because it's Qt-based, like KDE GTK-based, like Gnome, and thus isn't as interesting for me.
While Gnome looks most archaic in my opinion but is very well integrated in Ubuntu, KDE is slightly slower but offers a solid framework and a nice default theme. Enlightenment on the other hand offers by far the fasted window manager as well as its own set of graphic and system libraries. I won't go into details regarding Enlightenment, I already wrote enough about it; needless to say, it's my favourite.
Well, back to the main topic. Just a couple of hours after the official release of KDE 4.2 the Kubuntu team already offers updated packages in their experimental PPA repository (a speed which, by the way, reminds me a lot to Gentoo). Unsatisfied as I was about KDE 4.0 and 4.1, I gave it a shot, despite its 'experimental' tag. To my surprise it's a real joy and a huge step towards what I expected of the 4-series of KDE. Until now KDE 4 seemed to be slow and incomplete compared to KDE 3.5, with many feateres missing, such as custom key bindings which I so frequently use to start my most used applications. KDE 4.2 remedied this situation, ships with a brand new theme, a bunch of new desktop widgets (plasmoids), is much faster and I yet have to discover any bugs, not to mention all the bugs and glitches that have disappeared.
I hope the next revision will be as ground breaking.

Back to Ubuntu

Darn, I miss Gentoo already. But lately I found myself spending more time fixing or configuring my OS than actually using it. While this is usually really fun and greatly helps in understanding how things work under the hood it surely isn't desirable on the long run, especially when I want to get things done. Sure, there's always the option to use Gentoo "stable" but packages are simply too out of date and I usually end up using the ~amd64 keyword and a couple of overlays.

Going back to Ubuntu was a matter of 15 minutes - completely out of the league for Gentoo. In the peak of my Gentoo days I installed and configured a modern system in not less than 20 hours. Including actual "home folder" configuration that usually fills two entire days. I don't even dare to remember my first Gentoo installation...

Although Ubuntu's Gnome Desktop runs a little slower than Gentoo's compiler optimized binaries, it works flawlessly and includes enough optional packages to suit my needs. But what always surprises me anew is that fact that Ubuntu offers this certain "wow" effect due to its out-of-the-box working configuration. This is, of course, only possible because of Ubuntu's impressive community and thus manpower which other distribution simply lack. There's nothing easier than looking for a quick and simple solution in the Ubuntu forums or even the wiki.

Umlauts on the US keyboard with xmodmap

I'm totally dependent on my US layout keyboards, both on my stationary PC and my laptop because it proved to be my preferable way of coding, mainly because the majority of symbols used in most programming languages are easily accessible, sometimes even without the need for modifier keys (e.g. '{','}','\' and most of all '/'). Once I've tried the British English layout (back when I had the C1 'picture book' of Sony) which wasn't bad either and had some interesting ideas ('"' reachable via SHIFT+2) but it didn't convince me to make a full change.
I actually can even get used to QUERTZ-layouted keyboards (like the German or Swiss one) and other, non-coding-friendly QUERTY-layouts (like the Spanish one) quite fast because I get to use them from time to time when I use (read: fix) a friend's or family members' computer. But still, the US layout is my prefered choice and I wouldn't want to miss it on my own devices which was, by the way, one reason why I imported my last laptop from the states.
Nonetheless, every once in a while I need to use symbols like umlauts or the German eszett (ß) in order to write emails properly. That's why I'm using a couple of xmodmap rules to add that functionality in a convenient way: Umlauts are mapped to the ALT key and the corresponding vowel (e.g. ALT+u=ü), the same applies for the eszett (ALT+s=ß). My old rules stopped working some when because the keycode for the ALT key changed and thus interfered with other keys (in my case with the cursor keys). This is my current script which I'm using on X.org 1.5.2:

keycode 108 = Mode_switch
keycode 20 = minus underscore ssharp
keycode 34 = bracketleft braceleft udiaeresis Udiaeresis
keycode 47 = semicolon colon odiaeresis Odiaeresis
keycode 48 = apostrophe quotedbl adiaeresis Adiaeresis
keycode 26 = e E EuroSign EuroSign e E
keycode 30 = u U udiaeresis Udiaeresis u U
keycode 32 = o O odiaeresis Odiaeresis o O
keycode 38 = a A adiaeresis Adiaeresis a A
keycode 39 = s S ssharp ssharp s S

E17 migrates to subversion

After the E17 team has been busy converting the version control system from CVS to subversion I saw the need to update my build script to reflect this migration and finally adress a heap of bugs which have been around since the last version got released.
Along with various bugfixes, I've added functionality which further minimizes manual work needed to make packages compile. First of all, since configure.in files in subversion no longer generate Debian/changelog I've added a function which does just that: injecting the line necessary if needed. Furthermore there's now an option which generates a Debian/changelog file based on an adjustable amount of SVN log entries. The first two entries in the changelog.in of the emodules package might look like this:

emodules (@VERSION@-0cvs20080921) unstable; urgency=low

[ mekius ]
Fix history, thanks Sachiel :D

-- Falko Schmidt Thu, 11 Sep 2008 02:06:54 +0000

emodules (@VERSION@-0cvs20080921) unstable; urgency=low

[ kaethorn ]
remove the echo module.

-- Falko Schmidt Sat, 06 Sep 2008 09:46:58 +0000

Last but not least any source package which doesn't contain a Debian directory will automatically be ignored and thus not cause a build failure anymore. Packages either marked as not containing such a directory ("no debian dir") or which are mentioned in the DONTBUILDLIST variable ("skipped") will be marked accordingly in the "Check trunk" summary:



I've updated i386 and amd64 packages successfully and will from now on focus on updating and adding package descriptions in SVN. Downloads and installation instructions can be found here.

E17 sparc packages

I've compiled some sparc E17/EFL packages on a Sun Blade 1000 and uploaded them to the Debian repository. That means there are packages available for all those who need a fast desktop environment even on some of Sun Microsystem's older, but still very reliable workstations.

Usual installation procedures apply here as well.

E17 amd64 packages

Just an update to the previous post:

A new set of amd64 packages has been uploaded and should provide the same functionality as their i386 counterparts. Packages for alpha are planned for the following weeks, probably Sparc packages as well.

Big Debian E17 overhaul

As the PkgE Team is working hard to get E17 into Debian/experimental, I took some time to merge most of their packaging goodness into the debian/ folders of the enlightenment CVS tree. As a result the repository at debian.alphagemini.org now features higher consistency regarding package names and dependencies.
The major change involved renaming of -dev, -dbg and -bin packages of all libraries which now look like libfoo-dev instead of libfoo0-dev. The second change was the package name of enlightenment itself which is now available as e17. Furthermore libe-dbus is now called libedbus. libeet1 is now available from the official Debian repositories as well and should be compatible with the rest of the packages.
Due to all these changes installations will most probably break upon upgrade which can be prevented by uninstalling all previous packages from this repository beforehand. Afterwards a regular

apt-get install e17 emodules-all

should do the rest.

Apart from the above changes there are also some additions, namely three new emodules (execwatch, iiirk and notification) as well as the edje editor.

The AMD64 port will be updated accordingly as soon as the i386 port works flawlessly.

Thanks a lot to Guillaume, Sedat and Vit for reporting bugs and suggestions, and of course to the PkgE-Team for providing the package descriptions.

Further information and instructions can be found here.

ICC vs GCC-4.3

Since GCC-4.3.0 is about to be released I decided to take a look at its new intel Core 2 tuning and SSSE3 code generation by emerging the package found on Dirtyepic's overlay. I compared the time it would take to re-encode a video with ffmpeg and a WAV sample with oggenc. The video clip I used can be found here (1920x816 MOV, 1:46, 128.3MB) while the WAV file is just the extracted audio track thereof.

I used these four compiler collections and their CFLAGS:

1. GCC 4.1.2 (-march=nocona -O3 -pipe -msse3)
   
2. GCC 4.2.3 (-march=nocona -O3 -pipe -msse3)
3. GCC 4.3.0-pre20080302 (-march=core2 -O3 -pipe -mssse3)
   
4. ICC 10.1 20080112 (-O3 -xT -ipo -gcc)
   

My system's specs:

• Q6600(B3) @ 3.21GHz
• 400Mhz FSB (266Mhz northbridge strap)
  
• 2GB PC3-15000 1603Mhz (8-8-8-24)
  
• kernel 2.6.24-gentoo-r3 (kernel lock preemption and preemptible kernel model, 1000Hz timer freq, see config)

I recompiled the following packages with emerge after changing my environment to the appropriate compiler using gcc-config:

• x11-libs/libXau-1.0.3 USE="-debug"
  
• x11-libs/libXdmcp-1.0.2 USE="-debug"
• x11-libs/libXext-1.0.4 USE="-debug"
• x11-libs/libX11-1.1.3-r1 USE="ipv6 -debug -xcb"
• media-libs/libogg-1.1.3
• media-libs/faac-1.26-r1
  
• media-sound/lame-3.97-r1 USE="-debug -mp3rtp"
  
• media-libs/xvid-1.1.3-r3 USE="(-altivec) -examples"
  
• media-libs/x264-svn-20080301 USE="threads -debug"
• media-libs/a52dec-0.7.4-r5 USE="-djbfft -oss"
• media-libs/amrnb-7.0.0.0
• media-libs/faad2-2.6.1 USE="-drm"
• media-libs/libpng-1.2.25
• dev-libs/libxml2-2.6.31 USE="ipv6 python readline -bootstrap -build -debug -doc -examples -test"
• media-libs/libvorbis-1.2.0 USE="-doc"
• media-libs/speex-1.2_beta3 USE="ogg sse"
• media-libs/flac-1.2.1-r2 USE="cxx ogg sse -3dnow (-altivec) -debug -doc"
• media-libs/libtheora-1.0_beta2-r1 USE="encode -doc -examples"
• media-libs/freetype-2.3.5-r2 USE="X -bindist -debug -doc -utils"
• media-libs/giflib-4.1.6 USE="X -rle"
• media-sound/vorbis-tools-1.2.0 USE="flac nls ogg123 speex"
• media-video/ffmpeg-0.4.9_p20070616-r2 USE="X a52 aac amr doc encode ieee1394 imlib ipv6 mmx ogg sdl theora threads truetype v4l vorbis x264 xvid zlib (-altivec) -debug -network -oss -test"

All remaining system libraries which ffmpeg and oggenc might link to were compiled with gcc 4.3.0 (e.g. glibc).

Note on ICC: Multifile interprocedural optimizations didn't work for lame, flac, a52dec and faad2, where I needed to resort to single file interprocedural optimizations and thus used '-O3 -xT -ip -gcc'. Also, ICC didn't seem to compile ffmpeg. For that reason I needed to recompile libX11, libXau, libXdmcp and libXext with gcc-4.3.0 or else ffmpeg would complain about symbol lookup errors.

I used the following command for re-encoding the video clip:

ffmpeg -y -i 2744_trailer01-en_1920.mov \
-f avi -vcodec mpeg4 -b 800k -g 300 \
-bf 2 -acodec libfaac output.avi

I repeated it 5 times and got these results:

GCC-4.1.2: 437.24 sec
GCC-4.2.3: 436.98 sec
GCC-4.3.0: 436.17 sec
ICC 10.1: 429.72 sec

For ogg encoding I first extracted the audio track of the clip.

ffmpeg -y -i 2744_trailer01-en_1920.mov \
output.wav

and then encoded it with oggenc:

rm -f output.wav; oggenc output.wav

This command was repeated 30 times and resulted in the following times:

GCC-4.1.2: 217.00 sec
GCC-4.2.3: 216.97 sec
GCC-4.3.0: 206.90 sec
ICC 10.1: 191.91 sec

Doing the graphs I decided to truncate the bars and only show the relevant upper parts. Thus these graphs don't represent absolute values but demonstrate the differences in execution time between the code produced by each compiler collection:



It turns out that the GCC 4.3 branch yields quite a noticeable performance boost, probably thanks to its new Core 2 tuning option. ICC's optimizations are still unmatched and show that GCC could still need some improvement. After all ICCs lead in video encoding was most probably just caused by its shared libraries (e.g. flac) because ffmpeg itself was compiled with GCC (see above).

As a conclusion, GCC and especially the upcoming release produces code which is more than fast enough for a normal desktop system. Even with libraries that benefit greatly from ICC's vectorization techniques the advantage of ICC over GCC is negligible and wouldn't justify the time spent in recompilation and porting.

Overclocking the Q6600

Woot! I'm running a Q6600 at 3.01Ghz. Although it's the old B3 stepping, it seems to run smoothly at around 55°C under full load thanks to water cooling.
I didn't have to increase the core voltage, so it should remain stable and hopefully won't die on me too soon.

To monitor the temperatures I use ksysguard as shown here. It's very powerful in that it allows you to choose a multitude of ways to display system information. I'm mainly using the signal plotter because one can assign several sources to one plotter cell - in this example I'm graphing idle, nice, system and user load of a CPU core in one cell. In the second row ksysguard plots the temperatures in a scale from 0 to 100°C. I could even monitor remote systems running ksysguardd (e.g. via ssh). The latter only depends on some general purpose libraries and tools which is well suited for servers where I don't want to install the complete set of kde packages.

Folding@Home screen saver for KDE

It bugged me that there was no fancy visualization front end around for Folding@Home that yields more than a progress bar and whatever else it could extract from the unitinfo.txt file. Especially the PS3 client looks awesome (I've never seen the Windows client) so I thought it's time to do one myself. It didn't take me too long because I've already written a KDE screen saver for a molecule dynamics framework at University of Stuttgart/HLRS.

So, here it is. FAHss displays the current work unit as an OpenGL model. The configuration dialog is a main part of the application and aims to give the user a choice as to how it should handle rendering. It allows the user to choose to ...

• let the camera move (e.g. rotate around the center).
• set the speed with which the camera moves.
• select a background color.
• draw a grid box around the unit and choose its color.
• display the models solidly or as a wireframe.
• adjust the number of subdivisions on models to increase performance.

Furthermore the user can select which information to display as an OSD, e.g. the progress and the due date. The font type and color and its position can be specified. All changes being made in the dialog are directly reflected in the preview window it contains.

There are several features which aren't implemented yet (such as core status or CPU usage) but it's already usable. Maybe I'll set the project up on Google Code some when, but for now I'm hosting it on my own server. It can be checked out like this:

svn co http://svnro.alphagemini.org/fahss/

I've also set up a WebSVN service to track changes at http://svn.alphagemini.org

HDAPS patch for 2.6.24-rc8

I've been a Thinkpad fan since the first time I laid hand on one so my next portable companion had to be the model X61 which turned out to be exceptionally reliable (and portable), with all its features running in Linux including the fingerprint sensor and the hard disk Active Protection System (HDAPS). Since I've had reasons to use iwlwifi drivers for my 3945ABG Wifi card I chose to use the in-kernel driver provided by the upcoming 2.6.24 release. This kernel also supports the intel HDA sound card well. I prefer to have all drivers in one place other than having to compile them externally, so this was the way to go.
What's lacking was the HDAPS disk parking kernel patch which isn't yet included in the mainstream kernel. Up until release candidate 6 (don't know about 7) the patch 1077-002.patch found on thinkwiki.org worked after some fiddling with the line numbers. As of rc8 I needed to swap some functions around in order to make it compile, the resulting patch can be found here.

Maybe later I'll take a look at the 'error check fix' mentioned on thinkwiki.org as well.

dh_fixperms headaches

If you ever wondered why lines such as

install/mypackage:
chmod 4511 debian/tmp/usr/bin/my_setuid_binary

in your debian/rules file seem to be magically ignored in the resulting package, then look no further. It took me some time to figure it out, after trying all kind of combinations of post-install, pre-finalize and pre-install routines. In the end it was quite obvious.
By including the standard debhelper rules (/usr/share/cdbs/1/rules/debhelper.mk) there're a whole bunch of scripts being unleashed on your package and one of them is dh_fixperms(1) which "removes the setuid and setgid bits from all files in the package".
That's fine, but we want to override this with a statement in debian/rules

DEB_FIXPERMS_EXCLUDE := regex

where the regex excludes each file which matches. Good to know.

Software RAID on Gentoo and Debian

Up until now I backed up my hard drive with important files with unison. I did that by regularly plugging in a similar second hard drive into one of my empty drive bays, running unison, waiting for it to finish and finally pulling the new hard drive out again to store it on my shelf. This obviously spares at least one drive from running all the time but is a very tedious task, especially if it's IDE drives which aren't hot-swappable.

So I've recently gone through the very unpleasant course of loosing the data on one of my old IDE hard drives, of course the one which I don't back up with unison but located in the same file server. Rescuing data wasn't possible, even forensic tools like foremost and scalpel couldn't retrieve all the files, e.g. mp3 files were all either 7.2MB or 42MB in size and contained everything but a valid music stream. foremost didn't even finish and segfaulted after carving about 30% of the disk.

To make life easier and to possibly prevent such tragedies from happening in the future, I ordered two identical Seagate SATA hard drives, each 500GB. While Seagate doesn't precisely build silent drives but grants five years of warranty, these seemed to be perfect for the file server which is running in the cellar. But while planning on using RAID1 for the Seagate drives, I decided to do the same with my workstation - but this time not for reasons of redudancy but for the sake of access speed - and bought two Western Digital HDDs of the same size which promised to be less loud and also had 16MB of cache (instead of 8MB on the Seagate disks).

Once they arrived I added a spare PCI SATA RAID controller with a SiliconImage chipset into the file server and defined a RAID1 array in the card's BIOS setup tool. The stock Debian kernel contained all needed modules and after installing the dmraid with apt, I could view the setup

# dmraid -r
/dev/sda: sil, "sil_ahbgafcdfhah", mirror, ok, 976771120 sectors, data@ 0
/dev/sdb: sil, "sil_ahbgafcdfhah", mirror, ok, 976771120 sectors, data@ 0

# dmraid -s sil_ahbgafcdfhah
*** Active Set
name : sil_ahbgafcdfhah
size : 976771120
stride : 0
type : mirror
status : ok
subsets: 0
devs : 2
spares : 0

Mounting worked the usual way but instead of using the raw device node, you'd need to mount the respective mapped device, in my case /dev/mapper/sil_ahbgafcdfhah .

My workstation also has a (fake/software) RAID controller (onboard ICH7) but it wasn't as easy to set up because the RAID1 array acts as the boot drive and the kernel needs to be booted accordingly. Also, grub doesn't recognize RAID setups and thus I needed to create a separate small primary partition at the beginning of the RAID drive which is accessible, no matter if access it as a RAID drive or as its underlying disks. On this partition I store the kernel and its ramdisk, as well as grub's stage files. I did that by booting the Gentoo install CD, setting up dmraid as I did on the Debian box, formatting the RAID drive and copying the old drive's partitions with

# find . | cpio -pdum /mnt/target

The second problem was building a working kernel and its ramdisk. Custom built kernels didn't work because they're unable to initialize the mapping with dmraid. Luckily Gentoo's genkernel package saved my day.

# genkernel --dmraid all --menuconfig

After compiling the kernel in my chrooted target system, I copied the resulting files to my boot partition mentioned above and add the following entry to grub's config file:

title=Gentoo Linux 2.6.23 genkernel
root (hd0,3)
kernel /kernel-genkernel-x86_64-2.6.23-gentoo root=/dev/ram0 init=/linuxrc \
real_root=/dev/mapper/isw_bcaheaacjd_NewSystem2 dodmraid vga=792 ramdisk=8192
initrd (hd0,3)/initramfs-genkernel-x86_64-2.6.23-gentoo

Note that the fact that the root partition is the partition with number four has nothing to do with it being situated at the very beginning of the disk.