Archive for April, 2005

dpbuild

Friday, April 29th, 2005

Continuing on with the Gentoo development platform venture, I’ve updated the Portage-HEAD installation guide to take account for the fact that sys-apps/sandbox is now in the Portage tree, and also added a Portage patch which allows non-root users to run ebuild for normal operations.

I attempted to tackle an initial scenario: Patching a package (gamin) with a big patch which I got out of their CVS. Unfortunately too much had changed elsewhere in CVS for this to be successful against the latest gamin release, but I reached a milestone in that I hacked ebuild enough to work as a normal user, extracting source into the users home directory, which is slightly more convenient than having to be root and getting it extracted into /var/tmp with restrictive permissions.

I also added a utility to portage, dev-util/rej, which aims to help you apply diff/patch rejects. In this situation, it managed to automatically resolve some of the rejects, which was nice. A handy utility to keep in mind.

dpbuild is a wrapper script for ebuild, which currently just sets environment variables allowing ebuild to be used as a normal user (it will be extended to incorporate other hacks in the future). I’m only thinking in terms of unpacking/compiling here, installation to / obviously won’t work. See the Gentoo development platform page for more info and where to find it.

I recieved some feedback after my last weblog post on this topic. It’s good to know that there are people besides myself who would find some of the ideas I have mentioned to be beneficial to their working environment.

Installing Portage HEAD

Thursday, April 21st, 2005

I recently started a discussion on gentoo-dev about some ideas I’ve been considering about making Gentoo more suited for a software development environment.

In the mail, I raised just one issue that I’ve been thinking about – using upstream CVS-sources for a package as opposed to the sources included in the latest release tarball, while still installing the software through the portage build system. It seems that some of the people who replied either missed the point I was trying to make, or thought I’m making a fuss out of nothing – they pointed out that I can just write a CVS ebuild and be done with it. Yeah, that’s not too much trouble, but I’m talking about streamlining processes here guys. If I was to write a CVS ebuild, it would be almost identical in content to the “release” ebuild from the portage tree, so why should I have to write it in the first place? I’ve got the sources, someone elses has written an ebuild, let me combine the two with minimal effort :)

I did get some useful information from the discussion though. The development version of Portage makes some of the things I’m thinking about much more realistic. I’ll talk more about specifics later, for now I just want to document the procedure of installing the development version. Thanks to Stefan (genstef) for sharing this knowledge with me!

Installing Portage-CVS
Since the release of Portage 2.0.51, the portage developers have been doing some pretty rapid development, which appears to be mainly focused on new features, rewriting bottleneck code, and improving general performance. There’s some other good news in here: there’s a few features which really help make progress to improving gentoo as a development platform.

Portage-CVS (aka HEAD) is progressing quickly but has no guarantees for stability. If this software burns your house down, you are to blame for installing it in the first place. Don’t expect it to be flawless at this point in time.

That aside, here’s how to install it on your running system:

  1. Checkout the sources
    cvs -d YOURUSERNAME@cvs.gentoo.org:/var/cvsroot co gentoo-src/portage
    cd portage
    ./autogen.sh
    ./configure
    
  2. Make and copy portage-cvs tarball
    make dist
    cp portage-cvs.tar.bz2 /usr/portage/distfiles
    
  3. Install sys-apps/sandbox
    Get the ebuild here, put it in your overlay, in the sys-apps category. You probably want to add sandbox to your USE flags too.
  4. Install portage-cvs
    Get the ebuild here, put it in your overlay, in the sys-apps category.

    emerge -av --digest portage

While I’m going to be posting most of the progress I make on this topic here on my weblog, I’m going to archive and maintain everything here as well.

Extended Attribute Mono bindings

Wednesday, April 20th, 2005

After seeing it requested, I recently ported some of the Beagle code for manipulating extended attributes on files into the core Mono library base. It’s been added in the Mono.Unix.Syscall class, which is set to replace Mono.Posix.Syscall once Mono 1.2 is released and widespread.

This is my first contribution to Mono, with thanks to Jon Pryor (Mono.Unix maintainer) who pointed out that my first attempt at this was incorrect, and kindly went out of his way to describe the possible solutions in depth. Now to write some usage documentation…

Problems with GConf#

Tuesday, April 19th, 2005

Beginning to wonder if GConf for the configuration system was such a good choice after all. I’m getting slightly random Segfaults (yes, segmentation faults from a mono application, I thought it was odd too..).

Everything has been going according to plan until I start adding live update notification, so that beagled can immediately realise when a config option has been changed. Then (most of the time), when I change the configuration, I get segfaults before control is passed back to beagled. I’ve been advised that this might be a threading issue – I can’t get my head around why this might be the case, but then again, I can’t reproduce it in a simple test app.. Hmm…

On another note, I managed to sort out a problem with the timing of one of my exams which means I can now make GUADEC – I’ve got my flight booked :)

The definitive guide to CD writing on recent 2.6 kernels

Sunday, April 17th, 2005

Despite the fact it happened a few months ago, I still see people claiming that it is impossible to write CD’s with recent 2.6 kernels due to the internal changes. This is not true!

The first thing to check is that you aren’t using SCSI emulation. You don’t need this in Linux 2.6 – its ugly, and apparently doesn’t work. Disable SCSI emulation and use the new ATAPI features to burn “directly” (no extra configuration options needed other than general ATAPI cdrom support).

Next up, check you are running the latest version of cdrtools/cdrecord, even if this means going into the testing/unstable tree. The latest at the moment is cdrtools-2.01.01_alpha01-r1. The same applies for any other cd recording software you might be using.

It’s also important that you are running the latest stable kernel. This is currently 2.6.11. If you aren’t running this, then you are in no position to complain that things don’t work or file bugs.

A few months ago, the kernel code changed to restrict the commands which can be sent to CD writers. The restriction only allows commands that are related to CD reading and writing. The commands are listed in a table. At first, the table was too small, but thanks to user feedback I helped expand it to cover all commands needed for CD writing.

It seems that many users keep advocating a patch which completely bypasses the command table – and this patch is still included in some patchsets! This is no longer needed! If it does make a difference, then its a bug in either your CD writing software or the kernel command table is missing a valid command entry. If this is the case, send me an email, and I’ll fix things up with minimum effort involved. I’ve done this a few times before and I’m happy to do it again.

Another kernel change made at the same point in time resulted in the ability to write CD’s being restricted to users with write access to the CDROM device. That makes sense, right? (Previously you just needed read access.)

So the first thing to do is check that you have write access to your CDROM node. Assuming your CDROM drive is /dev/hdc, run the following:

# [ -w /dev/hdc ] && echo "OK" || echo "Fix up your permissions"

If you see a message telling you to fix up your permissions, the usual procedure is to use ls to see which group owns the node (usually cdrw or cdrom), add yourself to that group, and then try again. This requires that the group has access to write to the node too.

Moving on. Some software used to advise that you set the suid bit on your cdrecord binary. This means that cdrecord will always run with root priveleges, even if you execute it from a standard user account. Some kernel memory management changes broke cdrecord’s ability to run as root for a while. Even though it’s fixed now, it’s worth turning this off for the security implications alone:

# chmod -s /usr/bin/cdrecord

Note that xcdroast does weird things with suid bits too. For the purpose of testing, I suggest you keep away from xcdroast.

In terms of software usage, people seem to be historically used to running commands like cdrecord dev=ATAPI:0,0,0. An alternative method is cdrecord dev=/dev/hdc. There’s been an eternal debate as to which way should be encouraged – the cdrecord author doesn’t like the dev=/dev/hdc method, as you can probably gather from the warnings that appear. Right now, all you want to do is write CD’s, so no matter what you think, you need to use the dev=/dev/hdc method. Also, do avoid the dev=ATA:x,y,z notation too.

Another possible problem is that your CD writing software is opening up the CD node as read-only. The kernel will reject any write commands sent to it. The software needs to be modified to open the CD node as read-write. I fixed up some of the software such as dvd+rw-tools and cdrwtool a while back – but if there is any more problematic software, please let me know. (This is why its essential that you are running the latest versions of the software in question)

Finally, burn as your standard user account, not as root.

Hopefully this clears some things up. If you do experience problems after following the above precisely, please file a bug.

Beagle configuration system “preview”

Thursday, April 14th, 2005

Finished the basic implementation of the configuration system. As previously mentioned, this is based on GConf.

It’s far from complete: you have to choose your settings before beagled is running, or restart beagled after changing the configuration for the new settings to take effect. You can’t do much in the way of configuration right now, but you can add more filesystem roots to be indexed.

It includes a new command line tool, beagle-config. For a demo, read the full post.

I’ve posted it to bug 172283 (direct patch link here). There’s some implementation details included at the top of the patch.

Read on for an example of beagle-config… It’s quite fancy as the configuration app itself is quite simple (and will be simplified further) – it pulls everything (the list of configuration “sections”, the operations you can perform, their descriptions, the invokation output) from Beagle.Util.Conf.

(more…)

Planet Beagle

Wednesday, April 13th, 2005

I’ve noticed that I’m now aggregated onto Planet Beagle. I have no idea who maintains this, but thanks! As a result of this, I’ve started writing more about my beagle work, see yesterdays and todays posts on the configuration system I’m working on. Comments welcome. I’ll hopefully post a basic and working implementation before the end of the week.

I finally recieved my exam timetable today. It’s mostly good news because it means that attending GUADEC is possible, even if I do have to take risks with a Java exam before I fly out, and fly back a day early in order to catch my maths exam. Also looks like I’ll get to meet Stefan while out there. If any other Gentoo’ers are thinking of attending, please let me know!

Why even use a schema at all?

Wednesday, April 13th, 2005

Most of yesterdays post was based around generating and installing a GConf schema.

After further thought, why even use a schema at all? We don’t really want to be relying on external tools such as gconftool-2 to install the default keys on a per-user basis, and gconfsharp-schemagen is too basic for our needs.

So the new plan is just to write a Beagle.Util.Conf class which makes reading and manipulating settings easy to the rest of the code, leaving a possible window of opportunity for some automation to create this at compile-time in the future.

As for installing default setting keys, this seems easy enough even without a schema. Monodoc contains this code snippet:

try
{
MyVal = (string) gconfClient.Get ("/apps/monoapps/SampleApp/setting1"));
}
catch (GConf.NoSuchKeyException)
{
gconfClient.Set ("/apps/monoapps/SampleApp/setting1", "sample");
}

…which pretty much tells me all I need to know.

Early work on GConf-based Beagle config system

Tuesday, April 12th, 2005

I’ve been continuing work on making a configuration system for Beagle. We’ve decided to go for a GConf-based system if things can work out. This is a learning experience for me as I’ve never dealt with GConf before.

A few notes on the design, which was mostly spurred from the XML-based system I drafted previously.

  • A new class, Beagle.Util.Conf, handles the configuration. Currently only reading, but eventually will also handle writing and live notification of updates.
  • We have a schema file, which contains the schemas for the keys we use. Our keys are kept under /apps/beagle
  • We use gconfsharp-schemagen to generate a couple of new classes, Beagle.Util.Settings and Beagle.Util.SettingKeys.
  • We use gconftool-2 to install the schema and default keys

Before I go much further, there’s a few issues that need to be addressed.

  1. gconftool-2 –makefile-install-rule seems to overwrite any previous settings. This would mean users losing their settings on every upgrade/recompile?
  2. I can’t find much documentation, but the purpose of gconfsharp-schemagen seems to aim at creating convenient access classes to the GConf settings. This is pretty much the same as the aim behind Beagle.Util.Conf – do we really need to duplicate?

To elaborate on point 2, the classes autogenerated by gconfsharp-schemagen are quite basic. Ideally we need to be aware of things like threading, e.g. using the code already written for Beagle.Daemon.GConfThreadHelper. It would also need to be extended for things such as the command line configuration tool, beagle-config.

Where to go from here? We could either ditch the gconfsharp-schemagen thing and rely on always hardcoding the schema paths and accessors into the hand-crafted Beagle.Util.Conf class, or we could write our own autogeneration tool to produce something like Beagle.Util.Conf from the schema file, with the detail and functionality that we need…

This almost of boils down to one important implementation detail: If we were to hand-craft Beagle.Util.Conf (maybe using gconfsharp-schemagen as a base) then it would be quite possible in the future to change our storage method to something else, simply by modifying the Beagle.Util.Conf methods. On the other hand, if we go for a purely automated way using automatic class generation from the schemas and other metadata, then it most likely means hardcoding GConf specifics into the areas of beagle which are to use these configuration values, making future changes to another system more difficult…

Closing inactive bugs

Friday, April 1st, 2005

I just did my regular run over all the kernel@gentoo.org bugs and ended up closing about 10 of them because we’d asked for some more info, or for the reporter to test a newer kernel, but the user had not responded given a few weeks to do so.

Do other package maintainers see this kind of thing happening much? I don’t like to close bugs knowing that its quite likely that they still exist, but if they are left open they wouldn’t go anywhere either. I’d guess that testing a newer kernel is considerably more effort than trying a different version of most other packages, but I’d be interested to know if this pattern is repeated over the other areas of Gentoo, and how those maintainers deal with it.