I recieved two emails in response to my last post, both from people owning slightly obscure hardware wondering how they might be able to reverse engineer it and possibly get Linux driver production underway.

The first step is to spy on the device during normal usage. In the case of the Alauda, this means monitoring all USB port traffic while using the device under Windows using the vendor-supplied driver.

Fortunately this is relatively easy to do with USB, you can use software such as USBSnoop on windows which will log all the traffic to a file. Other hardware is not so easy, sometimes you have to resort to expensive “middle-man” hardware devices which monitor and record the traffic for you.

After logging a few sessions, you then have the daunting task of analysing the logs. They are cryptic and hard to understand at first. Here is a small part of a USBSnoop log right after plugging in the MAUSB-10:

[93 ms]  >>>  URB 6 going down  >>> 
-- URB_FUNCTION_VENDOR_DEVICE:
  TransferFlags          = 00000001 (USBD_TRANSFER_DIRECTION_IN, ~USBD_SHORT_TRANSFER_OK)
  TransferBufferLength = 00000002
  TransferBuffer       = f6a77d3c
  TransferBufferMDL    = 00000000
  UrbLink                 = 00000000
  RequestTypeReservedBits = 00000000
  Request                 = 00000008
  Value                   = 00000000
  Index                   = 00000001
IRQL=2
[94 ms]  < <<  URB 6 coming back  <<< 
-- URB_FUNCTION_CONTROL_TRANSFER:
  PipeHandle           = 81b72c20
  TransferFlags        = 0000000b (USBD_TRANSFER_DIRECTION_IN, USBD_SHORT_TRANSFER_OK)
  TransferBufferLength = 00000002
  TransferBuffer       = f6a77d3c
  TransferBufferMDL    = 81c029e0
    00000000: 80 01
  UrbLink              = 00000000
  SetupPacket          =
    00000000: c0 08 00 00 01 00 02 00

This starts to make more sense after reading the USB specs – the host computer is sending a command to the device and gets 2 bytes of data back (0x80 0x01) in return. I think that this particular command is querying the device status, and 80 01 means not ready yet. After 10 or so of these commands have been sent, the final one returns a new value of “0x14 0x01”, and after that, all kinds of crazy stuff starts happening, so 14 01 somehow indicates that the device is ready for action.

Once you have an idea about which command does what, and how to interpret the responses, you can then start writing code to mimick what the windows driver does. I’ve archived a good HOWTO (slightly out of date) which provides an introduction to this: Reverse engineering Windows USB device drivers for the purpose of creating compatible device drivers for Linux.

Obviously the process will be different for non-USB devices, but I hope this shows that while it’s an interesting experience, reverse engineering is pretty damn hard, especially you have so little to go on. It’s much easier if you can get a copy of the manufacturers technical documents, which is sometimes possible.

After some downtime after we switched from bitkeeper to gentoo svn, the genpatches website is now back up and automatically updating.

genpatches is the patchset that gentoo-sources is based on, as well as some other kernels in Portage.

http://dev.gentoo.org/~dsd/genpatches/

I got a few interesting responses to my last post.

Apparently I’m just a freak of nature who can sit down and write some fairly comprehensive documentation as a single task, whereas with coding, I often find it necessary (or at least beneficial) to take a break, go away and properly think about what I’m doing or trying to do.

People quoted reasons such as finding it hard to write documentation which is easy to follow, it just not being fun, reminding them of schoolwork, etc… Also, those with non-english nationality pointed out that they find it very hard to write documentation in (good) English and wouldn’t see as much recognition if it was only written in other languages.

I guess that answers my question. Still, my point remains, documentation is a very rewarding contribution in terms of direct feedback.

I’ve been thinking lately about documentation for open-source projects.

Most of my open-source contributions are code-based, but occasionally I write or contribute to documentation. Yet the email feedback I recieve is pretty much 100% about documentation that I have written as opposed to code I have developed. I get people just saying ‘thanks’, people giving suggestions, people asking permission to republish it elsewhere, people providing translations, its fantastic and much appreciated.

I had a recent conversation with roger55 on IRC which went something like:

<roger55> hey dsd, i just got back from linuxtage graz, it seems you have a few fans out here
<dsd> roger55: cool! what were they saying?
<roger55> dsd: they love the guides you write

ahem.. I write lots of code as well!! ;)

The documentation/documentation-user seems to be a much more thriving and active community than the developer/software-user community, and from my perspective, documentation authors get a lot more feedback and direct communication with the users. Writing documentation for an open-source project is an exciting and valuable contribution you can make. Tag your email address on the end and you’ll really know when people are benefitting from your work.

The question I am posing: Why are there so few open source documentation writers in comparison to software developers? Using Gentoo as an example, we have just 15-20 core documentation writers, compared to hundreds of developers who deal with software code and maintenance. Not using Gentoo as an example, many open source projects lack comprehensive and up-to-date documentation, and a lot of existing documentation is written by software developers who write it in response to being asked the same questions repeatedly.

Writing code is a specialist thing. In the UK, you don’t really get taught how to do it until your 14th year of full time education. It’s very technical and you need to be skilled and have a good knowledge of computers to be an effective developer.

Yet language and literacy is something you are taught throughout education and life, and using software is a daily thing for many people. No offense to the skilled documentation authors out there, but I don’t see it as such a specialist activity. So where is the mass of documentors, and do they know what they are missing out on?

Myself plus a small number of other Gentoo users/developers are planning a small meetup in central London on Friday (17th). If you’re interested in turning up, send me an email, and I’ll pass on the details.