Get to the heart of the matter

There’s a time in every programmer’s life when he wonders if he can write his own kernel. (I’m keen on understanding how the page management facilities of an operating system can be used to implement a write barrier for garbage collection – the kind of thing that Azul does in its Zing architecture)  After spending some time investigating, the answer comes along – there’s a lot of documentation on getting it working on Linux, but less on how to get things going on Windows. I spent the last week or so going around various tutorials and taking bits from each in order to get to a stage where I can write some C code, and then boot it under Bochs (an x86 emulator) and VMWare. I thought it would be good to note down how to do this in case anyone out there is struggling to get this going.

The first thing to get a handle on is the boot loader. The wiki at OsDev.org contains some good material on this, in particular this article on the boot sequence which links to here and here. Of course to really see what’s going on, it’s good to have some working code, and after some searching I came across MikeOS, a 16 bit operating system for x86. This is written entirely in x86 assembly, so you get to see the boot code itself (in sources\bootload\bootload.asm). This code uses calls into the BIOS to search the disk for the kernel executable, which it then loads and executes. The kernel then offers a basic command line interface, allowing you to run a number of modules that come with the system. These modules include a BASIC interpreter and other utilities. All the code is there, and is well commented so it is really easy to follow.

Building MikeOS is a little tricky – the lack of a loopback device on Windows requires the use of ImDisk (a virtual disk driver), but by following the instructions you can get something going which boots under VmWare fairly quickly.

I was more interested in running a 32 bit kernel and so started working through the tutorials by James Molloy. These are brilliant. They use GRUB2 (the Grand Unified Boot Loader) to get the kernel loaded, and use a combination of a very small amount of assembler and C to get a kernel going on Windows. This tutorial kernel offers memory protection and multi-tasking though, unfortunately, the tutorial stops before getting to how we interact with external devices (such as disc drives) so I’m still investigating such material. It is a complete joy to be writing some C again – the feeling of being close to the machine level while still using a high level language is really liberating.

The tutorial is aimed at Linux and very probably works there, but I had a load of trouble getting it to run under Windows. The tools expect a compiler tool chain that generates ELF format object files, but the default Cygwin compilers on Windows generate PE format. I had to go out and find a set of precompiled cross compiler tools to build things.

I then had trouble actually writing an image in the correct format – the Linux examples all use a file mounted via a loopback device, and though there are such device drivers on windows I couldn’t find one that successfully runs on 64bit machines. In the end I had to find another tutorial which uses GRUB, and modify that into the build scripts to get things to work.

I put a zip here, KernelBuilding.zip, that contains a working examples from midway through the JamesM tutorial, at a point where we have just added the PIT to the kernel. The zip contains the necessary cross compilers, CD writing tools and the parts of GRUB that are required. You’ll need to download Cygwin, which simulates a Unix environment on Windows, NASM, an assembler, and Bochs, an x86 emulator which allows much quicker development.

disto

We can then start a command shell, set up the path to Cygwin, and build the kernel.img.

compile

In order to run the GRUB boot loader, we are going to need to tell the loader the size of the kernel in 512 byte blocks (rounded up).

calculatesize

Now we can run the emulation under Bochs using the command which I added to the Makefile:

  make run

which starts the boot inside the Bochs x86 emulator.

Bochs

The 237 value on the kernel line is the size of the kernel in blocks.

RunningUnderBochs

Running under an emulator is all very well, but I’d prefer to run under a real machine (or rather VMWare). To this we need to make an iso file, and for this reason I included the cdrtools in the zip file.

First we have to pad our kernel.img to the size of a standard floppy. I did this using the fsutil command in Windows (which you need to be admin to run). Using this we can generate a pad file and generate a floppy.img file of the correct size.

  set /a (1474560-121672)
  fsutil file createnew pad3 1352888
  copy /b kernel.img+pad3  floppy.img
  cdrtools\mkisofs.exe -o floppy.iso -b floppy.img floppy.img

This produces an iso we can run under VMWARE.

vmwareIso

Running it gets us to the GRUB loader where we can then boot into our code.

Vmware

Having got the setup out of the way, I now hope to spend more time working through the tutorials in order to get a better kernel running. It’s pretty impressive that you can freely download all of the material that you need to do this, though the amount of different utilities means that it is some work to assemble a working set. Like all things, understanding how the kernel is working at the higher level helps to explain many of the higher level abstractions that you see, and is therefore a worthwhile learning exercise.

Advertisements
This entry was posted in Computers and Internet. Bookmark the permalink.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s