Ext2 Filesystems on Common Lisp Vectors

Sat, 25 Jun 2011 23:43:16 GMT

External Content not shown. Further Information. Tröööt! There is a new protocol joining the small group of network protocols which I like. The Network Block Device protocol. It seems like it is exactly as complicated as it needs to be. It just has a major disadvantage: It is not well documented. Surprisingly, the German Wikipedia has a quite nice article about NBD, and this article is not that bad! It even has a protocol documentation - as one wishes when consulting an encyclopedia. (So it will probably be deleted soon.)

Actually, I had a closer look at NBD while I was actually looking for something else, but the implementation of an NBD server took some time. I wanted to create an NBD server which allocates a specified amount of memory and populates it as block device - just as a proof of concept telling me that I have "understood" the protocol and that I am able to implement it.

I began with C and finally succeeded, after finding out that every adress has to be sent in network byte order, and finding some useful header files like <linux/nbd.h>. Still, my C implementation has some bug, and as I did not have any special aim, I re-implemented everything in Common Lisp. The code can be found at GitHub (exercise: find out where its name comes from).

Its only dependency so far is usocket, and as I did not yet create an asd file it has to be loaded manually, which I do via Quicklisp.

CL-USER> (ql:quickload "usocket")

Then I load my file and start a server with a 10 megabyte block device.

CL-USER> (load #P"~/projects/7-chloro-4-nitrobenzofurazan/nbdserver.lisp")
CL-USER> (defvar *pool* (run-rampool 13337 (* 10 1024 1024)))

This will block until an nbd client connected and returned again. So let us connect via the linux nbd client. We create an ext2 filesystem and then mount it and create a file GANS with the content NILPFERD in the shell

# nbd-client localhost 13337 /dev/nbd0
Negotiation: ..size = 10240KB
bs=1024, sz=10240
# mkfs.ext2 /dev/nbd0
[...]
# mount /dev/nbd0 /mnt
# echo "NILPFERD" > /mnt/GANS
# sync
# cat /mnt/GANS
NILPFERD

Then we unmount it again and detatch the device.

# umount /mnt
# nbd-client -d /dev/nbd0
Disconnecting: que, disconnect, sock, done

Now, the REPL is free again, and *pool* contains the data. We have just created an ext2 filesystem on a lisp vector. Now we should find the word "NILPFERD" in *pool*, and indeed

CL-USER> (search (map 'list #'char-code "NILPFERD") *pool*)
2098176

Now let us see what happens if we replace this string (as this returns the whole *pool*, we set *print-length* first):

CL-USER> (setf *print-length* 10)
10
CL-USER> (replace *pool* (map 'list #'char-code "nilpferd") :start1 2098176)
#(0 0 0 0 0 0 0 0 0 0 ...)

Now, let us re-run the nbd-server with the modified *pool*

CL-USER> (run-rampool 13337 (* 10 1024 1024) *pool*)

And attatch and mount it again

# nbd-client localhost 13337 /dev/nbd0
Negotiation: ..size = 10240KB
bs=1024, sz=10240
# mount /dev/nbd0 /mnt

Now, about the file:

# cat /mnt/GANS
nilpferd

Yep, it is indeed lowercase now. Which is ... nice ... somehow. Though useless, of course.

Besides that, I found an implementation of a server in python, though I have not tested it. Same for this Java implementation.

Unfortunately, there seems to be a lack of clients. A port to some other Unices and Windows would be nice. Especially, looking at these implementations of block device drivers, there should be people who can easily implement an NBD Client for Windows.