Neues in der Kategorie Linux

Especially old computers often fail to boot from USB Sticks, but, however, can do a PXE netboot - and as a person who often wants to install stuff on old machines, or does not have a USB stick of sufficient size when needed (let alone a cd), it is a good thing to have all the necessary software installed.

Dnsmasq has everything needed: A TFTP server, a DHCP server and a DNS server. Its central configuration file (under Debian) is /etc/dnsmasq.conf.

To enable the TFTP server with a given root directory to host, the options enable-tftp and tftp-root can be used. To then actually set the pxe boot path, there is the dhcp-boot option. To activate the dhcp server, there is the dhcp-range option, that gives the IP address range that can be passed to the dhcp clients. A further important option to set is the router option, which sets the gateway IP adress. If it is not set, it defaults to your computer, and if you did not configure your computer to actually route, you will not be able to access the internet. Just set it to the appropriate gateway adress of the actual router.

A dangerous but necessary option is the autoritative option, which makes the dhcp server autoritative. Do not use this option if you are not in your own network, or know exactly what you do: It overrides your router's dhcp server, and might keep other users from connecting.

I use this setup only temporarily, to quickly install a few clients and then turn it off again. Therefore, I turn the daemon off with the appropriate option in /etc/default/dnsmasq, per default.

This setup is comparably easy (even though still complicated), and for a complicated setup like ltsp, it is not sufficient, but for the quick setup of a few computers in the network, it is nice.

... after Linux.

At least the ext[2-4] filesystems support files with holes. They can be created in several ways, one possibility is the seek-argument of the dd command:

$ dd if=/dev/urandom of=test bs=4096 count=10 seek=10
10+0 records in
10+0 records out
40960 bytes (41 kB) copied, 0.0126733 s, 3.2 MB/s
$ wc -c test
81920 test

So I was interested in whether it is possible to actally find these holes. ZFS and XFS have their own API for that.

For ext*, there is also a possibility to find holes. Basing on the FIBMAP Ioctl (and similar), you need to have the CAP_SYS_RAWIO capability (that is, usually you have to be root). If you only want to watch the holes in the files, you can use for example hdparm (as shown here):

$ sudo hdparm --fibmap test

test:
 filesystem blocksize 4096, begins at LBA 0; assuming 512 byte sectors.
 byte_offset  begin_LBA    end_LBA    sectors
       40960    8928120    8928199         80

Another possibility is to use the filefrag-utility, which is contained in the debian squeeze package e2fsprogs (as shown here):

$ sudo filefrag -v test
Filesystem type is: ef53
File size of test is 81920 (20 blocks, blocksize 4096)
 ext logical physical expected length flags
   0      10  1116015              10 eof
test: 2 extents found

Now if you really want to use your own program, here is a nice example code I found, on which I based my own code:


#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <unistd.h>
#include <errno.h>
#include <fcntl.h>
#include <sys/ioctl.h>
#include <sys/types.h>
#include <sys/stat.h>
#include <linux/fs.h>


int main (int argc, char* argv[]) {
  int fd, blocknum, blocksize;
  struct stat fileinfo;

  if (argc < 1) {
    fprintf(stderr, "Syntax Errof\n");
    exit(EXIT_FAILURE);
  }

  if ((fd = open(argv[1], O_RDONLY)) < 0) {
    int errnum = errno;
    fprintf(stderr, "Cannot open '%s': %s\n", argv[1], strerror(errnum));
    exit(EXIT_FAILURE);
  }

  if (ioctl(fd, FIGETBSZ, &blocksize) < 0 ) {
    int errnum = errno;
    fprintf(stderr, "Cannot get blocksize: %s\n", strerror(errnum));
    exit(EXIT_FAILURE);
  }

  if (fstat(fd, &fileinfo) < 0) {
    int errnum = errno;
    fprintf(stderr, "Stat failed: %s\n", strerror(errnum));
    exit(EXIT_FAILURE);
  }

  blocknum = (fileinfo.st_size + blocksize - 1) / blocksize;

  printf("Filename: %s\nBlocksize: %d\nBlocknum: %d\n",
         argv[1], blocksize, blocknum);
 
  int i;
  for (i = 0; i < blocknum; i++) {
    int block = i;
    if (ioctl(fd, FIBMAP, &block)) {
      printf("ioctl failed: %s\n", strerror(errno));
    }
    printf("%10d\t", block);
  }
  close(fd);
  printf("\n");
  exit(EXIT_SUCCESS);
}

The output:

$ sudo ./fibmap test
Filename: test
Blocksize: 4096
Blocknum: 20
         0
         0
         0
         0
         0
         0
         0
         0
         0
         0
   1116015
   1116016
   1116017
   1116018
   1116019
   1116020
   1116021
   1116022
   1116023
   1116024

And that is indeed a list of ten null-pointers and ten consecutive blocks.

Tröööt! There is a new protocol joining the small group of network protocols which I like. The Network Block Device protocol. It seems like it is exactly as complicated as it needs to be. It just has a major disadvantage: It is not well documented. Surprisingly, the German Wikipedia has a quite nice article about NBD, and this article is not that bad! It even has a protocol documentation - as one wishes when consulting an encyclopedia. (So it will probably be deleted soon.)

Actually, I had a closer look at NBD while I was actually looking for something else, but the implementation of an NBD server took some time. I wanted to create an NBD server which allocates a specified amount of memory and populates it as block device - just as a proof of concept telling me that I have "understood" the protocol and that I am able to implement it.

I began with C and finally succeeded, after finding out that every adress has to be sent in network byte order, and finding some useful header files like <linux/nbd.h>. Still, my C implementation has some bug, and as I did not have any special aim, I re-implemented everything in Common Lisp. The code can be found at GitHub (exercise: find out where its name comes from).

Its only dependency so far is usocket, and as I did not yet create an asd file it has to be loaded manually, which I do via Quicklisp.

CL-USER> (ql:quickload "usocket")

Then I load my file and start a server with a 10 megabyte block device.

CL-USER> (load #P"~/projects/7-chloro-4-nitrobenzofurazan/nbdserver.lisp")
CL-USER> (defvar *pool* (run-rampool 13337 (* 10 1024 1024)))

This will block until an nbd client connected and returned again. So let us connect via the linux nbd client. We create an ext2 filesystem and then mount it and create a file GANS with the content NILPFERD in the shell

# nbd-client localhost 13337 /dev/nbd0
Negotiation: ..size = 10240KB
bs=1024, sz=10240
# mkfs.ext2 /dev/nbd0
[...]
# mount /dev/nbd0 /mnt
# echo "NILPFERD" > /mnt/GANS
# sync
# cat /mnt/GANS
NILPFERD

Then we unmount it again and detatch the device.

# umount /mnt
# nbd-client -d /dev/nbd0
Disconnecting: que, disconnect, sock, done

Now, the REPL is free again, and *pool* contains the data. We have just created an ext2 filesystem on a lisp vector. Now we should find the word "NILPFERD" in *pool*, and indeed

CL-USER> (search (map 'list #'char-code "NILPFERD") *pool*)
2098176

Now let us see what happens if we replace this string (as this returns the whole *pool*, we set *print-length* first):

CL-USER> (setf *print-length* 10)
10
CL-USER> (replace *pool* (map 'list #'char-code "nilpferd") :start1 2098176)
#(0 0 0 0 0 0 0 0 0 0 ...)

Now, let us re-run the nbd-server with the modified *pool*

CL-USER> (run-rampool 13337 (* 10 1024 1024) *pool*)

And attatch and mount it again

# nbd-client localhost 13337 /dev/nbd0
Negotiation: ..size = 10240KB
bs=1024, sz=10240
# mount /dev/nbd0 /mnt

Now, about the file:

# cat /mnt/GANS
nilpferd

Yep, it is indeed lowercase now. Which is ... nice ... somehow. Though useless, of course.

Besides that, I found an implementation of a server in python, though I have not tested it. Same for this Java implementation.

Unfortunately, there seems to be a lack of clients. A port to some other Unices and Windows would be nice. Especially, looking at these implementations of block device drivers, there should be people who can easily implement an NBD Client for Windows.

I hate firewalls, but I have no choice, with gigabytes of spam-traffic. By a mistake of mine, I probably locked out a lot of IP adresses that should not have been locked out. I am sorry for that.

If you notice that I locked somebody out, please let me know.

There is apparently no simple possibility to find out whether a given IP adress is blocked. So I cannot easily filter my logfiles. Above that, the default whois-answer gives an IP range, but iptables wants CIDR-notation.

I could not find any software calculating this (if somebody knows a good one, then please tell me). What I quickly wrote in a file range2cidr.c is:

#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <string.h>
#include <stdint.h>
#include <netdb.h>
#include <math.h>

int main (int argc, char ** argv) {
  if (argc != 3) {
    printf("Usage: %s lowerbound upperbound\n", argv[0]);
    exit(EXIT_FAILURE);
  } else {
    uint32_t lowip, highip;
    struct hostent *host;
    host = gethostbyname(argv[1]);
    lowip =
      ((((uint8_t) host->h_addr[0]) % 256) << 24) +
      ((((uint8_t) host->h_addr[1]) % 256) << 16) +
      ((((uint8_t) host->h_addr[2]) % 256) << 8) +
      ((((uint8_t) host->h_addr[3]) % 256));

    host = gethostbyname(argv[2]);
    highip =
      ((((uint8_t) host->h_addr[0]) % 256) << 24) +
      ((((uint8_t) host->h_addr[1]) % 256) << 16) +
      ((((uint8_t) host->h_addr[2]) % 256) << 8) +
      ((((uint8_t) host->h_addr[3]) % 256));

    uint32_t msk = lowip ^ highip;

    int i=0;
    while (msk != 0) {
      msk /= 2;
      i++;
    }

    printf("%s/%d\n", argv[1], 32-i);
           
    exit(EXIT_SUCCESS);
 
  }}

You might wonder why I calculated the IPs so high-level. Well, I just did not want to care about the whole lowlevel-fuss and still have it portable - I mean, this code does not need to be fast, it just needs to be correct.

Anyway, there has got to be better software. Any suggestions?

Even though I like ZFS, I am using Linux which has not yet good ZFS-Support, and LUKS. Using LUKS with an encrypted swap I want to be able to use for suspending, still the easiest possibility to achieve this is using an LVM [yes, this sentence is gramatically correct ...].

However, so far I only used this on one partition, which is already very flexible. Today I tried to use LVM on three USB-Sticks that I found occasionally. As they have different sizes, there is no point in creating a Raid, so I just used them directly.

So, I have three USB-Sticks, of different size, /dev/sdb, /dev/sdc, /dev/sdd. Firstly, I create the physical volumes on them:

root@thinkpad:/# pvcreate /dev/sdb
  Physical volume "/dev/sdb" successfully created
root@thinkpad:/# pvcreate /dev/sdc
  Physical volume "/dev/sdc" successfully created
root@thinkpad:/# pvcreate /dev/sdd
  Physical volume "/dev/sdd" successfully created

Then I create the volume group:

root@thinkpad:/# vgcreate vgusb1 /dev/sdb /dev/sdc /dev/sdd
  Volume group "vgusb1" successfully created

(Actually, a lot of rw-errors are emitted, but they are pointless, and I left them out, and I will leave them out in the further commands.)

Then I create a logical volume, with 40% of the size of the whole volume group:

# lvcreate -l 40%FREE -n test vgusb1
  Logical volume "test" created

Let us make an ext4 filesystem on it:

root@thinkpad:/# mkfs.ext4 /dev/vgusb1/test
mke2fs 1.41.12 (17-May-2010)
Filesystem label=
OS type: Linux
Block size=4096 (log=2)
Fragment size=4096 (log=2)
Stride=0 blocks, Stripe width=0 blocks
64512 inodes, 258048 blocks
12902 blocks (5.00%) reserved for the super user
First data block=0
Maximum filesystem blocks=264241152
8 block groups
32768 blocks per group, 32768 fragments per group
8064 inodes per group
Superblock backups stored on blocks:
    32768, 98304, 163840, 229376

Writing inode tables: done                           
Creating journal (4096 blocks): done
Writing superblocks and filesystem accounting information: done

This filesystem will be automatically checked every 39 mounts or
180 days, whichever comes first.  Use tune2fs -c or -i to override.
root@thinkpad:/# mount /dev/vgusb1/test /mnt

So, now I have the filesystem. Lets create a file with random content:

# dd if=/dev/urandom of=/mnt/rand bs=4K

Let us check the md5sum quickly

root@thinkpad:/# md5sum /mnt/rand
251e865156f990e10711f5ac3e10d0bd  /mnt/rand

Verilyyyyyyyyyyy... So now we will create a snapshot and assure ourselves that it has the same content:

root@thinkpad:/# lvcreate --snapshot /dev/vgusb1/test -n snap -l 40%VG
  Logical volume "snap" created
root@thinkpad:/# mount /dev/vgusb1/snap /mnt2/
root@thinkpad:/# md5sum /mnt2/rand
251e865156f990e10711f5ac3e10d0bd  /mnt2/rand

Ok. Now we have a snapshot. Let us unmount the original and extend it to the maximum size, and resize the ext4 filesystem.

root@thinkpad:/# umount /mnt
root@thinkpad:/# umount /mnt2
root@thinkpad:/# lvchange -a n /dev/vgusb1/test       
root@thinkpad:/# lvextend -l +100%FREE /dev/vgusb1/test
  Extending logical volume test to 1.48 GiB
  Logical volume test successfully resized
root@thinkpad:/# e2fsck -f /dev/vgusb1/test
e2fsck 1.41.12 (17-May-2010)
Pass 1: Checking inodes, blocks, and sizes
Pass 2: Checking directory structure
Pass 3: Checking directory connectivity
Pass 4: Checking reference counts
Pass 5: Checking group summary information
/dev/vgusb1/test: 12/64512 files (0.0% non-contiguous), 258045/258048 blocks
root@thinkpad:/# resize2fs /dev/vgusb1/test
resize2fs 1.41.12 (17-May-2010)
Resizing the filesystem on /dev/vgusb1/test to 387072 (4k) blocks.
The filesystem on /dev/vgusb1/test is now 387072 blocks long.
root@thinkpad:/# mount /dev/vgusb1/test /mnt
root@thinkpad:/# mount /dev/vgusb1/snap /mnt2

And now, let us fill /mnt/rand with new random information

# dd if=/dev/urandom of=/mnt/rand bs=4k

And check the checksums again

root@thinkpad:/# md5sum /mnt2/rand
251e865156f990e10711f5ac3e10d0bd  /mnt2/rand
root@thinkpad:/# md5sum /mnt/rand
55ee3477277947183c60d7190536f5c5  /mnt/rand

Snapshotting apparently worked. Apparently, it still works, when I unplug the sticks and plug them in again in a different order, and they get different devicenames.

LVM is nice.

The software sudo is useful to give users the right to execute commands that otherwise need higher permissions. Ubuntu does not even have a root-account anymore that you can login into, you need to run sudo (maybe in combination with su) to gain root privileges.

Besides that, the manpage for the sudoers-file (the file configuring sudo), is one of the worst manpages I know, introducing EBNF and giving an EBNF-Grammar - well, not quite, there are manpages not giving any information at all, or even confusing more than giving clarity. But from the manpages giving correct information, this has a chance to be the worst.

Since one has to use visudo or sudoedit anyway to edit the /etc/sudoers, it would be nice to have an intuitive GUI for doing this. Something in which one can add users to executables that they may run, etc.

I wonder if there is such an UI already - on Ubuntu, I did not find anything like that yes, but maybe SuSE or RedHat has one. In any case, it would be a nice thing to have.