Neues in der Kategorie Debian

Finding Holes in Files - The FIBMAP ioctl

| Keine Kommentare
At least the ext[2-4] filesystems support files with holes. They can be created in several ways, one possibility is the seek-argument of the dd command:

$ dd if=/dev/urandom of=test bs=4096 count=10 seek=10
10+0 records in
10+0 records out
40960 bytes (41 kB) copied, 0.0126733 s, 3.2 MB/s
$ wc -c test
81920 test


So I was interested in whether it is possible to actally find these holes. ZFS and XFS have their own API for that.

For ext*, there is also a possibility to find holes. Basing on the FIBMAP Ioctl (and similar), you need to have the CAP_SYS_RAWIO capability (that is, usually you have to be root). If you only want to watch the holes in the files, you can use for example hdparm (as shown here):

$ sudo hdparm --fibmap test

test:
 filesystem blocksize 4096, begins at LBA 0; assuming 512 byte sectors.
 byte_offset  begin_LBA    end_LBA    sectors
       40960    8928120    8928199         80


Another possibility is to use the filefrag-utility, which is contained in the debian squeeze package e2fsprogs (as shown here):

$ sudo filefrag -v test
Filesystem type is: ef53
File size of test is 81920 (20 blocks, blocksize 4096)
 ext logical physical expected length flags
   0      10  1116015              10 eof
test: 2 extents found


Now if you really want to use your own program, here is a nice example code I found, on which I based my own code:


#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <unistd.h>
#include <errno.h>
#include <fcntl.h>
#include <sys/ioctl.h>
#include <sys/types.h>
#include <sys/stat.h>
#include <linux/fs.h>


int main (int argc, char* argv[]) {
  int fd, blocknum, blocksize;
  struct stat fileinfo;

  if (argc < 1) {
    fprintf(stderr, "Syntax Errof\n");
    exit(EXIT_FAILURE);
  }

  if ((fd = open(argv[1], O_RDONLY)) < 0) {
    int errnum = errno;
    fprintf(stderr, "Cannot open '%s': %s\n", argv[1], strerror(errnum));
    exit(EXIT_FAILURE);
  }

  if (ioctl(fd, FIGETBSZ, &blocksize) < 0 ) {
    int errnum = errno;
    fprintf(stderr, "Cannot get blocksize: %s\n", strerror(errnum));
    exit(EXIT_FAILURE);
  }

  if (fstat(fd, &fileinfo) < 0) {
    int errnum = errno;
    fprintf(stderr, "Stat failed: %s\n", strerror(errnum));
    exit(EXIT_FAILURE);
  }

  blocknum = (fileinfo.st_size + blocksize - 1) / blocksize;

  printf("Filename: %s\nBlocksize: %d\nBlocknum: %d\n",
         argv[1], blocksize, blocknum);
 
  int i;
  for (i = 0; i < blocknum; i++) {
    int block = i;
    if (ioctl(fd, FIBMAP, &block)) {
      printf("ioctl failed: %s\n", strerror(errno));
    }
    printf("%10d\t", block);
  }
  close(fd);
  printf("\n");
  exit(EXIT_SUCCESS);
}


The output:

$ sudo ./fibmap test
Filename: test
Blocksize: 4096
Blocknum: 20
         0
         0
         0
         0
         0
         0
         0
         0
         0
         0
   1116015
   1116016
   1116017
   1116018
   1116019
   1116020
   1116021
   1116022
   1116023
   1116024


And that is indeed a list of ten null-pointers and ten consecutive blocks.

Locking with Bash

| Keine Kommentare
Today was the first time that I ran into an unexpected race condition. With a bash script for a self made automounter. The automounter is invoked by udev. I did not expect udev to run the same script twice parallely, and so, an error occured.

The theoretical solution is simple, use locking. But in fact, I never have seen bash scripts doing something beyond

[ -e my_lock_file ] || touch my_lock_file && ...
to create locks, and this is dangerous, because between testing and touching a logfile, there might be the time for another test - the locking is not atomic.

I would have been surprised if this problem did not have a solution, and in fact, it does:

lockfile-create /tmp/my.lock
...
lockfile-remove /tmp/my.lock
Nice. Especially, it allows simple experiments with lockfiles through remote file systems.

Wtf happened to Debian?

| Keine Kommentare
Still, some people believe that if software is in the Debian repository, this is an evidence for its quality. This is a reason why I wanted to return to Debian from Ubuntu after my last data loss.

I do not know a single distribution that does not support my tablet from scratch, as Xorg does. Well, Debian Squeeze does not, so I had to upgrade to Wheezy - which is testing. It is software that just works in every other distribution, even on Arch Linux. And Squeeze breaks it?

Even on Wheezy (!!!) there is no Firefox 4, there is a Squeeze-backport, but testing still has 3.6 - what the fuck? I mean come on, it cannot be that "unstable". Furthermore, the Flash Player does not work properly: It plays the audio well, but often sticks to some single image when watching videos.

Ah, additionally: Not even Virtual Box works anymore! It used to work from scratch. I cannot get it to work, the device node has the wrong permissions - all set to root. I do not want to write my own udev-rules for that - why should anybody want that kind of setup? Usually, you have a group "vboxusers", which can access /dev/vboxdrv. Well, at least a reason to return to qemu again. Still, this is bad quality.

Then somehow - and I really do not have time to investigate on this - somehow the whole network configuration is totally fucked up. I can use my UMTS-Modem and WLAN mostly - but it behaves strange.

Five things that usually work from scratch, even under a minimalistic distribution like Arch Linux after installing the appropriate packages. Instead of quality, it seems to me like some of the maintainers have fun in breaking packages. For whatever reason.

The main reason for not using Ubuntu anymore is that the new version will use Unity by default, and I do not like Unity. I already tried Xubuntu (in Qemu, since VirtualBox does not work ...), and it looks like a promising alternative. So maybe I will soon switch to Xubuntu.
A problem that sometimes arises. The solution is simple, but it is hard to find, so I though I will write it here, such that other Googlers will find it.

You can simply do this by adding a policy-rc.d-script. That is, create a script /usr/sbin/policy-rc.d returning 101, before installing anything.

When using packages, the above Link sais one must use the alternatives-System, which sounds plausible.

Well, I have learned that today. It is always interesting to learn something new.