aufs for Puppy

[Nathan F]

The files needed to get aufs working in Puppy-2.14 are here. This tarball contains a patched initrd.gz with the aufs module and a pet package that contains the aufs utilities. Instructions:

• 1) untar this archive
  2) do a normal frugal install (should also work on a usb stick but I don't have one to sacrifice right now)
  3) replace the initrd.gz that Puppy will install onto the hard drive with the one in this archive.
  4) boot it up
  5) (optional) install the pet package with the aufs utilities

The last step should only be necessary if you want to test aufs for things like swapping out branches on the fly, otherwise it serves no useful purpose. Of course the ability to do that is the motivating factor for why I compiled it at all. I still haven't tested whether or not it will do the trick or not, but intend to do so very soon. In any event, I think it would be good to move Puppy towards using aufs anyway, as unionfs is know to be buggy and aufs implements new features.

This was surprisingly easy to get working, I wish I had tried sooner.

Nathan

[Lobster]

I am an oaf when it comes to aufs

are they related to awks?

wot is it?

[Nathan F]

Aufs is a total rewrite and new implementation of unionfs, with supposedly much fewer bugs and enhanced features. There is quite a discussion going on elsewhere about the ability to add or remove union branches from the main filesystem, and many were hoping that aufs would make this possible. This would enable us to add or remove squashfile extension while the system was running, which greatly improves the flexibility and modularity of Puppy. Even if it didn't work, it would seem that aufs is going to provide better performance and reliability than unionfs ever has.

This is provided for testing by developers, although I don't think it will harm your system in any way. But it's pretty useless for end users right now, since it just does the same thing that unionfs is already doing in Puppy. But it may enable a lot of other cool things in the future.

Nathan

[sunburnt]

Lobster; It's the newer replacement for UnionFS that joins these 2 file's: pup_xxx.3fs & pup_xxx.sfs.
As Nathan said, UnionFS is buggy & not as capable as AUFS (Another Union File System).

Darn it! Nathan beat me to it!

Nathan; I'll check it out tonight & let you know...

[Lobster]

thanks guys

[Nathan F]

sunburnt (& anyone else who may have already downloaded) -

I tried it out and it is possible to add and remove branches on the fly, in fact it works beautifully now. But the included unionctl script does not work, and it requires two small utilities that are not in Puppy by default. I re-uploaded the taqrball without the unionctl script, and with the utilities rev and comm, so if you have already installed the aufs utilities package please remove and then reinstall with the new one.

To add a branch:

Code: Select all

mount-FULL -o remount,append:/initrd/pup_ro5 /

To remove a branch:

Code: Select all

mount-FULL -o remount,del:/initrd/pup_ro5 /

Replacing /initrd/pup_ro5 with whatever directory you want to add or remove. Use mount-FULL rather than just mount, as mount is a shell script in Puppy that calls busybox mount. The mount-FULL binary is the proper mount program with all features.

I added and removed several branches without any problems, ran losetup to check for the next available loop without errors, and rebooted no problem. I have yet to see if it will cause any stability issues (ie the system eventually slows down like it did with unionfs after adding a branch) but after these tests I'm both impressed and highly optimistic. This means I can get back to work soon on my squash program and port all of the code from Puppy-1.xx to Puppy-2.xx, and we should be able to add and remove squashfile extensions at will. Just as we were hoping

Any testing anyone else can provide would be highly appreciated here, to confirm my results.

Nathan

[raffy]

Very very interesting - sunburnt must be smiling - no, laughing!

[kirk]

Nathan,

I tried it out and it is possible to add and remove branches on the fly, in fact it works beautifully now.

Wow, that sounds great. Tried to down load your package, but couldn't log in to you ftp server. Tried anonymous, no luck.

Thanks,

Kirk

[Nathan F]

I'll post it elsewhere with an http link.

Nathan

[kirk]

Link working now. Thanks Nathan.

[kirk]

Works for me!

And it seems to be slightly faster too.


I tried to remove a branch, but I got this:

sh-3.00# mount-FULL -o remount,del:/initrd/pup_ro4 /
mount: invalid option -- i
sh-3.00#


The -i seems strange, the pup_ro4 on my rig is the kernel source, so it shouldn't have had anything in use. I haven't read all the Docs yet, I'll work on it some more.

Should I have ran any of these scripts/binarys that where put in /sbin after the install?

aufind.sh
aulchown
auplink
comm
mount.aufs
rev
umount.aufs

[Nathan F]

Subnurnt has been experiencing this same error, and frankly I don't know where it's coming from. I'm not getting it myself, but I might be doing something slightly different. It's on my to do list today to check more into this.

For the record, Kirk, can you give me the exact commands you are using to mount a squashfile and add the branch, and then top remove it? I have a certain suspicion I want to check out.

I don't know what every script in there does (I still need to look over the docs more myself) but I can explain a couple of them.

When adding a branch using "mount-FULL -o remount" /etc/mtab does not get updated, so typing mount will not show the correct branches in your aufs union. The mount.aufs script is supposed to fix this, but I haven't gotten it to work yet. However, if you read /proc/mounts it will show the correct information so mtab doesn't really bother me too much right now.

The comm command is a *nix utility that is missing from Puppy, which is required by mount.aufs, so I put it in. An online man page for it is here, if you want to know what it does.

Similarly, the rev binary is another *nix utility that is not in the standard Puppy. It was needed by the unionctl script that is designed to emulate the command by the same name in unionfs. I could not get unionctl to work either, so it can probably be removed. The rev command basically reverses the letters in a string.

The auplink program has something to do with the symbolic links in an aufs branch, but I need to read more about it. The rest I don't know enough about (yet) to comment on.

Nathan

[sunburnt]

The mount command showed the correct aufs line for me.
The added branch was the last one of course, as it was added with "append".

Nathan; What Puppy-2 version are you using?

If it just has trouble with certain ones, that could be acceptable.
Although it would really be nice to know what the actual problem is.

[Nathan F]

I think I found it, but not sure. It seems the mount.aufs and umount.aufs scripts are actually used even if not called directly. When I looked in them they of course called mount rather than umount. I've re-uploaded the tarball after adding the full umount binary (rather than the busybox applet) as umount-FULL, and changing references in the aufs scripts to point to mount-FULL and umount-FULL.

The -i option was being passed to umount through the umount.aufs script, which is where I believe the errors were coming from (although I could possible be wrong).

I've also uploaded just the utils package, without the initrd.gz, so you can just replace that part.
aufs utils package

Let me know if this fixes the errors you have been experiencing, or makes any change whatsoever. I think this just might do it.

Nathan

[sunburnt]

KILLER !!! I'll test as soon as I get to my machine... let you know.

[sunburnt]

I'm wondering what the umount-FULL is for, as the aufs commands are all:

mount-FULL -o remount,append:/initrd/pup_ro3 /
mount-FULL -o remount,del:/initrd/pup_ro3 /

It didn't work when I tried: umount-FULL -o remount,del:/initrd/pup_ro3 /
It froze same as before without the "-- i" error.

I don't think it's right, so I didn't try: umount-FULL /initrd/pup_ro3


Nathan; Are we suppost to be using a different un-mounting command?

Are you using GraphPup for testing this... or what? Maybe it's different somehow.

[kirk]

For the record, Kirk, can you give me the exact commands you are using to mount a squashfile and add the branch, and then top remove it? I have a certain suspicion I want to check out.

I haven't tried to add a branch yet. I was trying to remove a branch that init had added. My first post has the exact (only) command I was trying to use. Could init be adding branches in some different fashon?

Couldn't download your new package, I'll try again later tonight. Don't have time to test right now anyway. I'll also try adding a branch and then try to remove it.

Thanks again.

[Nathan F]

I'm wondering what the umount-FULL is for

If you look in the script umount.aufs that comes with aufs, it calls umount. Since umount is a link to busybox in Puppy and cannot accept the -i option I replaced it with the full binary, and changed the umount.aufs script to place the correct call. The commands add or remove branches are exactly the same as I mentioned before, and the aufs kernel module handles the rest.

[sunburnt]

Sorry, I caught that from your other post, but thought it had another purpose.
It solved the error: Invalid argument -- i
But the crash after that is apparently something different.

If it works for you, need to know why that Puppy version works & not others.

Terry

[Nathan F]

I'm using 2.14 with the patched initrd.gz and aufs_utils package I posted.

Nathan