Page 1 of 4

Puppy RamBoot - Everything (including save file) in RAM

Posted: Mon 02 Nov 2009, 06:20
by jemimah
I've hacked the 4.3.1 init script to bring you a fabulous new feature.

The ramboot option enables you to load all of your sfs files and your save file into RAM and run completely diskless. Of course you need to keep these files small enough to fit in the amount of RAM you have, leaving some space for the OS to operate. You can then use the hdparm utility to shut down your hard drive (assuming you're running without a swap space) , saving power and reducing noise. Plus you'll get even faster performance. The downside is, none of your changes will be persistent (at least not yet, I may, in the future, provide a shutdown script that copies your save file back to disk). So the procedure is: set up your save file how you want it, then use the ramboot option when you're not planning on doing anything you'd want to save.

To try out ramboot, replace your initrd.gz with the attached one. Then add 'ramboot=1' to the kernel line in your menu.1st or syslinux.cfg.

EDIT: I've added a shutdown script that saves your file back to disk. This is alpha software so please don't test with important data. You need to have enough room on your boot disk for 2 copies of your save file, as the script makes a backup of the old file before writing the new file to disk. The script assumes your boot drive has the same drive name as it did when you booted. This can be a problem with usb drives that have been removed and reinserted, so pay attention.

Edit: I've updated a new version of the initrd.gz to work with subdirectories.
Edit: 12/30/09 fix for shutdown script
Edit: 1/06/10 fix for copy2ram

nice

Posted: Mon 02 Nov 2009, 12:34
by raffy
Excellent idea, since the available DDR2 sticks today have at least 1GB RAM storage.

If the user can tell where pup_save.2fs is located, s/he can copy it to hard disk (or other storage) before shutdown. But of course the shutdown script should be able to handle that, as you described.

EDIT: I booted initially without pup_save. At shutdown, it was trying to update a pup_saveOLDxx.2fs. Did you happen to hard-code your pup_save into the init script?

Posted: Mon 02 Nov 2009, 17:06
by jemimah
You can't copy it yourself because it's still mounted, and copying a mounted file system almost always causes severe corruption.

The script copies any file in your save directory that ends in *fs to RAM. If you want it to ignore a file, change the extension, or move it out of the directory with your SFS files.

It's supposed to throw an error message and die if you try to boot without a Pupsave. It must have found your old one and tried to use it. Nothing is hard coded, so I don't really know where it got that file from. It's also supposed to be forced into pupmode 12, so you really shouldn't see it spending a lot of time updating while you are shutting down.

Posted: Tue 03 Nov 2009, 10:01
by vg1
Sounds a lot like PG's 'groundhog' exsercise he did quite some time ago [around version 4.00, I think]. Except there you had the option at shutdown to save your changes to pup_save or not.

Posted: Tue 03 Nov 2009, 19:51
by jemimah
Thanks for the tip vg1. It appears my implementation is a bit different than what Pizzasgood did. So his shutdown script won't help me much.

This version does have dynamic RAM allocation, and it doesn't copy the files out of your 2fs file. It just copies the whole thing to RAM and mounts it from there. The trick will be to figure out how to unmount the RAM copy of your 2fs so you can copy it back to disk. You'll also need extra disk space to store a backup copy of your 2fs so you won't lose it if a power loss occurs during the copy-back process.

aufs

Posted: Wed 04 Nov 2009, 03:17
by raffy
"Logrow" idea is able to write back to mounted sfs:
http://aufs.sourceforge.net/logrow.txt

Kirk and Okajima seemed working on this. Kirk has Fatdog puplet here in the forum.

everything in RAM

Posted: Wed 04 Nov 2009, 03:50
by prehistoric
I was away from Puppy for some time, so you regulars may be able to help me with this.

I remember a radical suggestion about a version without the layered file system. It would act like a full install to a ramdisk. This would certainly be fast and simple. Saving might require some more work. One thing I liked was the end of problems with "whiteout" files. Having files mysteriously revert to earlier versions when there are bugs in the layering would have driven me nuts, (if I hadn't already been there.)

For machines like the Eee-PC, where you can count on having 1 GB of RAM, this could be the way to go, when the whole OS takes about 100 MB. There are several different ways of putting "everything in RAM". Can someone outline the current alternatives?

Posted: Wed 04 Nov 2009, 04:22
by Lobster
load all of your sfs files and your save file into RAM and run completely diskless
I wonder if there will be cross fertilisation with Quirky?
http://puppylinux.org/wikka/Quirky

Posted: Wed 04 Nov 2009, 04:37
by disciple
Can someone outline the current alternatives?
A couple of people have been working on using .sfs files without unioning them

Posted: Wed 04 Nov 2009, 04:46
by disciple
"Logrow" idea is able to write back to mounted sfs:
sfs? Are you sure? Maybe I don't quite understand, but it sounds to me like it would be for dynamically increasing the size of the .2fs file when you run out of space.

Posted: Wed 04 Nov 2009, 14:50
by jemimah
The simplest way to do a full install to ram is to dump your whole filesystem intro the initrd. I have done this successfully, but I abandoned the idea because the above way is superior. Boot time is very slow, because first you have to load your giant initrd to RAM and then you have to decompress it. So not only is it slow, but you need enough RAM to hold both the compressed and uncompressed initrd at the same time.

There's also this method: http://www.justlinux.com/forum/showthread.php?t=151619

However, using squash files is better because they don't need to be decompressed, so boot is fast, and a lot more data fits.

I've done some more research and as far as I can tell, it's actually impossible to unmount the root filesystem so you can unmount your save file cleanly. I suppose you could copy the individual files out, but it's painfully slow. I plan to experiment and see if copying the save file back to disk, while it's still mounted, but after all processes have been killed, results in any kind of corruption.

fatdog logrow

Posted: Wed 04 Nov 2009, 20:31
by raffy
Here is rexterd's experiment with Fatdog and logrow:
http://www.puppy2.org/aufs/Fatdog112-logrow-aufs2.iso

The discussion of the aufs ideas by rexterd starts here and continues there..

Posted: Thu 05 Nov 2009, 02:00
by jemimah
I posted a shutdown script for people to test. It can't unmount the save file totally, but it seems to be alright since it waits until all the processes that would be writing to it are killed. I've tested it quite a few times and all seems well.

Posted: Mon 07 Dec 2009, 19:09
by flugwelpe
hm, regarding the unmounting problem - shouldnt it be possible to switchroot to a minimal system and then the original root doesn't matter?

Posted: Mon 07 Dec 2009, 20:02
by jemimah
I looked into it, but didn't find much useful documentation. If anyone knows of any similar projects that do that successfully after starting the main OS, I'd love to take a look.

The shutdown script was more of an experiment than anything. I still recommend running without it, and saving important files to USB or mounting a hard drive to save files outside your 2fs file. Running totally in Ram and trusting that the shutdown script is going to work every time, makes me nervous.

Posted: Thu 10 Dec 2009, 04:27
by seaside
jemimah,

I've being yearning to try out RamBoot and finally got around to it and here's what happened.

Booted up with the new initrd and saved a minimal pupsave of 32m to a usb hard drive. Rebooted with ramboot=1 option and got this error..

"superblock could not be read.....try alternate superblock e2fsck -b 8193 <device> " (I later tried that with no success)

It then failed to mount pup-rw, loaded pup-431.sfs, layered,and then went to kernel panic

When booting without the ramboot=1 option, all goes as expected and the new pupsave file loads without a problem.

Any ideas as to what might be happening?

Thanks,
s

Posted: Thu 10 Dec 2009, 05:07
by jemimah
Wow no, that's never happened to me!

Did you get that error after it loaded all the files to RAM? What does the rest of your bootloader config look like? Are you using a regular ext2 save file?

I've Got The Same Problem

Posted: Thu 10 Dec 2009, 14:06
by Mstar
Jemimah,

I thought it was just me. I have exactly the same problem. Machine is a P4, 2 gig Ram, Intel D865GBF Mobo, ATI x1650 video card. Puppy 4.3.1 frugal.

I've made a save file by booting from the RamBoot Initrd.gz but started with Pfix=Ram and then saved. Get exactly the same result when I boot with Ramboot=1. Also chokes on 2fs file made before trying RamBoot. Both will boot from the "normal" Initrd.gz.

In both cases I'm booting from the same Grub Menu.lst file, the only difference is editing before booting to add Pfix=Ram and eliminate Ramboot=1 when I created the 2fs save file to see if the original was somehow corrupted.

Posted: Thu 10 Dec 2009, 16:19
by seaside
Jemimah,

Since the pupsave file works when the "ramboot=1" isn't used, perhaps it has something to do with the actual filesystem that the pupsave is on. My pupsave file is on an "ext3" filesystem.

Is there any ramboot load coding that might get confused in this case?

Thanks for any insights you may have,
s
(Mstar, I'm also happy it's not just me :D )

Posted: Thu 10 Dec 2009, 16:24
by jemimah
I tested exclusively with ext2 filesystems, so that's probably it. Let me see if I can figure out how to fix it for ext3.

Edit: Wait.. Mstar you said you're using a .2fs, so it's probably not ext3 or at least not exclusively.