How do I get Puppy to run entirely in RAM with pup_save.3fs

[FuturePerfect]

I am currently running Puppy 2.14 from disk (using PUP4DOS with FAT32 file system & pup_save.3fs); alternately, I'm testing Puppy 4.0 from CD (using PFIX=RAM so Puppy entirely in RAM):

1) What do I need to do to make sure all my Puppy applications and libraries load from and run entirely in RAM?
(for example, have pup_save.3fs size not exceed RAM size of PC?)

2) Can I shrink an existing pup_save.3fs file to get rid of any empty space in it (so it might fit into RAM?)?

3) Do I need to / is it possible to flush RAM (other than rebooting), after opening & closing different applications, to make sure additional applications loaded don't run out of RAM?

Thanks for any help you can provide.

[sunburnt]

Hi FuturePerfect; I'll do my best to explain my understanding of this.
First, just because you boot from CD doesn't mean Puppy's all in ram.

1) The kernel (vmlinuz) & the inital image file (initrd.gz) is loaded in ram.
2) Ram is checked to see if there's enough space for the SFS file.
3) Ram is checked to see if there's enough space for the Save file.

So 128mb or greater of ram to load the SFS file, & more ram's needed for
whatever's the size of the Save file, a 256mb Save needs at least that much.
So it's safe to say that 512mb of ram's needed to fully load a 256mb Save.

If there's a swap file, then all of virtual swap ram can be filled, but this just
makes for waste as the files just get shoved back on to the swap drive.
So it's a good idea to have lots of ram in the PC to avoid this.
Unfortunately I don't think you can control the file loading, it's all automatic.
If this is so... It should be corrected in future Puppy releases.

[Pizzasgood]

You're right about the first two, but not the third. The pup_save file is never loaded into ram on any official Puppy, nor any puplet that I'm aware of.

I've made a couple posts/discussions in the past suggesting how someone could modify the boot scripts to have the pup_save file copied into ram. In the most basic method, you would simply change things so that Puppy creates a larger ramdisk when it boots, then track down the code that mounts the pup_save file and have it first copy (literally, with the cp command) the pup_save.2fs file into the same directory as Puppy copied the pup_xxx.sfs file, which is where the ramdisk is mounted.

Then you would also have to tweak the shutdown scripts to copy that file back to the harddrive. This probably can't be done cleanly though. You might also want to write a script that lets you make an immediate copy of the pup_save file while the system is running. You could even set up a periodic daemon to do it every 30 minutes or so, very similar to what Puppy already does when installed on a flash drive.

[sunburnt]

Yep; I'm not surprised at that. Apps. load from both the SFS & Save files.
But only the main boot SFS file's in ram, so addon SFS files load slower also.

The problem of the files being shoved back on to the HD swap is a big one.
There's little point in "running in ram" if it's not in ram, it's just HD intensive.
So even the main SFS file shouldn't be in ram unless there's lots of ram.

[Pizzasgood]

Unfortunately I don't think you can control the file loading, it's all automatic.

Puppy 4.00 should let you supply the puppy pfix=noram option to make Puppy mount the pup_xxx.sfs file directly, without copying to ram. Some derivatives also offer that option.

[Bruce B]

Yep; I'm not surprised at that. Apps. load from both the SFS & Save files.
But only the main boot SFS file's in ram, so addon SFS files load slower also.

The problem of the files being shoved back on to the HD swap is a big one.
There's little point in "running in ram" if it's not in ram, it's just HD intensive.
So even the main SFS file shouldn't be in ram unless there's lots of ram.

All programs should be running in RAM. Not all programs and other files get pre-copied to RAM, with the exception of using the pfix-ram argument, if there is enough room to fill the request.

Big files don't get shoved back to the HD swap, unless Linux determines it necessary for some reason.

[sunburnt]

Right... Like a PC with little ram to begin with.

After a few apps. are run, the SFS file'd be moved from initramfs to swap space.
I assume the loading of the SFS file is set to leave enough room to cover this.

[Bruce B]

Actually, if you have enough RAM and use an emulator, it would be very simple to RUN everything in RAM.

If you want to save changes you'd have to copy things back before rebooting, because RAM files tend to disappear between boots.

[FuturePerfect]

Sunburnt, Pizzasgood, Bruce B: thanks for all the information so fast! I'm just now getting to read; hope you guys still here!

Being a beginner, your knowledgeable comments are a little above my head. Maybe you could help me solve just 1 problem?

I'm using an old Dell Inspiron 3200 Pentium II 266 MHZ laptop with a little over 128MB memory. I'm trying to decide:
a) if upgrading to 256MB will help my speed with Puppy 4.0;
b) especially get Firefox to load faster (remove SeaMonkey?)

a) On Barry's developer blog, it looks like I MUST have 256MB to have ANY change of getting any RAM residency--April 26 2008:

Loading into RAM
Floborg asked whether Dingo copies the pup_xxx.sfs file into RAM on a 128MB system. The answer is no. A PC with 256MB is required. Most PCs have RAM capacities that are a multiple of 2, so after 128MB there is 256MB, etc. However, some older PCs are configured with a mix of RAM modules and have a total somewhere between 128MB and 256MB -- in that case it is likely that you will fall below the cutoff point and the pup_xxx.sfs will not be loaded into RAM.

For people in the situation where the pup_xxx.sfs is not copied into RAM, when you create a 'pup_save' file, Puppy asks if you want to copy the pup_xxx.sfs file to the same place as the 'pup_save' file. Or, if upgrading an existing pup_save, you can manually copy the pup_xxx.sfs off the CD. Whatever, if the pup_xxx.sfs exists in the hard drive then the CD will be freed for other uses after bootup.

However, you still won't get it loading into RAM. I set the cutoff at 256MB, as some applications, for example SeaMonkey or Firefox, need lots of RAM.

A little sad Puppy 4 now needs a 256MB PC to get anywhere near RAM residency speed. No reclamations of old RAM-starved machines with Puppy 4. But Puppy 4 apps primo!; Puppy 2 still RAM frugal.

Barry's write-up above triggered a few questions on b) above:

1) Re getting Firefox to load faster w Puppy 4, can I / how can I set things up so Firefox and its libraries load from RAM w 256MB?

*** Would removing SeaMonkey give space to help this happen?

2) You guys seemed to say that some applications are put in pup_400.sfs (which Barry says should load in RAM in a 256MB PC) BUT some applications get put in pup_save.3fs--any way to know which is where (or get Firefox and libraries in pup_400.sfs?)?

3) Pizzasgood: do you have any further information on your PET-Be-Gone working OK with Puppy 4? Especially removing SeaMonkey which I don't like (and Sylpheed since I use web mail). Latest post I found seems to say there's a good chance it will work OK:

http://www.murga-linux.com/puppy/viewto ... 2&start=15

Thanks again for all your help.

[mf2008]

3) Pizzasgood: do you have any further information on your PET-Be-Gone working OK with Puppy 4? Especially removing SeaMonkey which I don't like (and Sylpheed since I use web mail). Latest post I found seems to say there's a good chance it will work OK:

http://www.murga-linux.com/puppy/viewto ... 2&start=15

I think you might be referring to my post in that thread. I've been using petbegone with Puppy 4.00 since yesterday. No problems. I've been trying to remove as much stuff as I can to create a Puppy version that does nothing except run Firefox. (I'd like to run it as an "appliance" in a VMware Player on Windows, like a quarantine zone.).

I found that using VMWare Server (free, or Desktop, costs $) is really useful when experimenting with what can be removed. There are a lot of things that are remove. After that, I tell VMWare to take a snapshot. Remove something with petbegone, restart Puppy. If it broke something, I can revert to the snapshot. Very painless.

As someone noted elsewhere, if you're using the standard/Frugal install you won't notice notice any reduction in space. You have to do the menu->setup->remaster Live CD and reinstall in order to get the benefit of whatever you uninstalled with petbegone.

If you uninstall a lot, then you have to delve into redoing the menus to remove top-level items. Maybe also the configuration files associated with packages that were removed. I'm probably going to try building a new Puppy from unleashed to see how hard that is. But, if I can't handle that, at least I know I can get what I want using petbegone. (Thanks pizzasgood).

Mark

[Pizzasgood]

2) You guys seemed to say that some applications are put in pup_400.sfs (which Barry says should load in RAM in a 256MB PC) BUT some applications get put in pup_save.3fs--any way to know which is where (or get Firefox and libraries in pup_400.sfs?)?

With one exception, everything that makes up Puppy is inside the pup_xxx.sfs file. (The exception would be the things used at the very beginning of booting, including the kernel, initrd.gz, and isolinux).

The pup_xxx.sfs file is read only (it's compressed), so in order to allow the user to make changes we have the pup_save.2fs file. Whenever you add a new file to Puppy, the new file is stored in pup_save.2fs. Also, if you change a file, the changed copy will be stored in pup_save.2fs. It handles deletions by creating a "whiteout file" that blocks the original file from showing up. Then Puppy uses UnionFS or AUFS (depending on Puppy version) to merge pup_xxx.sfs and pup_save.2fs into the single filesystem you see.


Remastering builds a new Puppy with a pup_xxx.sfs file that includes most of the data you had in your pup_save.2fs file.



So, if you install Firefox, that means Firefox will be inside pup_save.2fs. You can also use PET-Be-Gone to remove Seamonkey, which won't initially help your ram usage. Then, you can use the remaster script to build a new Puppy which will have Firefox but not Seamonkey. After you do that, you should delete (or rename) your pup_save.2fs file and start over. Otherwise the copy of Firefox in pup_xxx.sfs will be ignored in favor of the one in the pup_save.2fs file.

[Bruce B]

Pizzagood,

Mostly I use Full Install

I have Puppy installed as Frugal also, but none of this has anything to do with a CD Disc, except for the initial install.

I don't see any advantage of deleting programs, if the delete is only an optical illusion. If that's what I want I could dereference the menu items.

If I really wanted to delete programs, I could remaster the CD, but from where to I want the programs deleted? The answer is most likely from the pup_xxx.sfs file.

Although I've never tried it, it seems easy enough to just build a new sfs file with the wanted changes. Would there be a problem with doing that?

Bruce

[Pizzasgood]

That would work fine. There are a couple apps out there for creating new .sfs files and at least two for "editing" existing files. Doing it manually isn't very difficult either.

[Bruce B]

Puppy runs in RAM with the right boot option and if you have enough RAM.

What's the problem with simply using that option?

The time it takes to setup the X server and Internet?

There are more programs you want to run than available with standard Puppy?

Saving files back to disk?

Something else?

(Pizzagood thanks for the answer)

[FuturePerfect]

Forgive my for unavoidable delay--hope folks still read this.

mf2008--thanks for referencing the remaster menu and the detailed petbegone install instructions--I didn't know about these;

pizzasgood--thanks for explaining when/why delete pup-save.3fs

Since I'm running Puppy 4 on a 10-yr-old laptop (you get attached to them!) with limited memory, I've bitten the bullet and ordered memory to expand it to the max--from 128MB to, wow, 384MB. Hopefully this enough for skinny all-memory-resident operation.

Now that I've had time to study these posts, I'd appreciate any corrections to these hopefully least-technical "cookbook" steps to make Puppy 4 load & run all in memory for speed on old PCs but in the least memory since they don't have much. This is for non-technical users like me. Also could you answer QUESTIONS at end?

I'll use this cookbook when I get my 384MB, noting any changes needed here. A relatively easy way to a memory-resident Puppy 4:

1) Remove any unwanted packages you ADDED to base Puppy. Use menu selection Setup > Puppy package manager to UNinstall any Installed packages you don't want any more (bottom, package manager window) or will cut to save space to get Puppy in memory.

2) Download, configure, install 2 packages needed (to remove any unwanted BASE Puppy 4 packages); thanks to mf2008 for b)-e) here:

---a) Use Setup > Puppy package manager to download/install prerequisite Opkgs_db-4.00 Package: push in radio button to left of All at top left and, near right of radio button for All; click on Opkgs_db-4.00 in the Package list to begin the install.

---b) Execute these commands [to get Opkgs_db-4.00 operational]
------[suggest use Utility > Rxvt terminal emulator to do so]

------pet2tgz 0pkgs_db-4.00.pet
------tar -zxf 0pkgs_db-4.00.tar.gz
------mv 0pkgs_db-4.00/root/.packages/* /root/.packages/.

---c) Download petbegone-0.2.pet from the first post in this thread:
------http://www.murga-linux.com/puppy/viewtopic.php?t=17035

---d) Click on the petbegone-0.2.pet package to install it.

---e) Reboot and find petbegone in the Start->Setup menu.
------[pizzasgood suggests Use Shutdown > Restart JWM as alternative]

3) Before you go to next step 4), if you want to be double safe, make a Backup copy of a file [click the eye icon in Rox--if done correctly you see the word (All) in the top menu bar--to be sure you can see hidden entries that start with the symbol .]: Backup:
---/root/.packages/livepackages.txt
---(although the application in step 4) automatically makes a backup of this file, you might want other copies)

---*** Maybe backup copies of pup_save.3fs and pup_400.sfs files too [you may need to shut down puppy and do this externally]

4) Remove any BASE Puppy 4 packages you don't want. To do this

---a) Use menu selection Utility > PET-Be-Gone

---b) Select & remove packages you wish to remove; press continue

---c) Use menu selection Shutdown > Restart JWM

---NOTE: an explanation by the author, pizzasgood, of what's going on when PET-Be-Gone runs, is the first post here:
---http://www.murga-linux.com/puppy/viewtopic.php?t=17035

5) To change the size of the key Puppy 4 file pup_400.sfs and make sure all your applications get there to be memory resident (that is, if they will fit into the RAM you have) do this:

---a) Make backup copies of pup_save.3fs and pup_400.sfs files [you may need to shut down puppy and do this externally]

---b) Use menu selection Setup > Remaster Puppy live-CD
------(don't worry if you don't have a CD burner in your Puppy PC--this lets you write an ISO file to disk--copy to flash drive then to a PC that DOES have a CD burner and burn the CD there)

---c) when you finish reading & following all thorough prompts accompanying each careful step in b) above, you should have a CD with the key new pup_400.sfs file on it (or, if your PC doesn't have a CD burner, copy ISO file you make in b) & burn elsewhere).

---d) After c), be sure you delete the pup_save.3fs you currently have--remember, you made a copy of it in a) above. (If you don't delete this pup_save.3fs, Puppy will try to read this file, which is not memory resident, and your memory residency is incomplete.)
(pizzasgood May 17,2008 post above in this thread explains best)

---e) After you've done all this, don't forget this entry (if you don't, Puppy won't all load into RAM, even after all this work!) i.e. last boot param(eter) entry before Puppy boots must be:
PFIX=RAM

IMPORTANT NOTE: When [/If] you add (or delete) subsequent Puppy applications, the pup_save.3fs file--that never copies itself to RAMdisk--will be read every time you execute such a new application. To free space in the desired memory-resident pup_400.sfs (and have such new applications load from this RAM), you have to follow these steps again:
3) (extra backups are always good!)
4) (if you can cut apps to make space for these subsequent apps)
5) (process that frees deleted app space and RAMs new apps)

Some things have so many options depending on how you run Puppy (from CD, USB, full or frugal or PUP4DOS hard drive install) that this can only be a general guide. If you are brand new to this, such things as how to burn an ISO, may overwhelm at first. But things that drove me crazy at first--such how to click the eye in Rox filer; trying to find programs in menus, are noted here.

QUESTIONS:

i) pizzasgood said:

Remastering builds a new Puppy with a pup_xxx.sfs file that includes most of the data you had in your pup_save.2fs file.

Does this mean some data still in pup_save.3fs?; if so, what?; is this data accessed only upon boot (e.g. Xorg parameters) or at other times?; would this make above memory residency fruitless?

ii) How do you know that an app won't fit into memory?; will the whole pup_400.sfs just not fit into memory?; any way to know something doesn't fit into memory except look for disk accesses?

iii) Is the above correct that the PFIX=RAM parameter must still be supplied?

iv) Is there a way to predict how big the pup_400.sfs will be even though it's compressed (to see in advance if remaster a waste)?

v) pizzasgood: my pup_save is a .3fs file whereas you refer to yours as a .2fs file--why is this?

[Pizzasgood]

i) The remaster script will leave out some of the configuration data, because it would be machine-specific. I don't remember exactly what it leaves out. I think maybe /etc and /root. There should be a stage of the remaster script that pauses and says something like "the new filesystem is in /tmp/root" or some-such. During that stage you can copy in files to ensure they are included. If you decide to copy in the entire /etc directory, make sure to delete the /etc/.XLOADED hidden file from the new location (otherwise it won't automatically boot into X).

ii) I guess you have to guess or check the app's documentation. If there isn't room for the pup_400.sfs file, the file won't load into ram and will be read from the harddrive.

iii) Do you mean after you have everything finished? Once you're done, if there isn't a pup_save.2fs file you don't need to use pfix=ram. If there is one, using pfix=ram will make Puppy ignore it. So if you want to be sure that Puppy is running completely in ram, and already know that there's enough ram for pup_400.sfs to load, then you only need to use pfix=ram when there is a pup_save.2fs file on the drive.

iv) You could check how big the entire filesystem is, then cut that almost in half. That will be somewhere in the range of the new compressed file.

Code: Select all

du -xsh /

(the x tells it not to also count the mounted filesystems, the s tells it to just print a summary, and the h tells it to output a "nice" number)

v) I say pup_save.2fs because that's what people usually use now. Some versions of Puppy used pup_save.3fs though. The difference is that .2fs files use a ext2 filesystem and .3fs files use ext3. The difference between ext2 and ext3 is that ext3 has a journal. That makes repairs to the filesystem faster. Unfortunately, from what I understand it is dangerous to use a journaling filesystem in a filesystem image. So because of this, Barry started using ext2 instead.

[zauserio]

A very interesting thread!

I'm also keen on deleting BASE programs from Puppy. My question is: Is it also possible and maybe easier to make following:

1. Full install
2. Delete programs which are unwanted
3. Remaster a new *.sfs file

Thanks a lot!
zauserio

[Pizzasgood]

Full installs normally don't cooperate with the remaster script. I have a modified version of it here which should work for Puppy 3.xx. There's an untested version for Puppy 4.00 further down the page.
http://www.murga-linux.com/puppy/viewto ... 172#182172

If you're going to delete programs and then remaster, it doesn't matter whether you use a full or other install, other than the above remastering quirk. Even though deleting things in a non-full install doesn't truly delete them, it does delete them enough that Puppy, the remaster script, and all other programs think that they are deleted.

So it seems to me that the easiest method would be to boot from the CD, install 0pkgs_db and PET-Be-Gone, uninstall any unwanted apps, and then remaster. No need to install to HD and fiddle with Grub that way.

[zauserio]

Thank you for your detailed answer. I try your described way.

Best wishes,
zauserio

[FuturePerfect]

pizzasgood: Thanks for your answers and my apologies for taking so long to post a thank you. My job sucked up a lot of intense time plus I was trying to finish/document a new Remaster to ask more questions and it all took longer than I expected.

In the future I will post a thank you immediately and explain that it may be a while before I can post again.

Hoping you'll still answer my 5> questions interspersed below.

Now that I've done a Remaster and figured out corrections to earlier Remaster notes for newbies above, I'm getting close to posting revised Remaster notes for newbies here--that's the least I can do for all the help I've gotten from you guys.

However, I had some Remaster problems I need follow-up suggestions on before I can finish revising my notes.

First of all, pizzasgood's PET-Be-Gone worked flawlessly in removing base Puppy 4 applications I wanted to be gone--Seamonkey, Sylpheed, Abiword, and Gnumeric--and the Remaster burned me a CD with new pup_400.sfs clean of these applications yet including new applications I added--firefox-3.Beta.5.pet (updated online to Release Candidate 2), Textmaker, & Planmaker.

The main problems I have is that many settings changes I made to Puppy packages did not carry over to the Remastered CD.

Re: setting changes that didn't Remaster-- 5> items especially

for base applications:
-----ROX no list view; no small icons; no settings for no single click
-----mtPaint Palette & toolbars not turned off (as set in View)
-----Gxine no load [click on any Gxine in /usr/bin fails too)
1> -Help won't load (> some way to associate with Firefox?)

for new applications:
2> -Firefox no addons, preferences, or updates (> any ideas?)
------Textmaker & Planmaker no settings (think I just saw a fix)

Note: I booted the Remastered CD using pfix=ram to be double sure no old .sfs or pup_save.2fs files on the disk used AND because saw no new pup_save.2fs generated anywhere in Remaster.

3> Is it really possible to have NO pup_save.2fs if all above settings transfer properly in Remaster and no further changes made or do I just save a small pup_save.2fs file for such things as ROX configurations above that only load once or infrequently?

4> Do I add (or how would I copy) some files from where at what point so my missing settings above are in the new pup_save.2fs?

Example: Are there any files I need to add at a middle Remaster screen that says, in part, regarding additions you can make for Remaster:
"... take a look at /tmp/root right now, and possibly add anything that you want from /root (or remove something!)..."

Example: Are there any files I need to add at a following Remaster screen that says, in part, regarding customizations I saved for my hardware:
"...you can now modify any files in /tmp/etc folder. This is just about to be added to /etc in the .sfs file..."

5> How do I enter boot parameters in isolinux.cfg that Remaster allows me to modify from default before it is burned to CD?

Example: at the step when in the Remaster where I clicked on Yes to add kernel parameters, isolinux.cfg opened in the Geany editor showing the last 2 default lines as:

append initrd=initrd.gz pmedia=cd
timeout 50

What syntax do I use in how many lines and where do I put it, to add these puppy boot parameters:

ide=nodma acpi=off pfix=usbcard

Thanks again for any help you can provide.