Loading SFS to RAM Without Requiring 2X RAM (Solved)

[bigpup]

Do you know about this:
http://bkhome.org/blog/?viewDetailed=02034
This is not fixed in the latest Lucid 5.2.

[emil]

Barry Kauler created an enigma .
I suggest to go back to the code. There are the facts. Extract initrd.gz and view the init script.

jamesbond already posted the part of the code where it is decided if the pup.sfs is loaded to Ram or not. Setting the COPY2RAM flag is the crucuial line:

Code: Select all

[ "$COPYCONTENDER" = "yes" ] && [ $RAMSIZE -gt 220000 ] && [ $RAMSIZE -gt $MINRAM2CPY ] && COPY2RAM="yes"

Ramsize and Minram2cpy were already discussed. Last remaining condition is the flag COPYCONTENDER: This is empty by default but get's modified just in the few lines before the abovementioned conditional:

Code: Select all

if [ "$PNOCOPY" != "yes" ];then
  [ "`echo -n "$ATADRIVES" | grep "$pdPATTERN"`" = "" ] && COPYCONTENDER='yes' #test if not fast internal drive.
  [ $PUPMODE -eq 5 ] && COPYCONTENDER='yes'
  [ "$DEV1FS" = "iso9660" ] && COPYCONTENDER='yes'
 fi
 [ "$PCOPY" = "yes" ] && COPYCONTENDER='yes'

I translate this as follows:
default is: don't copy to RAM

Under the assumption that enough RAM is present and
a) pfix=nocopy is NOT explicitly given:

• 1) if there is a slow drive copy to RAM ( how does that work?! )
  2) if Pupmode 5 (first boot) copy to RAM
  3) if the device is of type iso 9660 (CD-ROM) copy to RAM

b) pfix=copy is explicitly given

• copy to RAM

I think Puppy is so ingenious with its different modes, layers, flexibilities to install that it is hard not to be confused. At least I was not aware of this exact behaviour. I always vaguely thought about Puppy is running "mostly" in RAM "somehow".
But please, if trying to find a possible bug, lets stick to the code and possibly get some hard facts for debugging. Everything else is speculation.

Lots of the variables in the init script are written to the file /etc/rc.d/PUPSTATE, so it is easier to follow the code.

I modified the "debug" version of initrd.gz a bit. It writes now the file debug.log to /root.
emil

[l0wt3ch]

@ emil: Well, I remastered the disc with your initrd, and, while I couldn't find the debug file it was supposed to put in root, it seems to have fixed the problem. The sfs now loads into RAM with "puppy pfix=ram,copy".

So I remastered the Studio 3.3 non-realtime disc with your initrd, but no change. According to the posters on here, that's normal. 373 x 2 = more than 512 MB. But why does Studio 3.1, which was 394 MB and based on 511, load fine on the same machine?

I give up.

[john biles]

Hello l0wt3ch,
Maybe because your test compressed files are all the same ie 5 x same or 15 x same the compression app takes this in to account and knows it just has to unpack the same compressed file 5 or 15 times?

Have you tried making bigger compressed files and seeing if the final iso grows the same amount?

I've seen 1mb files compressed to 50kb's and thought something must have gone wrong. I uncompress them again and there fine?

Just a thought.

[l0wt3ch]

Hey John.

I don't think it matters that much. I had expanded the size of the sfs by 47 MB, and it wouldn't load into RAM with "puppy pfix=ram,copy". I thought that settled it. But now I used Emil's initrd (which had a debugging log feature), and somehow it fixed it. Not my original problem of why my 373 MB sfs won't load into RAM on boot with 520 when it would with 511, though.

[jamesbond]

l0wtech,

Barry has a lot on his plate, see http://bkhome.org/blog/?viewDetailed=02146 - and he specifically mentioned he won't be able to respond to PMs as fast as he'd like.

Back to your problem, though. I get Pstudio 3.1 (which is based on Lupu 511?) - and guess what? It has a different init! (older version I guess).

It will load into RAM, because before the check for RAMSIZE and MINRAM2CPY, way way earlier, it has this code

Code: Select all

#v405 decide whether to copy sfs's to ram...
COPY2RAM=""
COPYMSG='copying to ram' #purple
#v4.00 lowered rom 230000 to 220000... v403 added PUPSFSDEVMNTPT test... v404 explicit PCOPY needed...
[ $PUPMODE -eq 5 ] && PCOPY="yes" #well, override on first boot.
[ $PUPMODE -eq 77 ] && PCOPY="yes" #v406 multisession dvd.

#100406 this came about because aufs did not work if an sfs mounted via loop device as one layer
#was actually resident on another layer. however latest patched aufs may allow this...
if [ $KERNELSUBVER -lt 33 ];then
 #v404 absolutely must copy to ram, otherwise layerfs conflict...
 [ $PUPMODE -eq 6 -o $PUPMODE -eq 7 ] && COPY2RAM="yes"
 [ "$COPY2RAM" = "yes" ] && COPYMSG='forced copying to ram' #purple
fi

#w482 lower this again, so multisession will work in 256MB system...
#[ $RAMSIZE -gt 260000 -a "$PCOPY" = "yes" ] && COPY2RAM="yes" #256MB system. note, only checking physical ram. w003 incr. from 220000.
[ $RAMSIZE -gt 220000 -a "$PCOPY" = "yes" ] && COPY2RAM="yes" #note, only checking physical ram.

Basically, this one says that in PUPMODE=5 (ie first boot livecd), as long as your RAM is bigger than 220MB, it will copy to RAM. Because this check is done earlier and it sets the COPY2RAM variable directly, it will override any later checks - thus, no matter what, it will always try to load to RAM. This code has been known to cause problems - if you make a 700MB SFS, it will still try to load to RAM on a 512MB system - which will fail of course - you'll end up with kernel panic.

All your questions answered, now back to Puppy Studio, shall we?

cheers!

[l0wt3ch]

Do you really think that if I swap out some piece of code out of the initrd's with that one it'll fix it?

[jamesbond]

That's all right - I just pointed it out in case you miss it (sometimes Barry can be prolific in writing his blogs that we all miss his announcement one way or another).

Anyway, what objective would you try to achieve - are you trying to get that 3.1 does (ie, 397MB and still load into RAM), or do the right thing in Lupu 5.2?
If you would like to do what 3.1 does, I wouldn't recommend that you swap the code - who knows what other dependencies or side effects in between. Instead, let's use what is already in Lupu 5.2 code and work it out from there. Look at this code this part:

Code: Select all

MINRAM2CPY=$(($SIZESFSK * 2)) #100222 technosaurus: in case of very big puppies. 

Instead of making it 2 x SIZESFSK, just make it equal to SIZESFSK

Code: Select all

MINRAM2CPY=$SIZESFSK

That should do the job.

Good luck.

[emil]

Hey it is great that the problem seems fixed,
and I see no objection to change the script specifically for puppy studio, so that it loads to RAM the way jamesbond mentioned. But I don't know what happens if you execute a programm which needs some extra RAM ..
If you want to give a bit more slack say:

Code: Select all

MINRAM2CPY=$(($SIZESFSK +50000))  # this is SFS size + 50 MB

Try it yourself, it is easy to make this change (Howto in first link). If you have problems I can do this to make a customized initrd.gz. I can also remove the debug echos if you like.
Personally I learned lots from your problem, so thank you for going public !
emil

[l0wt3ch]

@jamesbond: Thanks for the suggestion, that works! 3.3 now boots straight into RAM with 512 MB of memory. (I didn't test if "puppy pfix=ram,copy" works, but it boots into RAM now, and that's all I care about.) PuppyStudio 3.3's performance advantages are back!

@emil: Thanks for your efforts in making that debug initrd. Appreciated!

And a big thank you to everyone else for taking the time to stop by and contribute!

[Flash]

So does this thread belong in the Bugs section of the forum or should I move to to, for instance, the Cutting edge section?

[emil]

Definitly it is not a "bug", but "cutting edge"? At least it sheds light on very basic puppy concepts, and it provides insight for new and experienced users alike

[jpeps]

I didn't test if "puppy pfix=ram,copy" works, but it boots into RAM now, and that's all I care about

The boot only recognizes the last pfix statement, so doesn't work if followed by pfix=fsck, etc. (tested in lupu, frugal).

[Karl Godt]

(4) on some systems s3_bios kicks video into text mode, and
acpi_sleep=s3_bios,s3_mode is needed.

/linux-2.6.36/Documentation/power/video.txt

NOT "... vga=normal acpi_sleep=s3_bios acpi_sleep=s3_mode psubdir=PSUBDIR ..."

BUT:

Driver built as module:
The module icn.o can be configured during "insmod'ing" it by
appending its parameters to the insmod-commandline. The following
syntax is accepted:
portbase=p membase=m icn_id=idstring [icn_id2=idstring2]

Driver built into the kernel:
The ICN driver can be configured using the commandline-feature while
loading the kernel with LILO or LOADLIN. It accepts the following syntax:
icn=p,m[,idstring1[,idstring2]]

init:253:mount -t proc none /proc

#

Code: Select all

CMDL=`cat /proc/cmdline`

#

Code: Select all

echo "$CMDL" | tr " " "\n" 

panic=56
root=/dev/sda7
ro
vga=0x31a
nosmp
log_buf_len=256k
printk.time=1

#

[l0wt3ch]

So does this thread belong in the Bugs section of the forum or should I move to to, for instance, the Cutting edge section?

I think you should move it to the "Users" or "Howto" sections. (At least, that's what I had in mind when I slightly edited it last night.)

[Flash]

l0wt3ch, could you write a proper how-to for the Howto section?

Also, I feel the subject line (title) of this thread could use some improvement, but I don't understand what the thread is about well enough to do it myself. Anyone have any suggestions for a title that would make sense to a reasonably intelligent beginner to Puppy? (l0wt3ch, feel free to change it yourself. )

[jpeps]

[quote="Karl Godt"][quote]

Thanks Karl...just noticed your subject line. Funny, I thought about scripting something that did that..figures it's already available.

[RetroTechGuy]

Maybe free memory is calculated differently in the Northern Hemisphere. But I just did a frugal install of Lupu 520 on a machine with 2 GB of RAM and got the same "nocopy" behaviour.

Is no one else seeing this?

On my machine with 2GB, it did copy. On my machine with 1 GB, it did not (and I don't recall turning it off to get that feature).

Curiously, I think that it also loaded on my 512MB machine...so I disabled that.

Perhaps it has something to do with the size or age of the HDD, or the IDE controller?... All of mine were IDE, not SATA...

I do like that pfix=nocopy works as advertised (since I generally prefer not loading to RAM, unless I've got a lot of free RAM -- Firefox tends to leak all over the memory, and cramps one's style if something else competes for that memory).

(Note: on 4.3.1 I had to hack through the initrd.gz to chop out all possibilities of loading -- just to stop it from doing so).