Fatdog64-700/701 [April 22 2015] [CLOSED]

Message

gcmartin · #861 Post by **gcmartin** » Mon 21 Sep 2015, 22:05

Thanks @Jasper. It's already been done over the years in PUP (and Linux) distros where the other community developers have continued to provide this PUP value.

If you, personally, have some code you would like to contribute beyond what the community has provided over the years, I'm sure some will find value. Or further, if you have some additional knowledge of SWAP use/absence, please contribute as we could benefit by what you know.

Be my guest to provide something you feel is of value for SWAP partition use. So share any additional knowledge you have on SWAP in FATDOG.

Maybe your intent was not to provide clarity on SWAP use, but, rather to disrupt (as has been the case for you in the past).

Yoni · #862 Post by **Yoni** » Mon 21 Sep 2015, 22:19

Can anyone link to any code that Jasper has contributed for anything?

Ted Dog · #863 Post by **Ted Dog** » Mon 21 Sep 2015, 22:30

Sounds interesting, on the idea of swap and RAM useable what ever happen to compressed swap in RAM? I could not see code changes so what if we just build the compressed savefile on the designation? I remembered a growisofs multisession on the fly piped with nc so why not. That should solve the run out of RAM issue ( I will work on the growisofs method ) worst that could happen is run out of space on whatever USB, HD or opitcal drive but that would happen anyway if its predone in RAM first.

gcmartin · #864 Post by **gcmartin** » Mon 21 Sep 2015, 23:09

In the Android world, they institute a rather interesting model for insuring there is space in RAM for priority work. They use an algorithm to kill running applications. So the onus is on ALL app developers to insure that when their app is killed, that the mechanism(s) are in place for resuming. This mechanism differs significantly from developer to developer. But, even Android has a planned movement to a Page/Swap approach in the wings as is their case for ChromeOS. Future processor/board speeds on todays RAM allow for this. But, the new multi-level RAM which can triple RAM capacity has an opportunity to forestall this move for quite some time allowing the kill model to remain. But, this is NOT Puppy Linux or FD7.

FD7 does not have apps that have this design. So, in an effort to insure stability, Linux/Apple/Microsoft/IBM/HP/Oracle/etc use system level controls in candidate selection and memory frame selection to insure things don't stop or fly out of control when priority work needs for invocation. This is not a new architecture as it has been in existence in every major OS I have been associated with over time. This is the paging or SWAP subsystem whose primary intent is to manage RAM resources as expeditiously as possible to maintain system integrity, application service, and most of all, stability.

It is with this understanding that many/most Unix/Linux/*ix users provide a SWAP even though at the outset of any system's use, there is little to no need of its use, while acknowledging the presence.

I am not trying to either persuade or dissuade. But, if there is some knowledge of system integrity being adversely affected when FD7 detects it, I, personally, would like to know. I have been a user of providing SWAP for the past 3 decades. If something has come along that has found this to not be a benefit, I would like to know. My perception and observation of its benefit, in all distros I use, is all I currently have to go on.

I have never known SWAP to have caused issues when any system has it provided to it. Yet, I may be having/contributing to a problem and not know I/we have such.

This is the reason I ask and wonder. And, the developers may know something which adds clarity to it in this Linux distro.

Yoni · #865 Post by **Yoni** » Mon 21 Sep 2015, 23:50

Unlike the other projects they have no interest in clarity as can be seen by the utter mess of fixes scattered all over the place that you just don't see from the other developers.

jamesbond · #866 Post by **jamesbond** » Tue 22 Sep 2015, 04:34

Ah so... Again, you miss the point.

No I don't - but you do. And I'm glad that my English isn't so terrible that only I can understand what I wrote - apparently Jasper and rcrsn51 can understand what I wrote, too.

The use of SWAP partition detection has been an automatic in every distro made in PUPPYland except the recent FATDOGs.

If you call beta versions of Fatdog 600 is "recent" (age: over 3.5 years), yes that's a correct statement.

Further, you twist the meaning of ...

I no longer take kindly or lightly for baseless accusations like this.

If you are going to make claim like this, you'd better prove that by quoting my post and providing a link to the original. I have over 2,500 posts, so it's not like you don't have much selections to choose from.

Otherwise, you're trolling. And trolls are not welcome in my book. Do I make myself clear?

None of what you have shared has providing anything convincing in why we all MUST remember that this benefit has been removed.

No, it does *NOT* work like that. I am *NOT* answerable to you. I am *NOT* beholden to you. I have *NO* obligations to you.

In short: I have *NO* need to convince you. On the contrary, it is *YOU* who needs to convince me. A good start would be by not putting words into my mouth.

And if you can't, as Jasper and many others have said - when why not roll up your sleeves and do it yourself? That's what an open source is good for; that's what a community is for. People take turns to help, people take turns to do things.

"Community" exists to help each other, it doesn't exist to serve you under the guise that you're representing "the voice of the community". I don't buy the royal "we" that you use. Nobody has appointed or elected you to be the spokesperson/trustee/leader/elder of this community; evidence points to the contrary. As far as I'm concerned, when you said "we", you're actually saying "I, me, and myself".

Oh, you're not technical enough? Oh, you're left programming decades ago? That's okay. You're a consultant; and you know what options to take in this situations. I don't have to spell it for you here.

Instead, you show "ire".

Of course I show "ire". I am human. Not Siri or Cortana. What do you expect? But read below why for the reason of my ire.

But, it is your distro. And if you want to get angry about a repeated request when I see benefit to the overall community that is your benefit as its lead developer.

I do not get angry about repeated requests. I simply ignore them. I posted my earlier entry as notice to others (not to you - since you already know the answer) of why I seem to ignore your requests.

But I do get upset when you put words into my mouth. That's what trolls do, and you're close to behaving like one. If you want to make a claim, then back it up with proof or evidence. Otherwise you're just like many of snake-oil consultants who over-promise and under-deliver.

But,I would like to hear the negative side of having SWAP as opposed to your upset with the request.

Your privilege to ask me questions have expired. I asked you a question first which you never bothered to answer. So why am I obliged to answer yours?

In the Android world blablablah ... full post here: http://murga-linux.com/puppy/viewtopic. ... 835#864835

This post shows that you are either don't get it; or you are trying to mislead others. The issue *YOU* raised has *NEVER* been about the usefulness of swap.

The issue that you keep on hammering is about *AUTO-DETECTING* presence of swap partition(s) on the user's harddisk, and then *USE* it without asking - even when running as LiveDVD. To which my answer is and has been NO, because I find the cons outweights the benefits - but convince me and I'll change my position.

jamesbond · #867 Post by **jamesbond** » Tue 22 Sep 2015, 04:38

Yoni wrote:Unlike the other projects they have no interest in clarity as can be seen by the utter mess of fixes scattered all over the place that you just don't see from the other developers.

Is this the best you can do, troll?

James C · #868 Post by **James C** » Tue 22 Sep 2015, 04:38

First quick point, Fatdog is not the first or only distro not to automatically detect and use swap.There are good reasons for that so it's a developer choice.

Google can be your friend.

I've got plenty of ram but I just added

Code: Select all

/dev/sdb5 swap swap sw,pri=1 0 0

to fstab just to demonstrate how easy it is to add....user choice.

Code: Select all

# free
             total       used       free     shared    buffers     cached
Mem:       8077400    1008056    7069344     253880      63852     454400
-/+ buffers/cache:     489804    7587596
Swap:      8396796          0    8396796

Damn that was hard......

gcmartin · #869 Post by **gcmartin** » Tue 22 Sep 2015, 05:04

OK, lets reset and try again. I dont know where I am putting words in your mouth and this is certainly without a doubt a troll. No words in your mouth. Everything I've offered is MY WORDS. As it stands now, this name calling is way out of hand and is certainly without merit!

So let reset and look at this as practical as one can.

SWAP partitions are something that a user must provide. It is done for a beneficial use in his/her/their systems. It doesnt get there on its own and it doesnt come with Windows or otherwise.

Users take care in providing this for their system's use. If a user has provided it, he/she has deemed its benefit.

I have shared that I have found benefit and have also shared it impact when it was discovered when not in use.

Again, I answered your question, earlier. If one provides it, as is the case with most distros, that they provided it for system discovery and use for the benefit it provides to a system's stability.

What is being asked is to consider that the user provides this for FD7 discovery and inclusion.

Further, it is removed but no reason on whether this helps or hinders system stability/performance is being shared.

Its your distro and I trust you see its value. I support your right to decide what you want to produce. If you choose to share it OK.

This exchange is between you and I. And its NOT a troll! At this point, I fear your anger will cloud your judgement and your best consideration.

jamesbond · #870 Post by **jamesbond** » Tue 22 Sep 2015, 05:08

Ted Dog wrote:Sounds interesting, on the idea of swap and RAM useable what ever happen to compressed swap in RAM?

It has been a while since I looked up at that subject. Things in Linux kernel is always in the state of flux so what I'm going to write may not be correct anymore; also my memory recall may be flawed, so take my words with a grain of salt. As Bruce Schenier likes to say: "trust, but verify".

There were two ways to use "compressed swap". One is by using "zram" block device. The other way is to use "zswap".
- "zram" basically creates a compressed RAM-disk. It works exactly like the plain old Linux ramdisk. It can be used for anything, not only for swap - but downside is that memory once occupied won't be available for other purposes.
"zswap" basically compress memory about to be swapped; and if the compression is successful, it won't be saved to disk. "zswap" is only for swap and can only work if swapping to disk is enabled.

These two are independent from each other and both can be enabled at the same time (Fatdog's default kernel is not compiled with ZSWAP support however - so you'll need to build custom kernel for this).

I could not see code changes so what if we just build the compressed savefile on the designation? I remembered a growisofs multisession on the fly piped with nc so why not.

You can "mkisofs" on the fly (without generating the ISO) and tell growisofs to burn it. Fatdog already does it this way. But, to be able to do this, the files (=the SFS) you're going to burn must already exist, because mkisofs needs to know the filesize etc. You can't generate the SFS on the fly, and pass it to mkisofs to generate the ISO on the fly, which then pass it to growisofs for burning.

That should solve the run out of RAM issue ( I will work on the growisofs method ) worst that could happen is run out of space on whatever USB, HD or opitcal drive but that would happen anyway if its predone in RAM first.

Yes, ideally. You can do this of you skip the mksquashfs stage and just burn the changed files directly - essentially this is what Puppy multisession does. But then there are downsides too (e.g. much slower loading during boot-up, possible filename/attribute problems due to forced conversion to ISO9960 format, large session size because it's uncompressed, etc).

jamesbond · #871 Post by **jamesbond** » Tue 22 Sep 2015, 06:03

@James C - thanks. That's absolutely what I mean.

gcmartin wrote:OK, lets reset and try again. I dont know where I am putting words in your mouth

You claimed that "I twisted the meaning of Paging Operating System". Show me my post that leads you to that conclusion. Otherwise I'll stand by my statement that you're making it up and putting words in my mouth.

SWAP partitions are something that a user must provide. It is done for a beneficial use in his/her/their systems. It doesnt get there on its own and it doesnt come with Windows or otherwise.

Users take care in providing this for their system's use. If a user has provided it, he/she has deemed its benefit.

I never disagree with this, so we should not even be talking about it.

I have shared that I have found benefit and have also shared it impact when it was discovered when not in use.

I have no doubt that you found the benefit immense, and the impact disastrous. But that because you use the system in a certain way. Others may not find the benefit (nor the impact) as great as yours, depending on how they use their system. If we're talking about personal experience, then both myself and kirk don't find swap (or lack of it) makes a great difference on how we use our systems.

Again, I answered your question, earlier.

The question I asked you which you didn't answer this:

gcmartin - I have a question for you. You are a LiveDVD user. One point that you keep raising over and over of LiveDVD benefit is that your harddisk is never "touched" or "modified" by the OS. Isn't automatic swap detection and usage on your harddisk a violation of your principle above? By having a LiveDVD OS detects ad use the swap already in your harddisk, your allowing your LiveDVD OS accessing your harddisk and depositing data in it. I thought you want to thoroughly avoid this?
(Please don't tell me you boot with LiveDVD and plug a USB flashdrive for use as a swap partition).

If one provides it, as is the case with most distros, that they provided it for system discovery and use for the benefit it provides to a system's stability.

Swap is a feature of the kernel. It is *NOT* a feature of a distro. It does *NOT* follow that because the kernel provides certain feature, the distro must also support system discover to use that feature (e.g. Linux kernel comes with bluetooth support and feature, I don't see you pushing for bluetooth support in every derivative of Puppy ...). And I disagree that swap *always* contributes to system stability; I'd concede that most time it will, but not always - you'll see why below.

What is being asked is to consider that the user provides this for FD7 discovery and inclusion.

I understood this - this is exactly what I said in my previous post to you. And the answer is NO, and has been NO since beta versions of Fatdog 600. NO automatic discover, NO automatic usage.

For the benefit of other users, here are my reasons:
a) discovery of swap makes boot time longer for *ALL* users whether or not they have swap partition (or intend to use it)
b) if run on a machine with failing harddisk (which happens to have swap partition), automatically mounting swap on the that bad harddisk makes system unstable and worthless for data recovery purposes
c) swap leaves traces of your system memory in the harddisk; which may include sensitive stuff that you'd prefer to be deleted as soon as the system is powered down.
d) if the swap is used by another Linux distro, and as it happens that that other Linux distro is being "hibernated" (=which keeps the state in the swap file), using that swap with Fatdog will destroy the state and makes the other distro unable to "resume" later on
e) activating swap makes the system slower even when no swap is being used (though the slowness may not be readily measurable).
f) swap is a stopgap solution. Swapping to disk, when it happens, is horrendously slow. If swapping happens very often, the system can slow down to a complete crawl. Depending on your usage scenario, it may be better to kill the programs that eats a lot of RAM rather than letting them continue to run, trashing the disk. Anecdotal example: - I tried to compile seamonkey in 512MB system (=on ARM). The compile phase (which went on with little to no swapping) went on for about 12 hours. The linking phase, which max-ed out the memory include swap, went on for over 72 hours with no sign of completion (before I finally killed it). The system spent over 95% of its time swapping rather than doing actual work; and were almost unresponsive to inputs (mouse keyboard).
g) If you encounters out of memory situation often, the better advice is to get more RAM into the machine, rather than activating swap.
h) Fatdog64 targets 64-bit system which has a lot less memory pressure (ie has a lot more RAM) than 32-bit system; making swap usually unnecessary unless you really run memory-heavy load (of which then you need to consider g) above). Compare this to Puppy which has to run in 128MB systems.

Those are what I can recall off my heads.

Further, it is removed but no reason on whether this helps or hinders system stability/performance is being shared.

No true. I shared a few of the points above in my earlier threads, and I remember putting more details in my PM to you; which you choose either to ignore and not disclose here. Unfortunately I have lost a copy of the PM that I sent you so it becomes my word against your word. I'll let the readers decide they choose to believe.

At this point, I fear your anger will cloud your judgement and your best consideration.

I'll let the readers be my judge. Trolls don't count, however.

Yoni · #872 Post by **Yoni** » Tue 22 Sep 2015, 08:25

jamesbond wrote:
Yoni wrote:Unlike the other projects they have no interest in clarity as can be seen by the utter mess of fixes scattered all over the place that you just don't see from the other developers.
Is this the best you can do, troll?

Well i don't see anyone else rushing in to disagree ?

jamesbond wrote:c) swap leaves traces of your system memory in the harddisk; which may include sensitive stuff that you'd prefer to be deleted as soon as the system is powered down.

Well that alone sounds like a good enough reason to me not to have a swap file so at least we agree on something.

Ted Dog · #873 Post by **Ted Dog** » Tue 22 Sep 2015, 17:58

if I replied to everyone I disagreed with JUST in linux I would have no time for anything else, and if everyone else did we would run of of space on the internet

I can see both points of view, neither one I disagree with but I have neither the skill or strong view in which is best. Trick is to make the views compatible, I am a bit blinded by using swap on harddrive if rest is running in RAM, only from a security issue. However I am first to throw out spot and run as root, So running in RAM for speed and swap as a safety valve

slavvo67 · #874 Post by **slavvo67** » Tue 22 Sep 2015, 23:47

Forgive my ignorance but can't you just delete the swap file when finished?

gcmartin · #875 Post by **gcmartin** » Wed 23 Sep 2015, 00:26

I have some corollary but, I dont think you will find it compelling enough to consider. I will say some/much of your reasons are valid, but some I do not agree and can show value.

It probably best to say, that I cannot/have not seen any instance of anyone going after page/swap in any effort to harvest valuable data...ever. Can it be done, I agree, that yes it could, but at what cost to person(s) who choose to attack a system to get meaningful info from its page or swap versus real and accurate data files; but, it some rare inexplicable (by me) cases, this could occur. This is in context to SWAP/Page do not lend itself to anything concatenated intelligently for anyone to really piece together as its framing are in constant "writeovers".

Again, I am happy that you have share your reasons and the problems you have personally had. I have not experienced such and I have never built anything on a Linux ARM system.

As far as I know, unless it has changed in recent PUPs, Linux, like UNIX, continues to use "lazy writes" in desktop distros; an I/O queuing mechanism which tries to concentrate on I/O when the CPU load drops. And, yes, if the workload reaches a level that exceeds the systems ability to get appl needs into the system before the scheduler react to run it back out, I can predict that outcome does elongate response times for use. In fact, in some systems the CPU is sitting waiting on the outcome of all the I/O that is generated coming in and going out of RAM to storage. This kind of thing has been a problem with ALL systems attempting to do workload management via the very slow storage media it uses.

But, these tend to be not be everyday and mainly Linux's SWAP activity will allow stability as it moves to some level of RAM balances.

As a cautionary, I can see where it is advisable for those of us who would use hibernation, to be aware of the potential downfall that might occur if we were not aware of that issue where a subsequent boot disrupts the prior hibernation..

The ideas presented by members about this is hoped to give users and developers some understanding of the benefits and draws that we have expressed.

On another note; What I stand by, on my comment that you reference is, in fact, my perception of what you pledged. It is not now or then a troll comment and it did NOT put words in anyone's mouth, excepting my own!

Hope this post is clear and I hope this draws an end to this bantering about SWAP.

Ted Dog · #876 Post by **Ted Dog** » Wed 23 Sep 2015, 01:17

I kinda learned a lot about SWAP during this exchange, the hybernate idea interested me since I do not use installed media ( outside of Windows or MacOS ) ie haven't needed to install a linux since puppylinux v0.97 ( frugal exception ) on my main computers.

Kai · #877 Post by **Kai** » Wed 23 Sep 2015, 04:51

Ted Dog wrote:I kinda learned a lot about SWAP during this exchange.

And i learned a new word " corollary "

Thanks Martin

jamesbond · #878 Post by **jamesbond** » Wed 23 Sep 2015, 05:22

slavvo67 wrote:Forgive my ignorance but can't you just delete the swap file when finished?

Yes you can, if you use a swap file (you have unmount it first - "swapoff" comes in handy).

But gcmartin wasn't talking about swap file. He was talking about permanent swap *partition*, not a swap file. Fatdog comes with a tool to create a swap file and use it - right from the Control Panel. But that's not what he's looking for.

You can of course zeros out the partition as well after unmounting it (dd if=/dev/zero of=/dev/your-swap-partition) but:
a) a big hassle
b) one typo and instead of your swap, your main partition is gone
c) if you automate this at every shutdown, then it slows down the shutdown process
Alterntively, you can use encrypted swap. But that comes with its own set of trade-offs, obviously. SFR did the experiment on that - you can ask him more details about it if you wish.

---

Conclusion:
========

For those who missed the drama: the whole discussion has *NEVER* been about the swap feature itself, nor was it about the usefulness of swap, the benefit of swap, the purpose of swap, the dangers of swap, etc (although obviously the interest it generates have encouraged some to learn more about swap - at least that's a good outcome).

Fatdog has always supported swap from the beginning (because it is has nothing to do with Fatdog, it is a Linux kernel feature). Beyond that, Fatdog provides a GUI to create a swap (file) and use it; and as James C and others pointed out, to use a pre-existing swap is just as easy as using either "swapon" or add the entry to /etc/fstab. So anyone who claims that Fatdog does not support swap OOTB (=out-of-the-box) is misguided at best.

What was contested before was gcmartin's repeated demands that Fatdog attempts to *search and find* any permanent swap partitions in the user's harddisk, and then *use* it without asking, because ... well, because:
a) it suits him
b) that's what some other puppies do - but mostly because it suits him

to which I have always been saying NO in the last 3.5 years; and this answer isn't going to change because he can't provide me a convincing reason that trumps above all my concerns given above.

Thanks for watching and I hope you enjoyed your popcorn.

jamesbond signing off.

James C · #879 Post by **James C** » Wed 23 Sep 2015, 05:41

Just a couple more things about swap.....

On this particular box, without adding the line to fstab, activating the existing swap partition is as simple as entering swapon /dev/sdb5 in the terminal.

For those wanting to learn a bit more, google (or use your favorite search engine) OOM Killer......a hint,it's built into the kernel.

Ted Dog · #880 Post by **Ted Dog** » Wed 23 Sep 2015, 15:50

ok yes missed that SWAP has always been part of Fatdog64. I agree that read only access is best position so lean toward JBs views on this matter, can it be made a kernel command line option if it turns out to add more value as a early pre switch_root setup ( allow a scan like gcmartin requests )

(old)Puppy Linux Discussion Forum

(old)Puppy Linux Discussion Forum

Fatdog64-700/701 [April 22 2015] [CLOSED]