(old)Puppy Linux Discussion Forum

Posted: **Tue 15 Sep 2009, 17:57**

12Nov09,
(I have uploaded SFS_Linker-431bk-1.1.pet for use in Puppy431 andSFS_Linker-no_def-1.1.pet which does not have modified /usr/local/defaultprograms for people who wish to have SFS_Linking without changing the desktop icon or MIME-type behavior.

These are the latest versions of SFS_Linker and seem to be preforming well.)

I have been looking for alternatives to load_sfs as used in ChoicePup412. Although it works well, it only works with the aufs1 file layering system.

The new kernels that Barry is introducing into Puppy4.3 use aufs2 which fails to mount the sfs's even if they're sfs4.

Looking at an idea put forward by sunburnt and reintroduced by trio for copying sfs into full installs I thought it might be possible to link the files in the mounted sfs to / rather than copying them. This would take no space in a frugal installs pup_save file.

I copied some of trio's code and put together this script:

Code: Select all

#!/bin/sh
#SFS linker

SFSNAME=`basename $1`

	mkdir /mnt/"$SFSNAME"
	mount -t squashfs "$1" /mnt/"$SFSNAME" -o loop
	cp -rs /mnt/"$SFSNAME"/* / #jrb-add -rs, remove -a symlink option 15sep09
	
	cd /mnt
	APATTERN="s/.\/$SFSNAME//g"
  	find ./$SFSNAME -type f -mount -mindepth 2 | sed -e "$APATTERN" | sed -e 's/\/root0\//\/root\//g' > /root/.packages/$SFSNAME.files
  	sync
  	find ./$SFSNAME -type l -mount -mindepth 2 | sed -e "$APATTERN" | sed -e 's/\/root0\//\/root\//g' >> /root/.packages/$SFSNAME.files
  	sync
	echo -e "\"${SFSNAME}\" \"${SFSNAME} : \" on \" \" \\"   >> /root/.packages/alienpackages.txt
	
	mkdir -p /root/my_links/linked_sfs
	ln -s $1 /root/my_links/linked_sfs/

	ldconfig
	fixmenus
	Xdialog --title "sfs_linker" --msgbox "Success! `basename $1` loaded.\n"  6 0
	jwm -restart

I have not built an uninstall script yet. You can remove the symlinks in petget manager although it leaves behind empty folders. You have to go to /mnt and unmount the sfs files and then delete their folders.

It works with most of the ch-sfs packages. Noticeable exceptions are the mozilla browsers and openoffice. I have a feeling that the $HOME variable my need to be set. ch-GoogleEarth-4.3.sfs opens but doesn't work.

Any advice or suggestions will be greatly appreciated (I still consider myself a linux noob so keep it simple).

This is for use in Puppy412 or 421. The petget manager is different in Puppy4.3, will have to rewrite for that.

If you feel brave start a new pup_save and try it out. Right click on an SFS, choose OpenWith-sfs_linker.

Thanks, J

Posted: **Tue 15 Sep 2009, 22:34**

Jrb,

I tried it with ch-mtpaint and got these errors ..

Code: Select all

# sfslinker ch-mtpaint-3.30.sfs 
cp: cannot create symbolic link `/root/Choices/MIME-types/image_bmp' to `/mnt/ch                                                              -mtpaint-3.30.sfs/root/Choices/MIME-types/image_bmp': File exists
cp: cannot create symbolic link `/root/Choices/MIME-types/image_gif' to `/mnt/ch                                                              -mtpaint-3.30.sfs/root/Choices/MIME-types/image_gif': File exists
cp: cannot create symbolic link `/root/Choices/MIME-types/image_jpeg' to `/mnt/c                                                              h-mtpaint-3.30.sfs/root/Choices/MIME-types/image_jpeg': File exists
cp: cannot create symbolic link `/root/Choices/MIME-types/image_png' to `/mnt/ch                                                              -mtpaint-3.30.sfs/root/Choices/MIME-types/image_png': File exists
cp: cannot create symbolic link `/root/Choices/MIME-types/image_tiff' to `/mnt/c                                                              h-mtpaint-3.30.sfs/root/Choices/MIME-types/image_tiff': File exists
cp: cannot create symbolic link `/root/Choices/MIME-types/image_x-pixmap' to `/m                                                              nt/ch-mtpaint-3.30.sfs/root/Choices/MIME-types/image_x-pixmap': File exists
cp: cannot create symbolic link `/root/Choices/MIME-types/image_x-png' to `/mnt/                                                              ch-mtpaint-3.30.sfs/root/Choices/MIME-types/image_x-png': File exists
cp: cannot create symbolic link `/root/Choices/MIME-types/image_x-xpixmap' to `/                                                              mnt/ch-mtpaint-3.30.sfs/root/Choices/MIME-types/image_x-xpixmap': File exists
cp: cannot create symbolic link `/usr/local/lib/X11/mini-icons/mini-camera.xpm'                                                               to `/mnt/ch-mtpaint-3.30.sfs/usr/local/lib/X11/mini-icons/mini-camera.xpm': File                                                               exists
cp: cannot create symbolic link `/usr/local/lib/X11/mini-icons/mini-palette.xpm'                                                               to `/mnt/ch-mtpaint-3.30.sfs/usr/local/lib/X11/mini-icons/mini-palette.xpm': Fi                                                              le exists
cp: cannot create symbolic link `/usr/local/lib/X11/pixmaps/paint48.png' to `/mn                                                              t/ch-mtpaint-3.30.sfs/usr/local/lib/X11/pixmaps/paint48.png': File exists
find: warning: you have specified the -mount option after a non-option argument                                                               -type, but options are not positional (-mount affects tests specified before it                                                               as well as those specified after it).  Please specify options before other argum                                                              ents.

find: warning: you have specified the -mindepth option after a non-option argume                                                              nt -type, but options are not positional (-mindepth affects tests specified befo                                                              re it as well as those specified after it).  Please specify options before other                                                               arguments.

find: warning: you have specified the -mount option after a non-option argument                                                               -type, but options are not positional (-mount affects tests specified before it                                                               as well as those specified after it).  Please specify options before other argum                                                              ents.

find: warning: you have specified the -mindepth option after a non-option argume                                                              nt -type, but options are not positional (-mindepth affects tests specified befo                                                              re it as well as those specified after it).  Please specify options before other                                                               arguments.

Despite the errors, everything seems to be working as usual.

Posted: **Wed 16 Sep 2009, 00:11**

Thanks seaside,
It's trying to create symlinks in folders where a file with that name already exists. I hadn't realized all those mimetypes were in the mtpaint package.

Just glad to see that the originals weren't removed when I uninstalled with petget manager. They may well have been if they had been placed there with another .pet. I think this method may require a bit more care and testing when putting together the sfs's.

I haven't tried all the sfs's but so far opera, flash, geany, dillo, pnethood, cups, wine, boost, pwidgets, mplayer, gxine, and mtpaint are all working well, very promising.

Posted: **Wed 16 Sep 2009, 03:05**

jrb; What I have concluded is that temp. files and links should go in the /tmp dir.
As in usb-auto, I make the dir.: /tmp/usb-auto to hold the links.
The next time Puppy boots, the links and dir. should all be gone.
Self cleaning... At least Puppy use to work this way, things keep changing.

Also rather than relying entirely on links, use PATH & LD_LIBRARY_PATH.
I made a GUI to manage the paths into the Squash file`s mount points.

I suggested apps. that will run mounted but not unioned be used that way.
Gimp only needed it`s config. file copied to /etc and it ran just fine!
With help from some Gurus like MU etc. more could be made to work.
Best to do the really important apps. first; Web, Media, Games, etc.

SFS files are nice as they don`t corrupt or take up space in the Save file.
A new extension is needed for them, like: ".sfa" (Squash File Attached).
With a mime file type, a simple click could attach & unattach the SFA file.
A new SFA file manager would be needed once there were lots of SFA files.

The SFA files don`t have to be made like SFS files are, they could be simpler.
A script inside the SFA file could setup the SFA file to work properly.
The script would also clean up when the SFA file was unmounted.
The script allows for handling the special setup needed by some apps.

SFA files are the first main step needed to get rid of the union file sys.
No union makes for a much simpler and more reliable OS.
The core OS can run in ram and the SFA files can be anywhere!
On a DVD, a HD, a USB, on the LAN, or on the Web! Web connect apps.!
Also the core OS can boot from anywhere and be distributed as well.

Posted: **Fri 18 Sep 2009, 04:21**

Hey jrb; Are you still working on this?
I have been thinking about it, and I have an idea for a SFA builder script.
It would install an app. package to a dir., then you can test it from there.
The apps. install dir. would act just like the SFA file`s mount point.
As links are made for the app., a file listing them is made in the SFA file.
Load and Unload the SFA file, the list is read and links made and deleted.
Once the app. is working from the dir., another script makes an SFA file.

Need shorter lengths and fewer additions to PATH and LD_LIBRARY_PATH.
So the SFA file would have just 1 dir. each: /bin, /sbin, /etc, /lib, and more.
Example: Only a /bin, not: /bin, /usr/bin, /usr/local/bin, so fewer paths.
And the files being only 1 dir. deep keeps the paths as short as possible.
With the paths set leading into the SFA file, only files that are referenced
with a full path need links, so fewer links need to be made and managed.

There`s other tricks that can be done also, but this adds good capability.

Posted: **Fri 18 Sep 2009, 08:10**

jrb; I was grabbing some more apps. for ChoicePup and thinking about this somemore...

How about making a small ram disk to hold all the links and use it as a union layer.
Then all the links are not cluttering up the Save file and don`t need to be cleaned up.
The links don`t take up any space, so a very small ram disk should do, ~ 16-32 MB ?
This still makes use of the FUSE union FS, but this is how ChoicePup still works.

I hope the use of links can be greatly reduced by the use of the 2 paths.
Imagine only 3 union layers! The Save file, the main SFS, and the links ram disk.
Less memory needed, and less cpu overhead to handle all those layers!

Posted: **Fri 18 Sep 2009, 09:45**

Imagine only 3 union layers! The Save file, the main SFS, and the links ram disk.
Less memory needed, and less cpu overhead to handle all those layers!

That sounds great, but then I struggle to imagine anything

Has anybody quantified the effect of using lots of union layers? Even a couple of dozen?
Have people even reported Puppy feeling slower if they add a couple of dozen .sfs addons?

Posted: **Fri 18 Sep 2009, 16:30**

disciple; Surely you don`t deny a union`s complexity and resources required?
At first it seems a great idea, but experience and understanding changes that.
Realize it`s unnecessary and cumbersome, there`s better methods to use.
### A better method:
App. Squash files are only mounted to the core OS running in memory.
OS dirs. that are written to often (/etc) are links pointing to dirs. in a Save file.
To change the core OS, make a new core SFS file, it`s small and quickly done.
Apps. in Squash files are the new package format, no installing, just use them.

Compared to Puppy or a Full Linux install... What a really great improvement!
Speed, simplicity, low cpu use, low memory use, modular packages and OS.
No installing-uninstalling apps. scattering files all through the OS, so messy!
Unix was great for a main frame admin., but Linux users change apps. about
as often as a 16 year old girl changes her clothes, managing the files is a pain.
That`s why ChoicePup is such a great Puppy build, it uses SFS files fully.

Posted: **Fri 18 Sep 2009, 22:53**

I'm not denying anything, I'm just wondering if anyone knows how much benefit this approach would have...

Posted: **Fri 18 Sep 2009, 23:30**

Gobolinux (and I think tiny core) use a similar method. I am testing a completely different method using ermine pro. It can pack an entire (uncompressed) filesystem into a single executable. Put the one executable in a sfs of the same name and modify the sfs mounter to try to execute a program of the same name as the sfs. Two clicks and you are there. The biggest downfall which is the same as sfs is dependencies. The great part about ermine pro is that it will automatically include all dependencies in the package unless you specify otherwise. Not too great for size if someone doesn't know how to package with it,but better for the user to wait an extra couple seconds on a download than have to track down dependencies.

Posted: **Sat 19 Sep 2009, 04:20**

technosaurus; I looked at GoboLinux years ago, and it`s rather complicated.
Rearranging the Linux dir. structure won`t make it better, it`s more complex.
The core OS here is Puppy`s 4 files, but no union FS complications.
The real change is to how the SFS files are used, and their structure.
Dependencies can always be in the SFS files of course, at any level.

disciple; The benefits are obvious, it`s simplicity is reason enough...
And the other obvious benefits are all the more reason to implement it.
We`ve had cars for 100 years now, but most see it as a bad thing...
To wonder if we should change to a better transit system is short sighted.

Posted: **Sat 19 Sep 2009, 08:59**

Haven't tried this on 4.3 but this is how rexterd does it on slitaz and he says it works on fatdog too:
http://murga-linux.com/puppy/viewtopic. ... 836#330836

Posted: **Sat 19 Sep 2009, 09:12**

There;s similar command for merging branches in unionfs too, but for some reason doesn't apear to work, maybe needs some modification of init script or something else compiled in kernel:
http://www.linuxjournal.com/article/7714

So mounting on the fly is possible in both aufs and unionfs but for some reason don't work on puppy

Posted: **Sat 19 Sep 2009, 09:31**

We're not talking about unioning on the fly, but about not unioning (well, reducing the number of unions, anyway).

Posted: **Sat 19 Sep 2009, 09:38**

I don't think more layers slows down anything but boot time
And I'm mentioning on the fly mounting cause it's related to topic

Posted: **Sat 19 Sep 2009, 18:16**

jrb is mounting SFS files but not unioning them. So the topic is non-unions.
The OS may or may not be running a union, but it`s not being used for this.

Posted: **Tue 22 Sep 2009, 16:37**

JRB,

I don't know whether you are still pursuing the "link" route over the "union" one, but I thought maybe one way to prevent any necessary installed files from being inadvertently removed by the Pet uninstaller is to test for "symlink files only" before removal.

"if not symlink file, don't remove" in the uninstaller code.

I realize there are many factors and tradeoffs between "linking" and "unioning" and perhaps "linking" might be a "safer" way.

How is your thinking going now?.

(I must make sure Choicepup stays healthy)

Posted: **Tue 22 Sep 2009, 20:43**

Hi seaside; My suggestion to have a small ram disk as another union layer solves this problem.
All the links go into it and so doesn`t make a mess of the file system.

BUT... Setting the paths is most important, it eliminates most of the links!

jrb; You have been silent, are you still with us?
I`ll provide details of how to use the paths if you`re still at it... Terry

Posted: **Tue 22 Sep 2009, 21:29**

@ sunburnt: I have been experimenting with using sfs files within a roxapp. This way the roxapp contains the script which mounts the sfs file and can copy any necessary files over. I have succeeded with one app thus far... Blender. I made it to contain all its dependencies as well as check for OpenGL and install it if necessary. I have uploaded it into the .pet store and I am waiting for it to be available.

Posted: **Tue 22 Sep 2009, 21:47**

Hi PupGeek; I assume that you`re using this method of "mount but don`t union"?
If so... Why would you mess up the file system with copied files?

As to dependencies, I had wrote a script that runs an app. and if it errors,
it runs "ldd" on it to get library dependencies and then downloads them.
It could be expanded to include all types of dependencies for download.
This way any app. package or SFS file doesn`t need the dependencies.
All that`s required is a solid fixed web site to point to the download sites.
This way it acts as a clearing house for the dependency download sites,
so the download sites can come and go and the main site is just updated.

(old)Puppy Linux Discussion Forum

Mounting and Linking SFS files for use

Mounting and Linking SFS files for use