Basic Shell (Console) operation for beginners

[Bruce B]

jpeps,

I tried it and it worked, sourcing did a path search. Something in the
past made me think it didn't and I never double or triple checked.
Just took it as a given.

This is why I wanted involvement, as a partial demonstration why
FOSS is such a superior development process.

One person knows a lot. Another person knows a lot. They don't know
things identically. The peer review teaches, people learn fast. Lots of
minds looking at the same thing.

Then Microsoft goes on writing closed source bloatware, and
operating systems full of holes like Swiss cheese.


Bruce

~

[technosaurus]

jpeps,

I tried it and it worked, sourcing did a path search. Something in the
past made me think it didn't and I never double or triple checked.
Just took it as a given.

Until recently there was a bug in busybox that ignored $PATH - I forget the specifics, but I think it is now solved

one thing that hasn't been discussed here much is substring manipulation, for simple things it can take the place of cut, sed, tr, grep, rev and others

One example of this is suffix retrieval and removal using

Code: Select all

EXT=${VAR##*.} #(only last extension)
BASE=${VAR%.*} #(only last extension)

EXT=${VAR#*.} #(all extensions)
BASE=${VAR%%.*} #(all extensions)

or replacing sed and tr for basic ops

Code: Select all

# Match at suffix (end) of string.
NEW=${OLD/%$ORIGINAL/$REPLACED}

for basic grep you would use a *$STRING* with the above

(these are internal functions and are thus much faster than calling the full cut/sed/grep/tr/rev/...)

[jpeps]

In working on bashbox I have developed several scripts into functions.
This allows you to add them to your .bashrc file and use them as a script. (I don't use the .bashrc in bashbox b/c I am aiming for ash/hush compatibility, but achieves the same effect)

for instance:

copy the contents of /root/my-applications/bin/colors to

colors() {
# insert contents of script here - without the top line (#!/bin/sh etc...)
}

and paste it into your .bashrc file

now use colors as if it were a script

it is a lot faster to call a function than to run a script (~20x) ... some scripts may require minor modification - particularly if they use $0 or rely on its relative directory location or similar

I have converted all puppy scripts to functions (over 1MB worth) and it still takes significantly less time to read them all as functions than to just load a single separate script

technosaurus,

Please correct me if I'm wrong on this. If the "colors" file (without #!/bin/sh) is in $PATH, you can just add ". colors" to .bashrc without having to include a function call. That seems faster, because you're not running a script or running a function...just reading in environmental variables. Again, correct me if I'm mistaken. Thanks

[PupGeek]

As I understand it, running a separate script would require opening up a new child shell, allocating memory for that script to run with, loading the script, and finally executing the code line by line, thus making it slower than running a function within the same script. I don't think running code directly would be much faster than running a function, but I do think that a function would be a lot easier to read than straight code, and my reason for defining the color variables in a separate script is to keep my .bashrc file nice and neat. Am I correct with why running a script takes longer than running a function bruce or techno? Also, would sourcing be faster by not opening a child shell?

[technosaurus]

Try it both ways using:
time script_with_function
And
time script_with_call

The more functions/calls the higher the advantage for functions... its only ~10x faster with only one call.

As far as "neatness" goes, check-out geany's fold all facilities... makes large scripts much more readable.

[PupGeek]

Try it both ways using:
time script_with_function
And
time script_with_call

The more functions/calls the higher the advantage for functions... its only ~10x faster with only one call.

As far as "neatness" goes, check-out geany's fold all facilities... makes large scripts much more readable.

oh i wasn't comparing the neatness of functions vs scripts, I was comparing neatness of functions against regular non-bracketed code. Also, "only ~10x faster" is still quite a bit. I believe, too, that the fold all facilities require code blocks, such as functions, for loops, and conditional blocks.

[Keef]

Just thought I'd poke my nose in....
I've been looking at ways of getting an IP address (I won't bore you with why...) Here are 3 of the ways I found, and the results of running with 'time':

Code: Select all

#!/bin/sh

ifconfig wlan0 | grep 'inet addr:'| cut -d: -f2 | awk '{ print $1}'

real 0m0.030s
user 0m0.008s
sys 0m0.020s
--------

Code: Select all

#!/bin/sh

ifconfig wlan0 | egrep -o '([0-9]{1,3}\.){3}[0-9]{1,3}' | sed -n '1p'

real 0m0.086s
user 0m0.052s
sys 0m0.028s

--------------

Code: Select all

#!/bin/sh
IP=`ifconfig wlan0`
IP=${IP#*inet addr:}
IP=${IP%% *}
echo $IP

real 0m0.011s
user 0m0.008s
sys 0m0.004s
-------------------

Is the last one an example of 'substring manipulation' ?
I can copy these things off the internet. Can't say I necessarily understand it!
What is the difference between real, user and sys? That is, which one is the most significant?

[jpeps]

Try it both ways using:
time script_with_function
And
time script_with_call

The more functions/calls the higher the advantage for functions... its only ~10x faster with only one call.

.

Compared:
A. color(){...} in .bashrc ## color script copied into function
B. . colors in .bashrc ## reference to file in $PATH

Procedure: "time ./testscript"

Results:

I found that it's a bit faster to just add ". colors" to the .bashrc file than running a function. More importantly, the colors were available to the terminal without first having to load the script. Also, the colors command wasn't recognized from within the testscript itself, but only worked by first loading "colors" from the terminal prior to running the testscript.

Note: enough of this nonsense....

[Bruce B]

Global Variables

Linux has Global Variables. Global defined by me; as existing for all
programs and operations. Most global variables exist because Barry
K put them there. Other variables, Linux puts in, e.g. PWD.

Barry obviously knows how to name and put values into variables
and export them. Very importantly, he knows when and where to do
it.

If you want to add global variables, do it like Barry does it and
where Barry does it.

To view the global variables run env

Script Variables

They exist until the script is finished and disappear.
And the Global variables also exist and can be used.

Caution

Don't use global variable names in your script as a script variable.

One person used PATH= to define something in the script. It
overrode the global PATH variable and the script bombed out.

You can use the global variables as is; with the values as is; no
problems.

~

[technosaurus]

Colors wasn't a real good example since it wasn't designed to be called like a script, variables and other declarations should stand alone outside of functions typically at the top.

It does help readability to have a standardized format though

#!/bin/sh or bash if you have bashisms
#copyright, license & description

#####includes#####
. /some/files

DEBUG="" #change to any value to enable debugging
[ $DEBUG ] && echo running with debugging on #for dev cycle - keep on its own line
#To remove debugging from a release version, you can either just reset the debug= or
#grep -v \$DEBUG Script_dev > Script_release #to cut size

#####user editable vars first ######
AUTOSAVE=TRUE

#####non editablevars next ######
[ $DISPLAY ] && DIALOG=Xdialog ¦¦ DIALOG=dialog
[ ! $CMDLINE ] && export CMDLINE='cat /proc/cmdline' #use backtick vs single quote

FunctionA() { ... }

FunctionZ() { ... }

#Rest of script here ... in C this would be main.

This layout order prevents odd issues like an include overriding a user defined variable, or a function not knowing a variable, or a function not being initialized before it is called.

Grr. Posting from a Droid is annoying... especially with code

Edit Barry uses /etc/profile for exporting global vars

[Bruce B]

Find Utilities

Introducing a multi-call script

BusyBox (used in Puppy) is a multi-call binary file, it works on
symlinks like this:

expr -> busybox

Find utils works like this

updatedb -> findutil
locate -> findutil

Why the multi-call script. One reason is for demonstration. Another
reason is both locate and updatedb need the same variables. Locate
and updatedb are only symlinks and cannot be modified. With a
multi-call script you have only one file to modify and work on. Any
changes you make in variables or directories or whatever,
automatically harmonize the symlinks.

This makes maintenance, updating and porting much easier.

Introduced in this post

» exit codes in scripts
» how to get basename of $0
» function programming - explained soon
» passing quoted command line variables to functions

Function based script programming

Findutils is a 100% function script. It has only one command outside
of functions, the command is main "$@". Bash reads the script from
top to bottom. It normally executes in the same order. Functions get
read in, but they don't execute unless told to do so.

Anytime locate or update is run, functions inside findutil get
executed, but not all functions, just the ones we want to execute.

Also, functions can be run in any order. We gain some control over
the flow of the script, by using functions.

More about the script

Earlier I showed how to change directories at lightening speed by
using aliases and the tool acd

Here I want you to show you how to find files at lightening speed.

This script can be modified, updated and improved and should be.
Presently, it is bare bones and will only search your Puppy frugal
system, which by the way has thousands of files.

It can be modified with fair ease to catalog your entire system and
more.

Detailed usage examples at bottom of post.

Code: Select all

#!/bin/bash

# Findutil Version 0.01
# For Puppy Frugal specifically

main() {

    case $1 in
        -h) showhelp ;;
        --help) showhelp ;;
        -?) showhelp ;;
        /?) showhelp ;;
        ?) showhelp ;;
    esac

    variables

    [ "$CALLPROGRAM" = "findutil" ] && showhelp

    $CALLPROGRAM "$@"

}

variables() {

    CALLPROGRAM=`basename $0`
    UPDATEDIR="/var/log"
    UPDATEFN="update.db"
    SEARCHDIRS="archive bin etc home lib lost+found opt root sbin usr var"

    # usually in Puppy /dev/shm is mounted, if so we can work in RAM
    WORKDIR="/tmp"
    </proc/mounts grep /dev/shm >/dev/null
    [ "$?" = "0" ] && WORKDIR="/dev/shm"

}

updatedb() {

    echo Updating $UPDATEFN

    # here we build the data base

    for i in $SEARCHDIRS ; do
        echo Searching /$i
        find /$i>>$WORKDIR/$UPDATEFN
    done

    mv $WORKDIR/$UPDATEFN $UPDATEDIR

    echo Done!

    exit 0

    # list of directories removed
    # automake dev initrd mnt proc selinux sys tmp

}

locate() {

    [ ! $1 ] && showhelp

    <$UPDATEDIR/$UPDATEFN grep -i "$@"

    exit 0

}


showhelp() {

printf "
Findutil 0.01 - help

  Findutil is a multi-call script, don't run it directly
  instead run [ updatedb | locate ]

  updatedb:
    updates your database, no arguments used

  locate:
    searches the data base for your files according to
    your search criteria and displays results on screen

  locate usage examples:

    locate /leafpad      : shows files beginning with /leafpad
    locate fpad$         : shows files ending with string fpad
    locate pad|grep xpm$ : shows files with string pad and ending with xpm

"

    exit 1

}


main "$@"

Instructions:

Unzip findutil.zip in your path statement

Run updatedb to make the data base

Here are some exercises:

Find the true type font directory

locate /ttf/

Find all true type fonts

locate '\.ttf$'

Find all .png files

locate '\.png$'

File all files in rc.d directory

locate /rc.d/

Find Firefox Cache location

locate /cache/ | grep -i firefox

or if seamonkey

locate /cache/ | grep -i seamonkey

look inside /root/.config

locate /root/.config/

Find all readme files

locate /readme

Then of course, after the output is displayed you can select a file to
open, delete, move or whatever. Also you can redirect output to a file.

~

[Bruce B]

It is fun providing tools like findutil.

What I mostly want is for people to learn. If after playing with
findutil and you have questions about the script, please ask.

~

[technosaurus]

It is fun providing tools like findutil.

the locate/updatedb helps with gtkdialog apps and images too(it uses locate if available). I'm not sure why they are not included by default. By running updatedb on /usr/share for instance, the image files in most *.desktop files are locatable by gtkdialog3 without the full path (making scripting _much_ easier)

you can combine cases using an "|"
case $1 in
-h|--help|-?|/?|?) showhelp ;;
esac

if directory exists use it, otherwise use /tmp
[ -d /dev/shm ] && WORKDIR="/dev/shm" || WORKDIR="/tmp"

some other tips
if you wanted to have a user input directory but fall back to something if they don't use one
DIR=${1:-/tmp} #use the first arg to the script otherwise use /tmp
#_can_ save having to do a lot of checks

you can use "shift" to remove the first arg from $@ (or shift 2 for first 2 args...) - this is useful for parsing args in a loop & for creating a multicall script

ex: calling "bashbox myfunction args" will run "myfunction args"
[ "`basename $0`" == "bashbox" ] && shift && $@

[Bruce B]

you can combine cases using an "|"
case $1 in
-h|--help|-?|/?|?) showhelp ;;
esac

Cool! I'm trying to make things very easy and readable. I hope if anyone
doesn't understand, they will ask. PS and I think wildcards like --h*elp.

if directory exists use it, otherwise use /tmp
[ -d /dev/shm ] && WORKDIR="/dev/shm" || WORKDIR="/tmp"

Unless something is wrong [ -d /dev/shm ] will always be true. The
question is: Is it mounted? Also, notice how fast updatedb catalogs the
directories?

I'll make a second post on your other tips, I have questions, thanks.

Bruce

[Bruce B]

Technosaurus,

My not so candid opinion is findutil is very powerful and valuable tool.

I put it at a low version number because I want it simple and
understandable, then gradually improve it.

(I can find files on a few terabytes in almost no time, then do operations
on them, that's how powerful)

I'll do the main introduction and all. But at a certain point, the user will
have to adopt it to his system. That's why the need for the user to have
command over it.

some other tips
if you wanted to have a user input directory but fall back to something if they don't use one
DIR=${1:-/tmp} #use the first arg to the script otherwise use /tmp
#_can_ save having to do a lot of checks

Remember I presented this as bare bones. I hope you will see as things
develop and interest increases, if it does.

you can use "shift" to remove the first arg from $@ (or shift 2 for first 2 args...) - this is useful for parsing args in a loop & for creating a multicall script

ex: calling "bashbox myfunction args" will run "myfunction args"
[ "`basename $0`" == "bashbox" ] && shift && $@

$0 doesn't change with shift

In your example; it is bashbox symlink args. Symlink is $0, at least it is in my script.

Maybe we aren't on the same page or something.

I wrote you a template demonstration script, please play with it. Then
repost it with comments. Because, I'm not understanding, the value of
shifting.

Code: Select all

#!/bin/bash

	# program flow, symlinks are named corresponding to function name

	# symlinks are also $0

	# we have three symlinks tool1, tool2, tool3

	# we pass CLI arguments to the functions


main () {

	
	# get basename of calling program

	callprog=`basename $0`
	$callprog "$@"
	
}

tool1() {

	echo Welcome from $callprog
	if [ ! "$1" ] ; then
		echo You entered no arguments
	else
		echo "Your arguments are: " "$@"
	fi

}

tool2() {

	echo Welcome from $callprog
	if [ ! "$1" ] ; then
		echo You entered no arguments
	else
		echo "Your arguments are: " "$@"
	fi


}

tool3() {

	echo Welcome from $callprog
	if [ ! "$1" ] ; then
		echo You entered no arguments
	else
		echo "Your arguments are: " "$@"
	fi



}

testtool() {

	echo You ran testtool directly
	if [ ! "$1" ] ; then
		echo You entered no arguments
	else
		echo "Your arguments are: " "$@"
	fi


}

main "$@"

File contents:

script testtool
link tool1 -> testtool
link tool2 -> testtool
link tool3 -> testtool


To use:

testtool
tool1
tool2
tool3

testtool args
tool1 args
tool2 args
tool3 args

Delete when no longer useful.

Bruce

~

[jpeps]

Technosaurus,

My not so candid opinion is findutil is very powerful and valuable tool.

~

I once wrote something similar, only it cataloged online repository info including deps. Then I could quickly check for updated app versions, etc.

[technosaurus]

@Bruce - I was showing how to implement the other busybox method. the shift; $@ would just go in a function of the same name as your script

[Bruce B]

@Bruce - I was showing how to implement the other busybox method. the shift; $@ would just go in a function of the same name as your script

Thanks, I now understand.

And by the manner BusyBox actually works, it must work along the lines
you illustrate in processing argument sequences.

~

[Bruce B]

Conflicts with help - how to get the right help

In this example, I want help for a common scripting command read.

• wrong: read --help
  right: help read

Bash tells you read --help was wrong or goofy by printing this:

-bash: read: --: invalid option

Then giving you the output of read --help

But bash doesn't tell you how to get the right help. (as a bash
programmer, you are supposed to know, I suppose, but we are just
learning, give us a break)

Try it and see the difference.

I hope the reasons are obvious.

~

[jpeps]

I hope the reasons are obvious.

~

Works for internal (builtin) commands only. Also, try help -s, -m, -d options (eg, "help -d read"). "Help" by itself prints the builtin commands.

edit: Looks like -d, -m options are only available with bash 4.1