BIT METER OS 0.7.6 (stable) & 0.8.0 (experimental)

[Mike Walsh]

Hi, gcm.

Hmm. Thinking about it, I guess you're right; this could very well lead to other developments. Admittedly, that wasn't uppermost in mind when I started the thread (!), but as with so many other things over the years, one thing will of course lead to another...

Of course, it all depends on how much interest is generated, and how many other folk see this potential.....and are willing to put in development time. There's the rub.

Certainly, a central data recording point (at the router) would make a whole heap of sense, where multiple machines on a single network are concerned...

Keep your fingers crossed; no telling what might happen!


Regards,

Mike.

[robd]

@robd:-

In fact, I DO have one question, now that I think about it. I moved to Linux about a year ago, as I said in my message to you. Nearly from the start, I looked for something similar to, and then discovered, BitMeterOS; I think the stable version then was 0,7.5, and 0.7.6 was the experimental version. When 0.8.0 was released, I moved to 0.7.6.

With 0.8.0, the web interface now shows the difference between total usage (up & down), and internet usage. It didn't take me long to find out that file-sharing via LAN was a lot more prevalent in the Linux world than it was in the Windows one. I've since come to realise that in fact, networking and 'file-sharing' is becoming quite standard these days, with the advent of media-streaming, and home-based media server setups.

Was this in your mind when you developed the 0.8.0 version?


Regards,

Mike.

Hi Mike,

Being able to differentiate between LAN and Internet traffic was by far the most requested feature from people using BitMeter OS and BitMeter 2 (my previous Windows-only meter) but unfortunately wasn't easy to implement - it meant using the pcap library to gather network stats rather than the much simpler method used in 0.7.6. The big advantage of using pcap however is that it lets you add filters for pretty much any type of internet traffic, the 4 default filters are the ones most people stick with but using the 'bmdb addfilter' command you can add others using tcpdump syntax

hope that helps

Rob D

[mikeb]

if the pcap fits...use it

sorry

[robd]

If you go into /var/lib, then click on the 'bitmeter' directory, there's 3 items, yes? I know what the 'runweb.sh' does, but there's two database entries; 'bitmeter.db', and 'bitmeter.db.new'. What's the difference between these; does BitMeter use the value from the latter, add it to the former, and then overwrite.....or something? In other words, is one temporary storage, until the total is calculated and written to the 'permanent' database?

Mike.

The bitmeter.db.new file isn't needed by BitMeter, it's just an empty database file which is included in the installer (if you still have one hanging around you can delete it).

If someone is installing BitMeter for the first time on their system then this file gets renamed to bitmeter.db and gets used by the application, however if someone is upgrading from an older version of BitMeter then obviously the installer shouldn't replace the existing bitmeter.db file containing all their data, so in this case the bitmeter.db.new file gets removed and the existing .db file is upgraded keeping all the data intact.

If you speak shell-script then you can see this going on here

[robd]

@Geoffrey, davids45:-
Now, then; seems we have an answer as to whether symlinking to a common database will work...

I think this should work, provided the location of the bitmeter.db file is correct from the point of view of each of the different operating systems - if you need to have the file somewhere other than the default location you can set an environment variable to point to it as described here

Obviously the file will also need to be both readable and writeable by each O/S as well.

[gcmartin]

I have added 1 additional paragraph which I think all will find favor in awareness on this earlier post.

[Mike Walsh]

Hi, Rob.

Thanks very much for clarifying some of that for us. Hopefully it will help out some of our 'experts' (I don't include myself in their number, as I'm very much a 'noob' when it comes to this kind of thing!)

Those of us who understand what you're referring to will no doubt be able to do something with it. I mean, I basically understand what's going on, but I don't know the first thing about scripting...so can't therefore, put it into practice myself.

I don't know whether you followed my description of trying to set up a common, symlinked database this morning; some of the development work here has been due to the need to make the debian packages work with Puppy's rather unique, union-layered filesystem. From what I can make out, this in turn is partly due to the sheer number of different ways there are of not only running, but booting Puppy, too.

I'm slowly getting the hang of it.....and this is where a dedicated, and very knowledgeable & friendly community comes in so handy; you can almost guarantee that whatever your problem is, somebody, somewhere, will have come across the same situation before.

You say the common database would need to be not only readable, but writeable, too. This is my big stumbling-block at present; permissions..! I'm guessing this is why BitMeter quit on me, the way it did...

One other query, from my point of view, if I may? This business of changing the environment variable to point to the *changed* location of the bitmeter.db file; I hate to get so basic over stuff like this, but where exactly do you make the change? Do I do this in the terminal...and if so, what commands would need to be used?

I know there's at least one other person who would like to know the answer to this...


Regards,

Mike.

[davids45]

G'day Mike Walsh, robd, gcmartin, etc,
I did try the simple symlink for the .db file and found like you it did not run .

However, another symlink strategy I use, particularly with big packages like LibreOffice and wine, is to set up an expanded pet of each such pet with Muppy-Filer on my data partition.
This gives me a subdirectory of a pet with the paths for a normal install of the pet retained as sub-subdirectories.

I now make symlinks for every sub-directory of the pet on the data partition to the same sub-directory of the running (Full) Pup. A bit manual in setting up but I'm used to it now. Sometimes, I only need to link a few bin files to their 'normal' places and the package runs as though it is installed in the Pup rather than sitting on another partition. Seamonkey is good this way, I just link the seamonkey executable to /usr/bin/ from its subdirectory on the data partition and it runs. I also link the root/.mozilla/seamonkey/ profile back to my common profile so all my mail and bookmarks are available on all Pups.

With Frugal Pups, it is much easier as I just make a ginormous sfs of my wanted packages, although the symlinks still work for things like my Seamonkey profile.

My data partition is mounted at boot for all my Pups via a /root/Startup script - which becomes the first thing I must to do after any new Pup install.

So, back to the matter in hand, as there is a pet of Bitmeter, I'll try this approach next. Expand the pet to its own sub-directory on my data partition then make links of these now expanded files to their intended locations in a Pup.

And see if I can get it to work . And if it does, then repeat this in another Pup. And if I see the totals are carried forward with both Pups' sets-up, try another.

All good fun.

David S.

[Geoffrey]

Now, then; seems we have an answer as to whether symlinking to a common database will work...(

Symlinking works, I moved bitmeter.db to the partition outside of the save file and linked it back, works fine as I thought it would, so if you run a different pup version then all you need do is remove the /var/lib/bitmeter/bitmeter.db and create a symlink to the bitmeter.db you previously moved, bitmeter.db could be moved anywhere as long as the location is mounted at boot.

G'day Mike Walsh, robd, gcmartin, etc,
I did try the simple symlink for the .db file and found like you it did not run .

In the terminal run

Code: Select all

/etc/init.d/bitmeter restart

[davids45]

G'day,
Geoffrey's pet worked when first installed on this HP desktop in Pups such as Wheezy, Slacko, Precise.

Setting up the symlink to centralise the network data onto my data partition (sda5) however causes problems.

"Page Load Error Failed to connect"
I've attached before and after screen shots in one of my Wheezy Pups.
I had the 'Page Load Error' in all Pups when doing the link with the Pup not running (used another Pup and mounted the target Pup partition to edit the location of the .db file).

"NaN PB" output to summary tab
If doing this symlink while Bitmeter is running, I saw the Bitmeter browser numbers (lots of MB and kB) display go to lots of 'NaN PB' values as per the attached screenshot.

The encouraging thing for me is when watching the Wheezy desktop/pinboard after the symlink and getting the page load error browser box, on the two open (file-manager) Rox windows, I could still see Bitmeter was scanning, and the linked file on sda5 was increasing in size when I refreshed that window.

So I think it's almost happening for Bitmeter on this multiPup computer, just the browser/db-file-symlink is not behaving, or I'm doing something wrongly?

I tried Geoffrey's /etc/init.d/ script (and when I typed it in correctly ), it reported the daemon was running. But still nothing via the browser except the 'page load error'.

Hoping these screenshots are readable and helpful.

David S.

[Geoffrey]

David S

I tried it again with the bitmeter.db on a partition other than the boot partition, this didn't work even though I had set it to mount at boot, must be a timing issue, works fine for me if it's on the boot partition, would have to run the daemon's after the mounting to see if that works, I don't know much about that and will have to see what I can do.

If anyone wants to jump in and help with this please do.

Geoff

[Mike Walsh]

@Geoffrey:-

Oh, yes; symlinking certainly does work, if you do it with the Pup currently running. I simply created a small, 5 GB partition in my HDD's logical partition (plenty of space there, and outside of my three primary Puppy partitions (see attachment)).That's where I moved the bitmeter.db file from Tahrpup (the first one that I tried), and then symlinked it back.

Like you, I thought about the need to have that partition mounted at boot. Since I'm running rcrsn51's Samba-TNG, along with the auto-start script Bill's provided, I simply added an entry for auto-mounting the extra partition at boot; the script includes this functionality.

Works like a charm.

The problem occurs when you then try deleting the bitmeter.db file in the other two Pups, and symlink the Tahrpup .db file back to them. Not only did BitMeter in the other two Pup's cease working, but Tahrpup's did too...

You could be right about a timing issue. Would it be worthwhile including a delay somewhere along the line (perhaps in the xbitmeter routine), to see if that would work? I'm only throwing out ideas here; as you know, I don't really understand the mechanics of all this yet.

Perhaps the bitmeter re-start command needs incorporating at some point.....or have you already done that? I get a bit lost trying to follow the scripts; Rob himself might have some ideas on this one.


Regards,

Mike.

[Geoffrey]

Mike Walsh,

I got it working, need a sleep statement after it detects the network connection, this gives it time to mount the partition, then restart both daemon's, a sleep time of 15 seems to work ok for me.

Give that a try.

Edit /root/Startup/xbitmeter

Code: Select all

#!/bin/sh

IFCONFIG="`ifconfig | grep '^[pwe]' | grep -v 'wmaster'`"
while [ "$IFCONFIG" != "" ]; do
   sleep 1
    ping -c 1 8.8.8.8
    if [ $? -eq 0 ];then
      break
   else
      ping -c 1 www.google.com
      if [ $? -eq 0 ];then
         break
      fi
   fi
done
sleep 15
/etc/init.d/bitmeter restart
/etc/init.d/bitmeterweb restart

[Mike Walsh]

@Geoffrey:-

Nice one! Coupla questions, if I may? I take it this needs to be edited in each OS that you want to symlink to the common database, yes? With regards to doing the editing; should BitMeter be stopped for the duration while you're editing Xbitmeter....and if so, what's the easiest way to stop it, then restart? I'm guessing this is a terminal command.

Sorry for the questions, but hey! I'm a noob...


Mike.

[Mike Walsh]

Now; I'm not jumping-in feet-first this time until I have a wee bit more information... What's the best way to edit the xbitmeter file; from within the mounted OS, or from outside, using one of my others?

Also, what bitmeter file should I use in the common partition? Do I use one of the existing ones, and then symlink this back to /var/lib/bitmeter (followed by symlinking the others in turn?

Or should a brand-new .db file be created.....and if this is the case, how do you do this? Mikeb did make some mention of this being an 'SQLite database'(?) I take it these need a special type of file to be created.....

I'll get this thing working one way or another, see if I don't!

One more thing; when I create the symlinks, do I use 'Link(Absolute)', or 'Link(Relative)'? Never been too sure what the difference between these two is...


Mike.

[Geoffrey]

What's the best way to edit the xbitmeter file; from within the mounted OS, or from outside, using one of my others?

Edit the /root/Startup/xbitmeter script on each of your pups that your wanting to use the linked bitmeter.db,
all your adding to the bottom of the script is the sleep 15 and /etc/init.d/bitmeterweb restart.

what bitmeter file should I use in the common partition? Do I use one of the existing ones, and then symlink this back to /var/lib/bitmeter (followed by symlinking the others in turn?

If you want a clean database then copy the bitmeter.db.new and rename it bitmeter.db else use the existing one with you old data, symlink it back to /var/lib/bitmeter,
do likewise with each of the Pup's your running, the symlink will pickup the bitmeter.db once a mount is established.

One more thing; when I create the symlinks, do I use 'Link(Absolute)', or 'Link(Relative)'? Never been too sure what the difference between these two is...

Make a Absolute link, these have the full path, Relative links are used in the same directory such as in /usr/lib, like libxxx.so.1 is a link to libxxx.so.1.2.3

[Mike Walsh]

@Geoffrey:-

All right; sounds easy enough. One last point I forgot about. When you symlink the common bitmeter .db file to /var/lib/bitmeter, what do you do with the existing .db file? Do you simply delete it.....or do you leave it there?


Mike.

[Geoffrey]

@Geoffrey:-

All right; sounds easy enough. One last point I forgot about. When you symlink the common bitmeter .db file to /var/lib/bitmeter, what do you do with the existing .db file? Do you simply delete it.....or do you leave it there?


Mike.

Delete it first, your using rox filer, I'm not sure if that offers to overwrite the existing file with a symlink as I use thunar file manager, though it should, if it don't replace the file with a link then delete and try again.

[Mike Walsh]

@Geoffrey:-

Oh, YES!! We have SUCCESS!!!

Geoff, old son, you're a 24K, gold-plated marvel. Give yourself a BIG pat on the back, from me. In fact, I'll go one better; I'll stand you a few rounds at your favourite watering-hole.....though whether or not you'll ever get to collect, is another matter entirely.....

That business of the sleep delay, and YOUR idea of re-using the bitmeter.db.new file (and re-naming it to use as the 'common' symlink) seems to have done the trick. Nice one..! I now have Tahrpup, Slacko 5.7.0, and ETP's 'Chromepup' (sic), all utilising a common database, and it means, for the first time since I've used this app, that I can at long last add together totals from ALL my distros....and get a true monthly usage reading.

Incidentally, ROX will ask if you want to overwrite on moving & copying.....but not when doing symlinks.

Like I said, there is the 'bmsync' utility.....but that's designed for collecting together data from different, physical machines that are running simultaneously; and of course, you can't do that with multiple Puppies on the same machine. Now, thanks to your good self, we can. Brilliant work, mate!

------------------------------------------------------------------------------------------------------------------------------------------------------------

So; for anybody wishing to utilise Geoffrey's method for linking multiple OSs on the same machine to one, common, BitMeterOS database, thereby being able to total bandwidth usage (both internet & LAN) from multiple Pups.....here's a re-cap:-


1) First, install either of Geoffrey's bitmeteros .pets (each post contains the download link);

bitmeteros-0.8.0-i386.pet for 32-bit Pups:- http://www.murga-linux.com/puppy/viewto ... 5&start=39

or

bitmeteros-0.8.0-amd64.pet for 64-bit Pups:- http://www.murga-linux.com/puppy/viewto ... 5&start=54


2) The next step is to create a common data partition, or folder (preferably at /mnt/home), to hold your common bitmeter.db database. As long as it's outside of the Puppyspace 'save-file', that's what we're after.....though, using a separate, small partition makes it easier.

2a) If you opt for the separate partition, it will need to be mounted at every boot. Since I use rcrsn51's Samba-TNG for file-sharing, I use his samba-autostart.pet, too; the script includes functionality for auto-mounting multiple partitions, or even multiple drives.

The folder method will work just as well. This is one of those instances where Puppy's sometimes-ridiculed running as root stands us in good stead.


3) In each of your OSs, in turn, perform the following steps:-

a) Locate /var/lib/bitmeter, and open it to see the following; 'bitmeter.db', 'bitmeter.db.new', and 'runweb.sh'. Using Rox (or whatever your file manager happens to be), open your partition (or folder), drag 'bitmeter.db.new' across to this window, and 'Copy'. Leave both windows open.

b) Rename 'bitmeter.db.new' to 'bitmeter.db'.

c) In the /var/lib/bitmeter window, delete 'bitmeter.db'.

d) From your partition (or folder) window, drag 'bitmeter.db' back to the /var/lib/bitmeter window, and 'Link(Absolute)'. This has now created your symlink back to the common database. You can now close your windows.


4) The next step is to edit 'xbitmeter' in '/root/Startup'. This is the routine which auto-starts BitMeterOS's daemons and web interface. Navigate to /root/Startup/xbitmeter, open (with an editor such as Geany), and change the following; from this:-

Code: Select all

#!/bin/sh

IFCONFIG="`ifconfig | grep '^[pwe]' | grep -v 'wmaster'`"
while [ "$IFCONFIG" != "" ]; do
   sleep 1
    ping -c 1 8.8.8.8
    if [ $? -eq 0 ];then
      break
   else
      ping -c 1 www.google.com
      if [ $? -eq 0 ];then
         break
      fi
   fi
done

/etc/init.d/bitmeter restart

to this:-

Code: Select all

#!/bin/sh

IFCONFIG="`ifconfig | grep '^[pwe]' | grep -v 'wmaster'`"
while [ "$IFCONFIG" != "" ]; do
   sleep 1
    ping -c 1 8.8.8.8
    if [ $? -eq 0 ];then
      break
   else
      ping -c 1 www.google.com
      if [ $? -eq 0 ];then
         break
      fi
   fi
done
sleep 15
/etc/init.d/bitmeter restart
/etc/init.d/bitmeterweb restart

adding 'sleep 15' to line 16, and '/etc/init.d/bitmeterweb restart' at line 18. This gives time for BitMeterOS to start the daemons, detect the common database, and re-initialise. Save, and close.


5) Reboot.


6) When you're back up & running, open your browser of choice. In the address bar, enter the following:-

Code: Select all

http://localhost:2605/index.html

In a few seconds, the BitMeter web interface will open. You can choose from a real-time monitor, history, a real-time summary; you can set alerts, to warn you when you might be approaching a data limit. In 'Preferences', you can change the filter colours, alter refresh intervals, etc. If you change any of these, then make sure you 'Save to Server'; this makes the change permanent. 'Save to Browser' will only save changes for the current sesson.

7) Ideally, you should bookmark at this point, so as to make it easy to re-deploy whenever you wish. Personally, BitMeterOS is the first tab I open.....and it stays open all the time I'm in the browser.

------------------------------------------------

Repeat steps 1) thru 5) for however many Pups you wish to link to the common database.

That's it!!

-----------------------------------------------------------------------------------------------------------------------------------------------------------

Thanks must go to the following individuals:-

OscarTalks, for producing the initial .pet of the then 'experimental' BitMeter OS 0.8.0.....

mikeb, for much sensible advice, many useful suggestions, and lots of encouragement.....

davids45, for his query about using symlinks (which got us all thinking!).....

gcmartin, for being the first to volunteer to test the 64-bit version.....


Special thanks must go to Geoffrey, for his tremendous efforts in helping to bring this all to fruition.....and for teaching yours truly a few new tricks in the process..!


And, of course, many, many thanks must go to Rob Dawson, the author of BitMeterOS.....without whom none of this would have been possible.

Apologies to anybody I've left out.....it's not intentional!



Regards,

Mike.

[davids45]

G'day,

Adding my thanks to Geoffrey for his fixings.

One thought I have had is, could the Bitmeter install process or the Preferences dialog include a line or three to ask about where to store the .db file?
For examples, Seamonkey/Firefox has its Preferences where I can install my profile on another partition, and where I can store my Mail (in Seamonkey), and where to hold the Cache.
LibreOffice has Options to default-store documents and templates on another partition.
Gimp lets me use an off-partition Temp directory for its temp files.

David S.

[off-topic, for Mike Walsh - who put a shot of his drive partitioning when first trying symlinking and doesn't seem put off.
If you ever think of going 'the whole hog' with links, here's my sda partitioning based on a maximal use of symlinks and a data partition.
I give only 3GB for each Full Pup (they rarely use more than half of this - the 'Used' column), I keep all the Frugals on one, bigger partition with the sfs packages shareable, and leave the rest of the drive for data storage and where possible for running programs too by symlinks from here. I have a small Linux swap file (1GB) which I suspect never gets used with 2GB of RAM.
My second hard drive is set up similarly so I have 2 large data partitions. Back-up is easy - I just use 'Puppy Backup' to update the two data partitions to a 1TB USB drive. Puppy is so simple and quick to install or re-install , a Pup itself doesn't need back-up in this scheme.]