Shinobi - - a ZoneMinder replacement using Node.js and mysql
Shinobi - - a ZoneMinder replacement using Node.js and mysql
A new project called Shinobi is under way to build an open-source solution for a network recording device. A possible replacement for ZoneMinder, using Node.js and mysql, addresses some of the short comings of ZoneMinder.
Using ffmpeg-release-32bit-static.tar.xz which runs nicely on tahr 6.0.5. Node.js 7.3 the mysql installed from the PPM and NPM manager for node.js and pm2 to daemonize Shinobi.
If anyone is interested I could throw out some tips to get it going.
Using ffmpeg-release-32bit-static.tar.xz which runs nicely on tahr 6.0.5. Node.js 7.3 the mysql installed from the PPM and NPM manager for node.js and pm2 to daemonize Shinobi.
If anyone is interested I could throw out some tips to get it going.
Last edited by rockedge on Thu 05 Jan 2017, 00:21, edited 1 time in total.
One report:
"4 cameras each doing 2048x1536@25fps recording h264 on copy
old machine with 4 cores, roughly sitting at 30% cpu
using about 1GB ram"
I am running 3 cams at 320x240 with one local webcam on a dual core 4 gigs of RAM...my results show 5% cpu 40 megs of memory in recording mode of all 4 cameras.
Set up with Tahr 6.0.5 updated and running Node 7.3 MySQL 5.5.53-0ubuntu0.14.04.1 (Ubuntu) with NPM and pm2 installed via PPM.
Node.js is fairly simple to set up. I downloaded node-v7.3.0-linux-x86.tar.xz and extracted to /root/my-applications then symlinked node and npm to /root/my-applications/bin
Using PPM find npm and install.
download the Shinobi tarball or zip from github extracted it to /root/my-applications and followed the install instructions to configure the mysql database then used pm2 to daemonize the camera.js
"4 cameras each doing 2048x1536@25fps recording h264 on copy
old machine with 4 cores, roughly sitting at 30% cpu
using about 1GB ram"
I am running 3 cams at 320x240 with one local webcam on a dual core 4 gigs of RAM...my results show 5% cpu 40 megs of memory in recording mode of all 4 cameras.
Set up with Tahr 6.0.5 updated and running Node 7.3 MySQL 5.5.53-0ubuntu0.14.04.1 (Ubuntu) with NPM and pm2 installed via PPM.
Node.js is fairly simple to set up. I downloaded node-v7.3.0-linux-x86.tar.xz and extracted to /root/my-applications then symlinked node and npm to /root/my-applications/bin
Using PPM find npm and install.
download the Shinobi tarball or zip from github extracted it to /root/my-applications and followed the install instructions to configure the mysql database then used pm2 to daemonize the camera.js
Shinobi is currently being equipped with software face and license plate recognition. As a efficient Network video recorder it is already a strong competitor to ZoneMinder and many commercial CCTV systems
the Node.js version 7.4 and MySQL installed via the PPM work well and also make a nice Node.js server on Tahrpup.
https://github.com/moeiscool/Shinobi
the Node.js version 7.4 and MySQL installed via the PPM work well and also make a nice Node.js server on Tahrpup.
https://github.com/moeiscool/Shinobi
- Max Headroom
- Posts: 421
- Joined: Wed 28 Jun 2006, 07:17
- Location: GodZone Kiwi
- Contact:
Yes Please Rockedge, I'm Very interested in this Shinobi CCTV Security System. I'm considering Running it on a VIA Epia EN Mini - itx small Form Factor PC which is / was previously a VadaXchange Phone System. if by any chance yoU've come across such hardware, I Welcome any Comments about Fitness 4 Purpose or eXperience. Any Updates? Thanx ;-)K
PC is Not Political Correctness :P , it's a Personal Computer! Boycott the Evil M$ & Google!
There has been a request to clarify a working recipe for installing Shinobi.
The recipe has been tested on Bionic64-v8 stock version with devx loaded.
once the system is up and running I believe one can unload the devx.
Get the devx_bionicpup64_8.0.sfs and load it using Menu->Setup->SFS-Load-on-the-fly
The devx sfs contains tools needed for compiling and has git included plus more of the PERL language modules.
Open a terminal ->
finish the prompts.
https://nodejs.org/en/download/
download nodejs LTS (in my case node-v10.16.0-linux-x64)
open it with xarchive (click on it)
extract to /root/my-applications
open /root/my-applications/node-v10.16.0-linux-x64/bin
sym-link the these files, open a termnal ->
open a terminal ->
you should see a response like :
Open the PPM
search for "mariadb" and select -> mariadb-server_10.1.40
Let PPM install it
Open a terminal and type ->
open a 2nd terminal ->
Return to the 1st terminal an type ->
reply Yes for all the prompts and fill in a password for the root user
close the terminals
mysql is running and the nodejs server is almost ready now.
open a terminal ->
note the version which on this Bionic64 is
you may need this command as well...doesn't hurt so go ahead and run it
Following from this page of the Shinobi docs -> https://shinobi.video/docs/start
Application Install
Download Shinobi with git (comes in the devx_bionicpup64_8.0.sfs)
Open Terminal in /root/my-applications
git clone https://gitlab.com/Shinobi-Systems/Shinobi.git Shinobi
Set permissions on the shinobi directory. Where camera.js is located.
chmod -R 755 /root/my-applications/Shinobi
Open Shinobi directory.
cd /root/my-applications/Shinobi
Setup SQL. Go to sql and install the SQL files in your database.
cd sql
Access MariaDB SQL Database from Terminal. The password will have been set during the installation of MySQL.
mysql -u root -p
OPTIONAL : Create New SQL User with privileges. If you choose to use your own pre-defined credentials skip this step.
source ./user.sql
or create your own
CREATE USER 'majesticflame'@'127.0.0.1' IDENTIFIED BY '';
GRANT ALL PRIVILEGES ON * . * TO 'majesticflame'@'127.0.0.1';
FLUSH PRIVILEGES;
while still in the SQL client. Install the Shinobi database. It will create a database called ccio.
source ./framework.sql
OPTIONAL : default_data.sql contains a demo user and a demo jpeg input monitor.
source ./default_data.sql
After importing the data. Exit the sql client.
exit
Go up one directory to enter the main directory. Where camera.js is located.
cd ..
Copy conf.sample.json as another file named conf.json.
cp conf.sample.json conf.json
Edit conf.json to reflect your sql credentials. Puppy Linux users can use geany or leafpad to modify any of the config files.
geany conf.json
Install Libraries
Run npm install while in the main directory. This will install the libraries Shinobi needs. PM2 is needed to use UPDATE.sh and to Daeomonize the process
npm install&&npm install pm2 -g
or simplify with
npm install
npm install pm2 -g
Copy super.sample.json as another file named super.json.
cp super.sample.json super.json
Launch Shinobi
then to start :
pm2 start camera.js
pm2 start cron.js
Run pm2 logs to see the console for any errors.
forever is another program to daemonize, but i've had more success with pm2.
To get your IP you can run the following command.
ifconfig
Open up http://YOUR_IP:8080 in your browser. The Default Login was originally set by adding default_data.sql.
Username : ccio@m03.ca
Password : password
REMEMBER:
all the passwords used are default and you can modify them as you wish.
once you have seen how it works one can place the components in any directories desired.
The recipe has been tested on Bionic64-v8 stock version with devx loaded.
once the system is up and running I believe one can unload the devx.
Get the devx_bionicpup64_8.0.sfs and load it using Menu->Setup->SFS-Load-on-the-fly
The devx sfs contains tools needed for compiling and has git included plus more of the PERL language modules.
Open a terminal ->
Code: Select all
adduser mysql
https://nodejs.org/en/download/
download nodejs LTS (in my case node-v10.16.0-linux-x64)
open it with xarchive (click on it)
extract to /root/my-applications
open /root/my-applications/node-v10.16.0-linux-x64/bin
sym-link the these files, open a termnal ->
Code: Select all
ln -s /root/my-applications/node-v10.16.0-linux-x64/bin/node /root/my-applications/bin/node
ln -s /root/my-applications/node-v10.16.0-linux-x64/bin/npm /root/my-applications/bin/npm
ln -s /root/my-applications/node-v10.16.0-linux-x64/bin/npx /root/my-applications/bin/npx
Code: Select all
node -v
Code: Select all
v10.16.0
search for "mariadb" and select -> mariadb-server_10.1.40
Let PPM install it
Open a terminal and type ->
Code: Select all
mysql_install_db
Code: Select all
mkdir /var/log/mysql
chown -R mysql /var/log/mysql
mysqld_safe
Code: Select all
/usr/bin/mysql_secure_installation
close the terminals
mysql is running and the nodejs server is almost ready now.
open a terminal ->
Code: Select all
ffmpeg
Code: Select all
ffmpeg version 3.4.4-0ubuntu0.18.04.1
Code: Select all
npm install ffbinaries
Application Install
Download Shinobi with git (comes in the devx_bionicpup64_8.0.sfs)
Open Terminal in /root/my-applications
git clone https://gitlab.com/Shinobi-Systems/Shinobi.git Shinobi
Set permissions on the shinobi directory. Where camera.js is located.
chmod -R 755 /root/my-applications/Shinobi
Open Shinobi directory.
cd /root/my-applications/Shinobi
Setup SQL. Go to sql and install the SQL files in your database.
cd sql
Access MariaDB SQL Database from Terminal. The password will have been set during the installation of MySQL.
mysql -u root -p
OPTIONAL : Create New SQL User with privileges. If you choose to use your own pre-defined credentials skip this step.
source ./user.sql
or create your own
CREATE USER 'majesticflame'@'127.0.0.1' IDENTIFIED BY '';
GRANT ALL PRIVILEGES ON * . * TO 'majesticflame'@'127.0.0.1';
FLUSH PRIVILEGES;
while still in the SQL client. Install the Shinobi database. It will create a database called ccio.
source ./framework.sql
OPTIONAL : default_data.sql contains a demo user and a demo jpeg input monitor.
source ./default_data.sql
After importing the data. Exit the sql client.
exit
Go up one directory to enter the main directory. Where camera.js is located.
cd ..
Copy conf.sample.json as another file named conf.json.
cp conf.sample.json conf.json
Edit conf.json to reflect your sql credentials. Puppy Linux users can use geany or leafpad to modify any of the config files.
geany conf.json
Install Libraries
Run npm install while in the main directory. This will install the libraries Shinobi needs. PM2 is needed to use UPDATE.sh and to Daeomonize the process
npm install&&npm install pm2 -g
or simplify with
npm install
npm install pm2 -g
Copy super.sample.json as another file named super.json.
cp super.sample.json super.json
Launch Shinobi
then to start :
pm2 start camera.js
pm2 start cron.js
Run pm2 logs to see the console for any errors.
forever is another program to daemonize, but i've had more success with pm2.
To get your IP you can run the following command.
ifconfig
Open up http://YOUR_IP:8080 in your browser. The Default Login was originally set by adding default_data.sql.
Username : ccio@m03.ca
Password : password
REMEMBER:
all the passwords used are default and you can modify them as you wish.
once you have seen how it works one can place the components in any directories desired.
Last edited by rockedge on Sat 15 Jun 2019, 14:59, edited 6 times in total.
examples of Shinobi running
1 & 2 are results of using pm2
1 & 2 are results of using pm2
- Attachments
-
- Screenshot(1).png
- (25.96 KiB) Downloaded 446 times
-
- screenshot-1.png
- (98.54 KiB) Downloaded 472 times
-
- screenshot-2.png
- (102.18 KiB) Downloaded 402 times
-
- Screenshot.png
- (249.32 KiB) Downloaded 449 times
Thnx rockedge
Thank you very much rockedge for making the above instructions. It was really frustrating to get it to work because of a lot of obstacles for a new Puppy user like me. Like for example,
1.) I didnt know I can just download and mount devx_bionicpup64_8.0.sfs to be able to use
Instead of downloading the large 133MB git-all package from PPM when i only needed git clone.
2.) I was searching for 'MySQL server' in PPM and couldnt find it. Thats because its 'MariaDB' I should be searching per your instructions.
3.) Installing MySQL client was also a pain since theres a lot of mysql packages in PPM but installing it was as easy as inputting,
which I assume is because of devx_bionicpup64_8.0.sfs
4.) I didnt need to download ffmpeg from shinobi doc page since devx_bionicpup64_8.0.sfs also seems to carry it
5.) Following the Shinobi doc instructions for Ubuntu is also causing me problems because apparently Puppy doesnt have a fully functioning
I just want to add a couple of additions to your recipe. Since you quoted the Shinobi doc instructions for Ubuntu, the command
doesnt exist for Puppy so instead I downloaded 'nano-tiny' from PPM and I was able to run
I dont know why Shinobi dev included this instruction since 'conf.json' can be edited by a text editor anyway so installing 'nano-tiny' is optional.
The ffmpeg from devx_bionicpup64_8.0.sfs doesnt have ffbinaries according to pm2 log and it suggested the command which fixed the problem.
The Shinobi doc instructions for Ubuntu also forgot to mention to execute command and not just
I ran into a lot of "Cannot read property 'forEach' of undefined...." errors in pm2 logs without cron.js running.
Must also execute command and after starting camera.js and cron.js.
Theres also an instruction, To test MariaDB daemon, which doesnt work because the MariaDB from PPM doesnt have 'perl mysql-test-run.pl' file nor is there '/usr/mysql-test' directory. I still dont have a proper way to test the MariaDB daemon but if i can execute and it shows camera.js and cron.js online and i can login to [xxx.xxx.xxx.xxx:8080] then I assume MariaDB is running.
I now have Shinobi running and currently testing it. Thnx again.
1.) I didnt know I can just download and mount devx_bionicpup64_8.0.sfs to be able to use
Code: Select all
git clone
2.) I was searching for 'MySQL server' in PPM and couldnt find it. Thats because its 'MariaDB' I should be searching per your instructions.
3.) Installing MySQL client was also a pain since theres a lot of mysql packages in PPM but installing it was as easy as inputting,
Code: Select all
mysql_install_db
4.) I didnt need to download ffmpeg from shinobi doc page since devx_bionicpup64_8.0.sfs also seems to carry it
5.) Following the Shinobi doc instructions for Ubuntu is also causing me problems because apparently Puppy doesnt have a fully functioning
Code: Select all
apt-get
Code: Select all
nano
Code: Select all
nano-tiny conf.json
The ffmpeg from devx_bionicpup64_8.0.sfs doesnt have ffbinaries according to pm2 log and it suggested the command
Code: Select all
npm install ffbinaries
The Shinobi doc instructions for Ubuntu also forgot to mention to execute command
Code: Select all
pm2 start cron.js
Code: Select all
pm2 start camera.js
Must also execute command
Code: Select all
pm2 startup
Code: Select all
pm2 save
Theres also an instruction, To test MariaDB daemon,
Code: Select all
mysql-test-run.pl cd '/usr/mysql-test' ; perl mysql-test-run.pl
Code: Select all
pm2 list
I now have Shinobi running and currently testing it. Thnx again.
Thank you very much for the info...I can edit and improve the instructions to include the obstacles you faced. As well as the small details.
the mysql_install_db is from the mariadb package and is the easiest way to set up the mysql.
Yes the nano or nano-tiny are great to have around but not necessary for Puppy Linux users in this set up since Geany or leafpad will do just fine
If you are connecting via the browser you have Shinobi running.
use Menu->System->htop
to see what is running
the mysql_install_db is from the mariadb package and is the easiest way to set up the mysql.
Yes the nano or nano-tiny are great to have around but not necessary for Puppy Linux users in this set up since Geany or leafpad will do just fine
If you are connecting via the browser you have Shinobi running.
use Menu->System->htop
to see what is running