(old)Puppy Linux Discussion Forum

In the same line of thought, all Puppies also have sqlite3 and awk.

I'll give this join function a try to see how fast it is:

https://linux.die.net/man/1/join

I want to add md5 sums to the puppy repo data bases.

https://gitlab.com/sc0ttj/Pkg/issues/75#note_246129899

If the join function is fast, then maybe putting this info in a separate table will work well

Reminded me of this old thread
http://www.murga-linux.com/puppy/viewtopic.php?t=86146
Where I demonstrated 1 way to use your filesystem as a database using only a shell script... just a proof of concept

s243a wrote:I'll give this join function a try to see how fast it is:

https://linux.die.net/man/1/join

I want to add md5 sums to the puppy repo data bases.

https://gitlab.com/sc0ttj/Pkg/issues/75#note_246129899

If the join function is fast, then maybe putting this info in a separate table will work well

I was thinking about this and in theory a join command should be fast if the input data is sorted.

technosaurus wrote:Reminded me of this old thread
http://www.murga-linux.com/puppy/viewtopic.php?t=86146
Where I demonstrated 1 way to use your filesystem as a database using only a shell script... just a proof of concept

I thought about doing this but to know where near the same degree as you have implemented. I'll have to check this out later

As a further thought, I bet you can replace the input files to join with process substitution: http://tldp.org/LDP/abs/html/process-sub.html

and if you do this then the input files don't necessarily need to be text files. The input files to the join utility could actually be the output from different databases queries or alternatively we could have a custom binary file that has similar properties to a database such as fast searching and filtering capabilities.

NoSQL RDBMS

NoSQL is a shell-level relational database management system
for UNIX. Unlike most DBMS'es, NoSQL is not a single large program,
rather it is a set of small programs that are run from the shell.
This allows the user to utilize the full power of the UNIX shell
in coordination with NoSQL.

It uses the Operator/Stream DBMS paradigm described in "Unix
Review", March, 1991, page 24, entitled "A 4GL Language".
There are a number of "operators" that each perform a unique
function on the data. The "stream" is suplied by the UNIX
Input/Output redirection mechanism. Therefore each operator
processes some data and then passes it along to the next
operator via the UNIX pipe function. This tends to be rather
efficient as UNIX pipes are implemented in memory. NoSQL is
compliant with the "Relational Model". Read the document
"4gl.ps", included with the distribution, for further details.

Is this it?
http://www.strozzi.it/cgi-bin/CSA/tw7/I ... ome%20Page

musher0 wrote:Is this it?
http://www.strozzi.it/cgi-bin/CSA/tw7/I ... ome%20Page

Thankyou for this gcav and musher0. I'm going to have to give it a try

NoSql tutorial: http://www.troubleshooters.com/lpm/200704/200704.htm
---

Also see this post about csvsql from the csvkit package: https://stackoverflow.com/a/44177446/5479837

And there is Miller: https://github.com/johnkerl/miller

Ans also something really cool called VisiData: https://visidata.org/

(I went to a workshop with the guy who made VisiData, he showed it off, it's great!)

Helpful link of theory of joins, unions etc, in shell: http://matt.might.net/articles/sql-in-the-shell/

Thanks for sharing the good find. Really appreciate it.

https://rbql.org/

...provides SQL-like language for data-transformation and data-analysis queries for structured data (like CSV or TSV files). RBQL query is executed using one of the available general-purpose "backend" languages (currently Python or JavaScript). RBQL allows to use non-SQL expressions inside SQL statements, e.g for JavaScript backend...

Thank you sc0ttman, VisiData seems like a real gem!

VisiData's author wrote:I live in Seattle but the terminal is my home.

And that says a lot!

Hello all.

I've been reading the VisiData docs. What's a "panda" ?
Not the black and white vegetarian bear, I suppose!
First time I've seen this word used in a data context.

TIA

musher0 wrote:What's a "panda" ?

https://www.geeksforgeeks.org/python-pandas-dataframe/

Thanks, 6502coder.
I.e. a regular grid or matrix, but with 2 axes only and
possibly containing heterogeneous data.