Features/gjp miner

[schampoux]

PR created from feature/rqlite-integration branch containing the rqlite read-only node script. This PR adds the following:

• Updates environment variables to run start_read_node.sh.
• Adds functions for miners to interact with the global job pool (GJP) snapshot on their local machine. These functions allow miners to select max_workers number of jobs based on priority, or directly specify the pdb_id to select only one job. They can also choose specific columns that they would like to fetch. Lastly, miners can download the input and output files of selected jobs for analysis.
• Updates instructions in README.md for the above information.

[mccrindlebrian]

comments

[mccrindlebrian]

With this, I think it would be also necessary to indicate how to use this table / how to start the rqlite db. If we can do that in either docs or something similar, then I think we can merge to main to allow miners to sample it.

A particular use case is that miners want to be able to pull cpt files of other jobs in a separate script for analysis and comparison. Can you also illustrate this in a scripts/query_rqlite_db.py file? Something simple that shows how to do this effectively

[Sarkosos]

LGTM