csvq: Search and filter queries on datasets (CSV)

• An all-inclusive UI-based tool to perform various search and filter queries on csv datasets.
• Can also run SQL queries and natural language queries.
• Can select columns from the active tables and impose conditions to filter them using a simple button-based GUI.

HOW TO RUN:

Locally(for Development):

• Clone the repository, followed by running the backend and the frontend in separate terminal tabs:

git clone https://github.com/tathagata-raha/csvq

cd server/

pip install -r requirements.txt

python app.py

cd now-ui-dashboard-react-main/

npm install

npm start

Using Docker:

• Clone the repository, followed by starting the docker containers using a single command:

docker-compose up

• Head over to http://localhost:3000/admin/sql_query

---

Features

Select CSV Dataset and Table to perform queries on

• List available datasets on the Data Foundation server.
• Select datasets to run queries on.
• Select one or multiple CSV tables frmo the chosen datasets.

Upload new datasets

• Create new dataset domains
• Upload new CSV datasets

SQL Queries

The user can enter SQL queries on the selected CSV tables and execute them in browser to yield downloadable outputs. The following features are implemented:

• SQL query validator
• Dataset Preview & Datatype preview for all columns
• Output Preview
• Output Direct Download
• Sharable link for download

Filters

In addition the user can select columns from the active tables and impose conditions to filter them using a simple button-based GUI.

Natural Language Queries

We run an NLP models to infer SQL queries from natural language text input from the user and execute it similar to the SQL pipeline. Note that this is a Beta feature and needs to be trained on more data for more accurate results.

Query Suggestions

We can extract information from the active table in one-click with query suggestions based on your query history. The most frequent historical queries are returned as top suggestions.

Hierarchical Queries

We provide the feature to perform multiple types of queries on the outputs of each query forming a hierarchical query structure. The outputs of the hierarchical queries are saved temporarily for each user session and wiped off at the end of the session.

API Docs

The API documentation for our application is neatly documented using Swagger UI and can be accessed at: http://localhost:5000/docs

🚀 Technology Stack:

• Frontend:
• Backend:
• Version Control:
• Deployment:

Made with ❤️️ as part of Data Foundation Systems course, Spring 2022, IIIT Hyderabad.