Skip to content

Commit

Permalink
Dev to staging (#931)
Browse files Browse the repository at this point in the history
* removed QA integration

* created neo4j from existing index

* modified script

* Integrate local search to chat details (#746)

* added the commuties tab

* removed unused variables

* removed scipy libarary

* added the mode check

* Integrated the communities tab

* added the cjheck

* enabled the top entities mode

* tabs order rearange

* added the loader to sources tab for entity search+vector

* fixed the chat mode per prop

---------

Co-authored-by: kartikpersistent <[email protected]>

* removal of unused code

* removed entity label

* Added Description to chat mode menu (#743)

* added tooltips to chat mode menu

* addition of description to menu

* 741-tooltips-to-selectOptions

* menu changes

* acommunities name change

* close changes

* name changes

* format fixes

* Update log_struct method to add severity

* community check

* Entity Empty Label fix and Icon

* Update Utils.ts

* Retry processing - node and rels count update condition for start from beginning (#737)

* Remove TotalPages when save file on local (#684)

* file_name reference and verify_ssl issue fixed (#683)

* User flow changes for recreating supported vector index (#682)

* removed the if check

* Add one more check for create vector index when chunks are exist without embeddings

* removed local files

* condition changes

* chunks exists check

* chunk exists without embeddings check

* vector Index issue fixed

* vector index with different dimension

* Update graphDB_dataAccess.py

---------

Co-authored-by: Pravesh Kumar <[email protected]>

* Remove TotalPages when save file on local (#684)

* file_name reference and verify_ssl issue fixed (#683)

* User flow changes for recreating supported vector index (#682)

* removed the if check

* Add one more check for create vector index when chunks are exist without embeddings

* removed local files

* condition changes

* chunks exists check

* chunk exists without embeddings check

* vector Index issue fixed

* vector index with different dimension

* Update graphDB_dataAccess.py

---------

Co-authored-by: Pravesh Kumar <[email protected]>

* Reapply "Dockerfile changes with VITE label"

This reverts commit a83e0855fbf54d2b5af009d96c4edf0bcd7ab84a.

* Revert "Dockerfile changes with VITE label"

This reverts commit 2840ebc9e6156c51465a9f54be72ca2d014147c2.

* Concurrent processing of files (#665)

* Update README.md

* Droped the old vector index (#652)

* added cypher_queries and llm chatbot files

* updated llm-chatbot-python

* added llm-chatbot-python

* updated llm-chatbot-python folder

* Added chatbot "hybrid " mode use case

* added the concurrent file processing

* page refresh scenario

* fixed waiting files processing issue in refresh scenario

* removed boolean param

* fixed processedCount issue

* checkbox with waiting check

* fixed the refresh scenario with processing files

* processing files check

* server side error

* processing file count check for processing files less than batch size

* processing count check to handle allselected files

* created helper functions

* code improvements

* __ changes (#656)

* DiffbotGraphTransformer doesn't need an LLMGraphTransformer (#659)

Co-authored-by: jeromechoo <[email protected]>

* Removed experiments/llm-chatbot-python folder from DEV branch

* redcued the password clear timeout

* Removed experiments/Cypher_Queries.ipynb file from DEV branch

* disabled the closed button on banner and connection dialog while API is in pending state

* update delete query with entities

* node id check (#663)

* Status source and type filtering  (#664)

* status source

* Name change

* type change

* rollback to previous working nvl version

* added the alert

* add BATCH_SIZE to docker

* temp fixes for 0.3.1

* alert fix for less than batch size processing

* new virtual env

* added Hybrid Chat modes (#670)

* Rename the function #657

* label and checkboxes placement changes (#675)

* label and checkboxes placement changes

* checkbox placement changes

* Graph node filename check

* env fixes with latest nvl libraries

* format fixes

* removed local files

* Remove TotalPages when save file on local (#684)

* file_name reference and verify_ssl issue fixed (#683)

* User flow changes for recreating supported vector index (#682)

* removed the if check

* Add one more check for create vector index when chunks are exist without embeddings

* removed local files

* condition changes

* chunks exists check

* chunk exists without embeddings check

* vector Index issue fixed

* vector index with different dimension

* Update graphDB_dataAccess.py

---------

Co-authored-by: Pravesh Kumar <[email protected]>

* ndl changes

* label and checkboxes placement changes (#675)

* label and checkboxes placement changes

* checkbox placement changes

* env fixes with latest nvl libraries

* format fixes

* User flow changes for recreating supported vector index (#682)

* removed the if check

* Add one more check for create vector index when chunks are exist without embeddings

* removed local files

* condition changes

* chunks exists check

* chunk exists without embeddings check

* vector Index issue fixed

* vector index with different dimension

* Update graphDB_dataAccess.py

---------

Co-authored-by: Pravesh Kumar <[email protected]>

* env fixes with latest nvl libraries

* format fixes

* User flow changes for recreating supported vector index (#682)

* removed the if check

* Add one more check for create vector index when chunks are exist without embeddings

* removed local files

* condition changes

* chunks exists check

* chunk exists without embeddings check

* vector Index issue fixed

* vector index with different dimension

* Update graphDB_dataAccess.py

---------

Co-authored-by: Pravesh Kumar <[email protected]>

* Status source and type filtering  (#664)

* status source

* Name change

* type change

* added the alert

* temp fixes for 0.3.1

* label and checkboxes placement changes (#675)

* label and checkboxes placement changes

* checkbox placement changes

* env fixes with latest nvl libraries

* format fixes

* User flow changes for recreating supported vector index (#682)

* removed the if check

* Add one more check for create vector index when chunks are exist without embeddings

* removed local files

* condition changes

* chunks exists check

* chunk exists without embeddings check

* vector Index issue fixed

* vector index with different dimension

* Update graphDB_dataAccess.py

---------

Co-authored-by: Pravesh Kumar <[email protected]>

* ndl changes

* env fixes with latest nvl libraries

* format fixes

* User flow changes for recreating supported vector index (#682)

* removed the if check

* Add one more check for create vector index when chunks are exist without embeddings

* removed local files

* condition changes

* chunks exists check

* chunk exists without embeddings check

* vector Index issue fixed

* vector index with different dimension

* Update graphDB_dataAccess.py

---------

Co-authored-by: Pravesh Kumar <[email protected]>

* env fixes with latest nvl libraries

* format fixes

* User flow changes for recreating supported vector index (#682)

* removed the if check

* Add one more check for create vector index when chunks are exist without embeddings

* removed local files

* condition changes

* chunks exists check

* chunk exists without embeddings check

* vector Index issue fixed

* vector index with different dimension

* Update graphDB_dataAccess.py

---------

Co-authored-by: Pravesh Kumar <[email protected]>

* added cypher_queries and llm chatbot files

* updated llm-chatbot-python

* added llm-chatbot-python

* updated llm-chatbot-python folder

* page refresh scenario

* fixed waiting files processing issue in refresh scenario

* Removed experiments/llm-chatbot-python folder from DEV branch

* disabled the closed button on banner and connection dialog while API is in pending state

* node id check (#663)

* Status source and type filtering  (#664)

* status source

* Name change

* type change

* rollback to previous working nvl version

* added the alert

* temp fixes for 0.3.1

* label and checkboxes placement changes (#675)

* label and checkboxes placement changes

* checkbox placement changes

* env fixes with latest nvl libraries

* format fixes

* User flow changes for recreating supported vector index (#682)

* removed the if check

* Add one more check for create vector index when chunks are exist without embeddings

* removed local files

* condition changes

* chunks exists check

* chunk exists without embeddings check

* vector Index issue fixed

* vector index with different dimension

* Update graphDB_dataAccess.py

---------

Co-authored-by: Pravesh Kumar <[email protected]>

* ndl changes

* env fixes with latest nvl libraries

* format fixes

* User flow changes for recreating supported vector index (#682)

* removed the if check

* Add one more check for create vector index when chunks are exist without embeddings

* removed local files

* condition changes

* chunks exists check

* chunk exists without embeddings check

* vector Index issue fixed

* vector index with different dimension

* Update graphDB_dataAccess.py

---------

Co-authored-by: Pravesh Kumar <[email protected]>

* env fixes with latest nvl libraries

* format fixes

* User flow changes for recreating supported vector index (#682)

* removed the if check

* Add one more check for create vector index when chunks are exist without embeddings

* removed local files

* condition changes

* chunks exists check

* chunk exists without embeddings check

* vector Index issue fixed

* vector index with different dimension

* Update graphDB_dataAccess.py

---------

Co-authored-by: Pravesh Kumar <[email protected]>

* Status source and type filtering  (#664)

* status source

* Name change

* type change

* added the alert

* temp fixes for 0.3.1

* label and checkboxes placement changes (#675)

* label and checkboxes placement changes

* checkbox placement changes

* env fixes with latest nvl libraries

* format fixes

* User flow changes for recreating supported vector index (#682)

* removed the if check

* Add one more check for create vector index when chunks are exist without embeddings

* removed local files

* condition changes

* chunks exists check

* chunk exists without embeddings check

* vector Index issue fixed

* vector index with different dimension

* Update graphDB_dataAccess.py

---------

Co-authored-by: Pravesh Kumar <[email protected]>

* ndl changes

* env fixes with latest nvl libraries

* format fixes

* User flow changes for recreating supported vector index (#682)

* removed the if check

* Add one more check for create vector index when chunks are exist without embeddings

* removed local files

* condition changes

* chunks exists check

* chunk exists without embeddings check

* vector Index issue fixed

* vector index with different dimension

* Update graphDB_dataAccess.py

---------

Co-authored-by: Pravesh Kumar <[email protected]>

* env fixes with latest nvl libraries

* format fixes

* User flow changes for recreating supported vector index (#682)

* removed the if check

* Add one more check for create vector index when chunks are exist without embeddings

* removed local files

* condition changes

* chunks exists check

* chunk exists without embeddings check

* vector Index issue fixed

* vector index with different dimension

* Update graphDB_dataAccess.py

---------

Co-authored-by: Pravesh Kumar <[email protected]>

* property spell fix

---------

Co-authored-by: vasanthasaikalluri <[email protected]>
Co-authored-by: Jayanth T <[email protected]>
Co-authored-by: abhishekkumar-27 <[email protected]>
Co-authored-by: Prakriti Solankey <[email protected]>
Co-authored-by: Jerome Choo <[email protected]>
Co-authored-by: jeromechoo <[email protected]>
Co-authored-by: Pravesh Kumar <[email protected]>

* env changes

* format fixes

* set retry status

* retry processing backend

* added the retry icon on rows

* vite changes in docker compose

* added retry dialog

* Integrated the Retry processing API

* Integrated the Extract API fro retry processing

* Integrated ndl toast component

* replaced foreach with normal for loop for better performance

* types improvements

* used toast component

* spell fix

* Issue fixed

* processing changes in main

* function closing fix

* retry processing issue fixed

* autoclosing the retry popup on retry api success

* removed the retry if check

* resetting the node and relationship count on retry

* added the enter key events on the popups

* fixed wikipedia icon on large file alert popup

* setting nodes to 0 and start from last processed chunk logic changes

* Retry Popup fixes

* status changes for upload failed scenario

* kept condition specific

* changed status to reprocess from retry

* Reprocess wording changes

* tooltip changes

* wordings and size changes

* Changed status to Reprocess

* updated node count for start from begnning

---------

Co-authored-by: Pravesh Kumar <[email protected]>
Co-authored-by: kartikpersistent <[email protected]>
Co-authored-by: Prakriti Solankey <[email protected]>
Co-authored-by: vasanthasaikalluri <[email protected]>
Co-authored-by: Jayanth T <[email protected]>
Co-authored-by: abhishekkumar-27 <[email protected]>
Co-authored-by: Jerome Choo <[email protected]>
Co-authored-by: jeromechoo <[email protected]>

* uncommented the Retry Processing

* removed __Entity__ labels

* spell fix

* fixed postprocessing method invoking issue for odd no files

* lint fix

* Added filesource and name in chunks

* Preload=True remove from HSTS

* Graph communities (#748)

* UI changes

* modes enable disable

* separated sources entities chunk communities

* communities added into separate component

* Update ChatInfoModal.tsx

* added filename and source for chunksinfo

* removed the console.log

* mode disable changes

---------

Co-authored-by: kartikpersistent <[email protected]>

* aria label addition

* code improvements used URL class for host url check

* host level check

* Update Security header

* encryption of localstorage values

* 'mode-selection-changes'

* added local chat history

* added neo4j from existing index to entity vector mode

* label changes

* commented security header

* Communities (#721)

* added communities creation

* added communities

* removed tqdm

* removed __Entity__ labels

* removed graph_object

* removed graph object in the function

* Modified queries

* added properties and modified to entity labels

* Post processing call after all files completion (#716)

* Dev To STAGING (#532)

* format fixes and graph schema indication fix

* Update README.md

* added chat modes variable in env updated the readme

* spell fix

* added the chat mode in env table

* added the logos

* fixed the overflow issues

* removed the extra fix

* Fixed specific scenario  "when the text from schema closes it should reopen the previous modal"

* readme changes

* removed dev console logs

* added new retrieval query (#533)

* format fixes and tab rendering fix

* fixed the setting modal reopen issue

---------

Co-authored-by: kartikpersistent <[email protected]>
Co-authored-by: vasanthasaikalluri <[email protected]>

* Dev (#535)

* format fixes and graph schema indication fix

* Update README.md

* added chat modes variable in env updated the readme

* spell fix

* added the chat mode in env table

* added the logos

* fixed the overflow issues

* removed the extra fix

* Fixed specific scenario  "when the text from schema closes it should reopen the previous modal"

* readme changes

* removed dev console logs

* added new retrieval query (#533)

* format fixes and tab rendering fix

* fixed the setting modal reopen issue

---------

Co-authored-by: Prakriti Solankey <[email protected]>
Co-authored-by: vasanthasaikalluri <[email protected]>

* Dev (#537)

* format fixes and graph schema indication fix

* Update README.md

* added chat modes variable in env updated the readme

* spell fix

* added the chat mode in env table

* added the logos

* fixed the overflow issues

* removed the extra fix

* Fixed specific scenario  "when the text from schema closes it should reopen the previous modal"

* readme changes

* removed dev console logs

* added new retrieval query (#533)

* format fixes and tab rendering fix

* fixed the setting modal reopen issue

---------

Co-authored-by: Prakriti Solankey <[email protected]>
Co-authored-by: vasanthasaikalluri <[email protected]>

* Fix typo: correct 'josn_obj' to 'json_obj' (#697)

* Staging To Main (#495)

* Integration_qa test (#375)

* Test IntegrationQA added

* update test cases

* update test

* update node count assertions

* test changes

* update changes

* modification test

* Code refatctor test cases

* Handle allowedlist issue in test

* test changes

* update test

* test case execution

* test chatbot updates

* test case update file

* added file

---------

Co-authored-by: Pravesh Kumar <[email protected]>

* recent merges

* pdf deletion due to out of diskspace

* fixed status blank issue

* Rendering the file name instead of link for gcs and s3 sources in the info modal

* Convert is_cancelled value from string to bool

* added the default page size

* Issue fixed Processed chunked as 0 when file re-process again

* Youtube timestamps (#386)

* Wikipedia source to accept all valid urls

* wikipedia url to support multiple languages

* integrated wiki langauge param for extract api

* Youtube video timestamps

---------

Co-authored-by: kartikpersistent <[email protected]>

* groq llm integration backend (#286)

* groq llm integration backend

* groq and description in node properties

* added groq in options

---------

Co-authored-by: kartikpersistent <[email protected]>

* offset in chunks (#389)

* page number in gcs loader (#393)

* added youtube timestamps (#392)

* chat pop up button (#387)

* expand

* minimize-icon

* css changes

* chat history

* chatbot wider Side Nav

* expand icon

* chatbot UI

* Delete

* merge fixes

* code suggestions

---------

Co-authored-by: kartikpersistent <[email protected]>

* chunks create before extraction using is_pre_process variable (#383)

* chunks create before extraction using is_pre_process variable

* Return total pages for Model

* update requirement.txt

* total pages on uplaod API

* added the Confirmation Dialog

* added the selected files into the confirmation modal

* format and lint fixes

* added the stop watch image

* fileselection on alert dialog

* Add timeout in docker for gunicorn workers

* Add cancel icon to info popup (#384)

* Info Modal Changes

* css changes

* recent merges

* Integration_qa test (#375)

* Test IntegrationQA added

* update test cases

* update test

* update node count assertions

* test changes

* update changes

* modification test

* Code refatctor test cases

* Handle allowedlist issue in test

* test changes

* update test

* test case execution

* test chatbot updates

* test case update file

* added file

---------

Co-authored-by: Pravesh Kumar <[email protected]>

* fixed status blank issue

* Rendering the file name instead of link for gcs and s3 sources in the info modal

* added the default page size

* Convert is_cancelled value from string to bool

* Issue fixed Processed chunked as 0 when file re-process again

* Youtube timestamps (#386)

* Wikipedia source to accept all valid urls

* wikipedia url to support multiple languages

* integrated wiki langauge param for extract api

* Youtube video timestamps

---------

Co-authored-by: kartikpersistent <[email protected]>

* groq llm integration backend (#286)

* groq llm integration backend

* groq and description in node properties

* added groq in options

---------

Co-authored-by: kartikpersistent <[email protected]>

* Save Total Pages in DB

* Added total Pages

* file selection when we didn't select anything from Main table

* added the danger icon only for large files

* added the overflow for more files and file selection for all new files

* moved the interface to types

* added the icon accoroding to the source

* set total page for wiki and youtube

* h3 heading

* merge

* updated the alert on basis if total pages

* deleted chunks

* polling based on total pages

* isNan check

* large file based on file size for s3 and gcs

* file source in server side event

* time calculation based on chunks for gcs and s3

---------

Co-authored-by: kartikpersistent <[email protected]>
Co-authored-by: Prakriti Solankey <[email protected]>
Co-authored-by: abhishekkumar-27 <[email protected]>
Co-authored-by: aashipandya <[email protected]>

* fixed the layout issue

* Populate graph schema (#399)

* crreate new endpoint populate_graph_schema and update the query for getting lables from DB

* Added main.py changes

* conditionally-including-the-gcs-login-flow-in-gcs-as-source (#396)

* added the condtion

* removed llms

* Fixed issue : Remove extra unused param

* get emb only if used (#278)

* Chatbot chunks (#402)

* Added file name to the content  sent to LLM

* added chunk text in the response

* increased the docs parts sent to llm

* Modified graph query

* mardown rendering

* youtube starttime

* icons

* offset changes

* removed the files due to codespace space issue

---------

Co-authored-by: vasanthasaikalluri <[email protected]>
Co-authored-by: kartikpersistent <[email protected]>

* Settings modal to support generating the labels from the llm by using text given by user (#405)

* added the json

* added schema from text dialog

* integrated the schemaAPI

* added the alert

* resize fixes

* fixed css issue

* fixed status blank issue

* Modified response when no docs is retrived (#413)

* Fixed env/docker-compose for local deployments + README doc (#410)

* Fixed env/docker-compose for local deployments + README doc

* wrong place for ENV in README

* by default, removed langsmith + fixed knn score string to float

* by default, removed langsmith + fixed knn score string to float

* Fixed strings in docker-compose env

* Added requirements (neo4j 5.15 or later, APOC, and instructions for Neo4j Desktop)

* Missed the TIME_PER_PAGE env, was causing NaN issue in the approx time processing notification. fixed that

* Support for all unstructured files (#401)

* all unstructured files

* responsiveness

* added file type

* added the extensions

* spell mistake

* ppt file changes

---------

Co-authored-by: kartikpersistent <[email protected]>

* Settings modal to support generating the labels from the llm by using text given by user with checkbox (#415)

* added the json

* added schema from text dialog

* integrated the schemaAPI

* added the alert

* resize fixes

* Extract schema using direct ChatOpenAI API and Chain

* integrated the checkbox for schema to text dialog

* Update SettingModal.tsx

---------

Co-authored-by: Pravesh Kumar <[email protected]>

* gcs file content read via storage client (#417)

* gcs file content read via storage client

* added the access token the file state

---------

Co-authored-by: kartikpersistent <[email protected]>

* pypdf2 to read files from gcs (#420)

* 407 remove driver from frontend (#416)

* removed driver

* removed API

* connecting to database on page refresh

---------

Co-authored-by: kartikpersistent <[email protected]>

* Css handling of info modal and Tooltips (#418)

* css change

* toolTips

* Sidebar Tooltips

* copy to clip

* css change

* added image types

* added gcs

* type fix

* docker changes

* speech

* added the toolip for dropzone sources

---------

Co-authored-by: kartikpersistent <[email protected]>

* Fixed retrival bugs (#421)

* yarn format fixes

* changed the delete message

* added the cancel  button

* changed the message on tooltip

* added space

* UI fixes

* tooltip for setting

* updated req

* wikipedia URL input (#424)

* accept only wikipedia links

* added wikipedia link

* added wikilink regex

* wikipedia single url only

* changed the alert message

* wording change

* pushed validation state persist error

---------

Co-authored-by: aashipandya <[email protected]>

* speech and copy (#422)

* speech and copy

* startTime

* added chunk properties

* tooltips

---------

Co-authored-by: vasanthasaikalluri <[email protected]>
Co-authored-by: kartikpersistent <[email protected]>

* Fixed issue for out of range in KNN API

* solved conflicts

* conflict solved

* Remove logging info from update KNN API

* tooltip changes

* format and lint fixes

* responsiveness changes

* Fixed issue for total pages GCS, S3

* UI polishing (#428)

* button and tooltip changes

* checking validation on change

* settings module populate fix

* format fixes

* opening the modal after auth success

* removed the limit

* added the scrobar for dropdowns

* speech state (#426)

* speech state

* Button Details changes

* delete wording change

* Total pages in buckets (#431)

* page number NA for buckets

* added N/A for gcs and s3 pages

* total pages for gcs

* remove unwanted logger

---------

Co-authored-by: kartikpersistent <[email protected]>

* removed the max width

* Update FileTable.tsx

* Update the docker file

* Modified prompt (#438)

* Update Dockerfile

* Update Dockerfile

* Update Dockerfile

* rendering Fix

* Local file upload gcs (#442)

* Uplaod file to GCS

* GCS local upload fixed issue and delete file from GCS after processing and failed or cancelled

* Add life cycle rule on uploaded bucket

* pdf upload local and gcs bucket check

* delete files when processed and extract changes

---------

Co-authored-by: Pravesh Kumar <[email protected]>

* Modified chat length and entities used (#443)

* metadata for unstructured files (#446)

* Unstructured file metadata (#447)

* metadata for unstructured files

* sleep in gcs upload

* updated

* icons added to chunks (#435)

* icons added to chunks

* info modal icons

* Dev (#433)

* Integration_qa test (#375)

* Test IntegrationQA added

* update test cases

* update test

* update node count assertions

* test changes

* update changes

* modification test

* Code refatctor test cases

* Handle allowedlist issue in test

* test changes

* update test

* test case execution

* test chatbot updates

* test case update file

* added file

---------

Co-authored-by: Pravesh Kumar <[email protected]>

* recent merges

* pdf deletion due to out of diskspace

* fixed status blank issue

* Rendering the file name instead of link for gcs and s3 sources in the info modal

* Convert is_cancelled value from string to bool

* added the default page size

* Issue fixed Processed chunked as 0 when file re-process again

* Youtube timestamps (#386)

* Wikipedia source to accept all valid urls

* wikipedia url to support multiple languages

* integrated wiki langauge param for extract api

* Youtube video timestamps

---------

Co-authored-by: kartikpersistent <[email protected]>

* groq llm integration backend (#286)

* groq llm integration backend

* groq and description in node properties

* added groq in options

---------

Co-authored-by: kartikpersistent <[email protected]>

* offset in chunks (#389)

* page number in gcs loader (#393)

* added youtube timestamps (#392)

* chat pop up button (#387)

* expand

* minimize-icon

* css changes

* chat history

* chatbot wider Side Nav

* expand icon

* chatbot UI

* Delete

* merge fixes

* code suggestions

---------

Co-authored-by: kartikpersistent <[email protected]>

* chunks create before extraction using is_pre_process variable (#383)

* chunks create before extraction using is_pre_process variable

* Return total pages for Model

* update requirement.txt

* total pages on uplaod API

* added the Confirmation Dialog

* added the selected files into the confirmation modal

* format and lint fixes

* added the stop watch image

* fileselection on alert dialog

* Add timeout in docker for gunicorn workers

* Add cancel icon to info popup (#384)

* Info Modal Changes

* css changes

* recent merges

* Integration_qa test (#375)

* Test IntegrationQA added

* update test cases

* update test

* update node count assertions

* test changes

* update changes

* modification test

* Code refatctor test cases

* Handle allowedlist issue in test

* test changes

* update test

* test case execution

* test chatbot updates

* test case update file

* added file

---------

Co-authored-by: Pravesh Kumar <[email protected]>

* fixed status blank issue

* Rendering the file name instead of link for gcs and s3 sources in the info modal

* added the default page size

* Convert is_cancelled value from string to bool

* Issue fixed Processed chunked as 0 when file re-process again

* Youtube timestamps (#386)

* Wikipedia source to accept all valid urls

* wikipedia url to support multiple languages

* integrated wiki langauge param for extract api

* Youtube video timestamps

---------

Co-authored-by: kartikpersistent <[email protected]>

* groq llm integration backend (#286)

* groq llm integration backend

* groq and description in node properties

* added groq in options

---------

Co-authored-by: kartikpersistent <[email protected]>

* Save Total Pages in DB

* Added total Pages

* file selection when we didn't select anything from Main table

* added the danger icon only for large files

* added the overflow for more files and file selection for all new files

* moved the interface to types

* added the icon accoroding to the source

* set total page for wiki and youtube

* h3 heading

* merge

* updated the alert on basis if total pages

* deleted chunks

* polling based on total pages

* isNan check

* large file based on file size for s3 and gcs

* file source in server side event

* time calculation based on chunks for gcs and s3

---------

Co-authored-by: kartikpersistent <[email protected]>
Co-authored-by: Prakriti Solankey <[email protected]>
Co-authored-by: abhishekkumar-27 <[email protected]>
Co-authored-by: aashipandya <[email protected]>

* fixed the layout issue

* Populate graph schema (#399)

* crreate new endpoint populate_graph_schema and update the query for getting lables from DB

* Added main.py changes

* conditionally-including-the-gcs-login-flow-in-gcs-as-source (#396)

* added the condtion

* removed llms

* Fixed issue : Remove extra unused param

* get emb only if used (#278)

* Chatbot chunks (#402)

* Added file name to the content  sent to LLM

* added chunk text in the response

* increased the docs parts sent to llm

* Modified graph query

* mardown rendering

* youtube starttime

* icons

* offset changes

* removed the files due to codespace space issue

---------

Co-authored-by: vasanthasaikalluri <[email protected]>
Co-authored-by: kartikpersistent <[email protected]>

* Settings modal to support generating the labels from the llm by using text given by user (#405)

* added the json

* added schema from text dialog

* integrated the schemaAPI

* added the alert

* resize fixes

* fixed css issue

* fixed status blank issue

* Modified response when no docs is retrived (#413)

* Fixed env/docker-compose for local deployments + README doc (#410)

* Fixed env/docker-compose for local deployments + README doc

* wrong place for ENV in README

* by default, removed langsmith + fixed knn score string to float

* by default, removed langsmith + fixed knn score string to float

* Fixed strings in docker-compose env

* Added requirements (neo4j 5.15 or later, APOC, and instructions for Neo4j Desktop)

* Missed the TIME_PER_PAGE env, was causing NaN issue in the approx time processing notification. fixed that

* Support for all unstructured files (#401)

* all unstructured files

* responsiveness

* added file type

* added the extensions

* spell mistake

* ppt file changes

---------

Co-authored-by: kartikpersistent <[email protected]>

* Settings modal to support generating the labels from the llm by using text given by user with checkbox (#415)

* added the json

* added schema from text dialog

* integrated the schemaAPI

* added the alert

* resize fixes

* Extract schema using direct ChatOpenAI API and Chain

* integrated the checkbox for schema to text dialog

* Update SettingModal.tsx

---------

Co-authored-by: Pravesh Kumar <[email protected]>

* gcs file content read via storage client (#417)

* gcs file content read via storage client

* added the access token the file state

---------

Co-authored-by: kartikpersistent <[email protected]>

* pypdf2 to read files from gcs (#420)

* 407 remove driver from frontend (#416)

* removed driver

* removed API

* connecting to database on page refresh

---------

Co-authored-by: kartikpersistent <[email protected]>

* Css handling of info modal and Tooltips (#418)

* css change

* toolTips

* Sidebar Tooltips

* copy to clip

* css change

* added image types

* added gcs

* type fix

* docker changes

* speech

* added the toolip for dropzone sources

---------

Co-authored-by: kartikpersistent <[email protected]>

* Fixed retrival bugs (#421)

* yarn format fixes

* changed the delete message

* added the cancel  button

* changed the message on tooltip

* added space

* UI fixes

* tooltip for setting

* updated req

* wikipedia URL input (#424)

* accept only wikipedia links

* added wikipedia link

* added wikilink regex

* wikipedia single url only

* changed the alert message

* wording change

* pushed validation state persist error

---------

Co-authored-by: aashipandya <[email protected]>

* speech and copy (#422)

* speech and copy

* startTime

* added chunk properties

* tooltips

---------

Co-authored-by: vasanthasaikalluri <[email protected]>
Co-authored-by: kartikpersistent <[email protected]>

* Fixed issue for out of range in KNN API

* solved conflicts

* conflict solved

* Remove logging info from update KNN API

* tooltip changes

* format and lint fixes

* responsiveness changes

* Fixed issue for total pages GCS, S3

* UI polishing (#428)

* button and tooltip changes

* checking validation on change

* settings module populate fix

* format fixes

* opening the modal after auth success

* removed the limit

* added the scrobar for dropdowns

* speech state (#426)

* speech state

* Button Details changes

* delete wording change

* Total pages in buckets (#431)

* page number NA for buckets

* added N/A for gcs and s3 pages

* total pages for gcs

* remove unwanted logger

---------

Co-authored-by: kartikpersistent <[email protected]>

* removed the max width

* Update FileTable.tsx

* Update the docker file

* Modified prompt (#438)

* Update Dockerfile

* Update Dockerfile

* Update Dockerfile

* rendering Fix

* Local file upload gcs (#442)

* Uplaod file to GCS

* GCS local upload fixed issue and delete file from GCS after processing and failed or cancelled

* Add life cycle rule on uploaded bucket

* pdf upload local and gcs bucket check

* delete files when processed and extract changes

---------

Co-authored-by: Pravesh Kumar <[email protected]>

* Modified chat length and entities used (#443)

* metadata for unstructured files (#446)

* Unstructured file metadata (#447)

* metadata for unstructured files

* sleep in gcs upload

* updated

* icons added to chunks (#435)

* icons added to chunks

* info modal icons

---------

Co-authored-by: abhishekkumar-27 <[email protected]>
Co-authored-by: Pravesh Kumar <[email protected]>
Co-authored-by: kartikpersistent <[email protected]>
Co-authored-by: vasanthasaikalluri <[email protected]>
Co-authored-by: Prakriti Solankey <[email protected]>
Co-authored-by: Ajay Meena <[email protected]>
Co-authored-by: Morgan Senechal <[email protected]>
Co-authored-by: karanchellani <[email protected]>

* fixed gcs status message issue

* added if check for failed count

* Null issue Fixed from backend for upload API and graph_document when model name mismatch

* added word break issue

* Added neo4j-rust-ext

* processing time estimation based on bytes

* File extension upper case fixed, File delete from GCS or local based on env variable.

* timer per byte

* Update Dockerfile

* Adding sort rows on the table (#451)

* Gcs upload folder hashed (#453)

* implement foldername hashed in GCS bucket uplaod

* Raise exception if invalid model selected

* folder name for gcs upload

---------

Co-authored-by: aashipandya <[email protected]>

* upload all unstructuredfiles to gcs (#455)

* Mofified chunk query (#454)

* Added libre office for fixing error -- soffice command was not found. Please install libreoffice
on your system and try again.

- Install instructions: https://www.libreoffice.org/get-help/install-howto/
- Mac: https://formulae.brew.sh/cask/libreoffice
- Debian: https://wiki.debian.org/LibreOffice"

* Fix the PARTIAL CONTENT issue

* File-table no data found (#456)

* 'file-table''

* review comment

* Llm format change (#459)

* changed the llm models format to lowercase

* added the error message

* llm model changes

* format fixes

* removed unused import

* added the capitalize method

* delete files from merged_file_path only if source is local file

---------

Co-authored-by: aashipandya <[email protected]>

* commented total page code (#460)

* format fixes

* removed the disabled check on dropdown

* Large file env

* DEV to STAGING (#461)

* Integration_qa test (#375)

* Test IntegrationQA added

* update test cases

* update test

* update node count assertions

* test changes

* update changes

* modification test

* Code refatctor test cases

* Handle allowedlist issue in test

* test changes

* update test

* test case execution

* test chatbot updates

* test case update file

* added file

---------

Co-authored-by: Pravesh Kumar <[email protected]>

* recent merges

* pdf deletion due to out of diskspace

* fixed status blank issue

* Rendering the file name instead of link for gcs and s3 sources in the info modal

* Convert is_cancelled value from string to bool

* added the default page size

* Issue fixed Processed chunked as 0 when file re-process again

* Youtube timestamps (#386)

* Wikipedia source to accept all valid urls

* wikipedia url to support multiple languages

* integrated wiki langauge param for extract api

* Youtube video timestamps

---------

Co-authored-by: kartikpersistent <[email protected]>

* groq llm integration backend (#286)

* groq llm integration backend

* groq and description in node properties

* added groq in options

---------

Co-authored-by: kartikpersistent <[email protected]>

* offset in chunks (#389)

* page number in gcs loader (#393)

* added youtube timestamps (#392)

* chat pop up button (#387)

* expand

* minimize-icon

* css changes

* chat history

* chatbot wider Side Nav

* expand icon

* chatbot UI

* Delete

* merge fixes

* code suggestions

---------

Co-authored-by: kartikpersistent <[email protected]>

* chunks create before extraction using is_pre_process variable (#383)

* chunks create before extraction using is_pre_process variable

* Return total pages for Model

* update requirement.txt

* total pages on uplaod API

* added the Confirmation Dialog

* added the selected files into the confirmation modal

* format and lint fixes

* added the stop watch image

* fileselection on alert dialog

* Add timeout in docker for gunicorn workers

* Add cancel icon to info popup (#384)

* Info Modal Changes

* css changes

* recent merges

* Integration_qa test (#375)

* Test IntegrationQA added

* update test cases

* update test

* update node count assertions

* test changes

* update changes

* modification test

* Code refatctor test cases

* Handle allowedlist issue in test

* test changes

* update test

* test case execution

* test chatbot updates

* test case update file

* added file

---------

Co-authored-by: Pravesh Kumar <[email protected]>

* fixed status blank issue

* Rendering the file name instead of link for gcs and s3 sources in the info modal

* added the default page size

* Convert is_cancelled value from string to bool

* Issue fixed Processed chunked as 0 when file re-process again

* Youtube timestamps (#386)

* Wikipedia source to accept all valid urls

* wikipedia url to support multiple languages

* integrated wiki langauge param for extract api

* Youtube video timestamps

---------

Co-authored-by: kartikpersistent <[email protected]>

* groq llm integration backend (#286)

* groq llm integration backend

* groq and description in node properties

* added groq in options

---------

Co-authored-by: kartikpersistent <[email protected]>

* Save Total Pages in DB

* Added total Pages

* file selection when we didn't select anything from Main table

* added the danger icon only for large files

* added the overflow for more files and file selection for all new files

* moved the interface to types

* added the icon accoroding to the source

* set total page for wiki and youtube

* h3 heading

* merge

* updated the alert on basis if total pages

* deleted chunks

* polling based on total pages

* isNan check

* large file based on file size for s3 and gcs

* file source in server side event

* time calculation based on chunks for gcs and s3

---------

Co-authored-by: kartikpersistent <[email protected]>
Co-authored-by: Prakriti Solankey <[email protected]>
Co-authored-by: abhishekkumar-27 <[email protected]>
Co-authored-by: aashipandya <[email protected]>

* fixed the layout issue

* Populate graph schema (#399)

* crreate new endpoint populate_graph_schema and update the query for getting lables from DB

* Added main.py changes

* conditionally-including-the-gcs-login-flow-in-gcs-as-source (#396)

* added the condtion

* removed llms

* Fixed issue : Remove extra unused param

* get emb only if used (#278)

* Chatbot chunks (#402)

* Added file name to the content  sent to LLM

* added chunk text in the response

* increased the docs parts sent to llm

* Modified graph query

* mardown rendering

* youtube starttime

* icons

* offset changes

* removed the files due to codespace space issue

---------

Co-authored-by: vasanthasaikalluri <[email protected]>
Co-authored-by: kartikpersistent <[email protected]>

* Settings modal to support generating the labels from the llm by using text given by user (#405)

* added the json

* added schema from text dialog

* integrated the schemaAPI

* added the alert

* resize fixes

* fixed css issue

* fixed status blank issue

* Modified response when no docs is retrived (#413)

* Fixed env/docker-compose for local deployments + README doc (#410)

* Fixed env/docker-compose for local deployments + README doc

* wrong place for ENV in README

* by default, removed langsmith + fixed knn score string to float

* by default, removed langsmith + fixed knn score string to float

* Fixed strings in docker-compose env

* Added requirements (neo4j 5.15 or later, APOC, and instructions for Neo4j Desktop)

* Missed the TIME_PER_PAGE env, was causing NaN issue in the approx time processing notification. fixed that

* Support for all unstructured files (#401)

* all unstructured files

* responsiveness

* added file type

* added the extensions

* spell mistake

* ppt file changes

---------

Co-authored-by: kartikpersistent <[email protected]>

* Settings modal to support generating the labels from the llm by using text given by user with checkbox (#415)

* added the json

* added schema from text dialog

* integrated the schemaAPI

* added the alert

* resize fixes

* Extract schema using direct ChatOpenAI API and Chain

* integrated the checkbox for schema to text dialog

* Update SettingModal.tsx

---------

Co-authored-by: Pravesh Kumar <[email protected]>

* gcs file content read via storage client (#417)

* gcs file content read via storage client

* added the access token the file state

---------

Co-authored-by: kartikpersistent <[email protected]>

* pypdf2 to read files from gcs (#420)

* 407 remove driver from frontend (#416)

* removed driver

* removed API

* connecting to database on page refresh

---------

Co-authored-by: kartikpersistent <[email protected]>

* Css handling of info modal and Tooltips (#418)

* css change

* toolTips

* Sidebar Tooltips

* copy to clip

* css change

* added image types

* added gcs

* type fix

* docker changes

* speech

* added the toolip for dropzone sources

---------

Co-authored-by: kartikpersistent <[email protected]>

* Fixed retrival bugs (#421)

* yarn format fixes

* changed the delete message

* added the cancel  button

* changed the message on tooltip

* added space

* UI fixes

* tooltip for setting

* updated req

* wikipedia URL input (#424)

* accept only wikipedia links

* added wikipedia link

* added wikilink regex

* wikipedia single url only

* changed the alert message

* wording change

* pushed validation state persist error

---------

Co-authored-by: aashipandya <[email protected]>

* speech and copy (#422)

* speech and copy

* startTime

* added chunk properties

* tooltips

---------

Co-authored-by: vasanthasaikalluri <[email protected]>
Co-authored-by: kartikpersistent <[email protected]>

* Fixed issue for out of range in KNN API

* solved conflicts

* conflict solved

* Remove logging info from update KNN API

* tooltip changes

* format and lint fixes

* responsiveness changes

* Fixed issue for total pages GCS, S3

* UI polishing (#428)

* button and tooltip changes

* checking validation on change

* settings module populate fix

* format fixes

* opening the modal after auth success

* removed the limit

* added the scrobar for dropdowns

* speech state (#426)

* speech state

* Button Details changes

* delete wording change

* Total pages in buckets (#431)

* page number NA for buckets

* added N/A for gcs and s3 pages

* total pages for gcs

* remove unwanted logger

---------

Co-authored-by: kartikpersistent <[email protected]>

* removed the max width

* Update FileTable.tsx

* Update the docker file

* Modified prompt (#438)

* Update Dockerfile

* Update Dockerfile

* Update Dockerfile

* rendering Fix

* Local file upload gcs (#442)

* Uplaod file to GCS

* GCS local upload fixed issue and delete file from GCS after processing and failed or cancelled

* Add life cycle rule on uploaded bucket

* pdf upload local and gcs bucket check

* delete files when processed and extract changes

---------

Co-authored-by: Pravesh Kumar <[email protected]>

* Modified chat length and entities used (#443)

* metadata for unstructured files (#446)

* Unstructured file metadata (#447)

* metadata for unstructured files

* sleep in gcs upload

* updated

* icons added to chunks (#435)

* icons added to chunks

* info modal icons

* fixed gcs status message issue

* added if check for failed count

* Null issue Fixed from backend for upload API and graph_document when model name mismatch

* added word break issue

* Added neo4j-rust-ext

* processing time estimation based on bytes

* File extension upper case fixed, File delete from GCS or local based on env variable.

* timer per byte

* Update Dockerfile

* Adding sort rows on the table (#451)

* Gcs upload folder hashed (#453)

* implement foldername hashed in GCS bucket uplaod

* Raise exception if invalid model selected

* folder name for gcs upload

---------

Co-authored-by: aashipandya <[email protected]>

* upload all unstructuredfiles to gcs (#455)

* Mofified chunk query (#454)

* Added libre office for fixing error -- soffice command was not found. Please install libreoffice
on your system and try again.

- Install instructions: https://www.libreoffice.org/get-help/install-howto/
- Mac: https://formulae.brew.sh/cask/libreoffice
- Debian: https://wiki.debian.org/LibreOffice"

* Fix the PARTIAL CONTENT issue

* File-table no data found (#456)

* 'file-table''

* review comment

* Llm format change (#459)

* changed the llm models format to lowercase

* added the error message

* llm model changes

* format fixes

* removed unused import

* added the capitalize method

* delete files from merged_file_path only if source is local file

---------

Co-authored-by: aashipandya <[email protected]>

* commented total page code (#460)

* format fixes

* removed the disabled check on dropdown

* Large file env

---------

Co-authored-by: abhishekkumar-27 <[email protected]>
Co-authored-by: kartikpersistent <[email protected]>
Co-authored-by: aashipandya <[email protected]>
Co-authored-by: vasanthasaikalluri <[email protected]>
Co-authored-by: Prakriti Solankey <[email protected]>
Co-authored-by: Ajay Meena <[email protected]>
Co-authored-by: Morgan Senechal <[email protected]>
Co-authored-by: karanchellani <[email protected]>

* DEV to STAGING (#462)

* Integration_qa test (#375)

* Test IntegrationQA added

* update test cases

* update test

* update node count assertions

* test changes

* update changes

* modification test

* Code refatctor test cases

* Handle allowedlist issue in test

* test changes

* update test

* test case execution

* test chatbot updates

* test case update file

* added file

---------

Co-authored-by: Pravesh Kumar <[email protected]>

* recent merges

* pdf deletion due to out of diskspace

* fixed status blank issue

* Rendering the file name instead of link for gcs and s3 sources in the info modal

* Convert is_cancelled value from string to bool

* added the default page size

* Issue fixed Processed chunked as 0 when file re-process again

* Youtube timestamps (#386)

* Wikipedia source to accept all valid urls

* wikipedia url to support multiple languages

* integrated wiki langauge param for extract api

* Youtube video timestamps

---------

Co-authored-by: kartikpersistent <[email protected]>

* groq llm integration backend (#286)

* groq llm integration backend

* groq and description in node properties

* added groq in options

---------

Co-authored-by: kartikpersistent <[email protected]>

* offset in chunks (#389)

* page number in gcs loader (#393)

* added youtube timestamps (#392)

* chat pop up button (#387)

* expand

* minimize-icon

* css changes

* chat history

* chatbot wider Side Nav

* expand icon

* chatbot UI

* Delete

* merge fixes

* code suggestions

---------

Co-authored-by: kartikpersistent <[email protected]>

* chunks create before extraction using is_pre_process variable (#383)

* chunks create before extraction using is_pre_process variable

* Return total pages for Model

* update requirement.txt

* total pages on uplaod API

* added the Confirmation Dialog

* added the selected files into the confirmation modal

* format and lint fixes

* added the stop watch image

* fileselection on alert dialog

* Add timeout in docker for gunicorn workers

* Add cancel icon to info popup (#384)

* Info Modal Changes

* css changes

* recent merges

* Integration_qa test (#375)

* Test IntegrationQA added

* update test cases

* update test

* update node count assertions

* test changes

* update changes

* modification test

* Code refatctor test cases

* Handle allowedlist issue in test

* test changes

* update test

* test case execution

* test chatbot updates

* test case update file

* added file

---------

Co-authored-by: Pravesh Kumar <[email protected]>

* fixed status blank issue

* Rendering the file name instead of link for gcs and s3 sources in the info modal

* added the default page size

* Convert is_cancelled value from string to bool

* Issue fixed Processed chunked as 0 when file re-process again

* Youtube timestamps (#386)

* Wikipedia source to accept all valid urls

* wikipedia url to support multiple languages

* integrated wiki langauge param for extract api

* Youtube video timestamps

---------

Co-authored-by: kartikpersistent <[email protected]>

* groq llm integration backend (#286)

* groq llm integration backend

* groq and description in node properties

* added groq in options

---------

Co-authored-by: kartikpersistent <[email protected]>

* Save Total Pages in DB

* Added total Pages

* file selection when we didn't select anything from Main table

* added the danger icon only for large files

* added the overflow for more files and file selection for all new files

* moved the interface to types

* added the icon accoroding to the source

* set total page for wiki and youtube

* h3 heading

* merge

* updated the alert on basis if total pages

* deleted chunks

* polling based on total pages

* isNan check

* large file based on file size for s3 and gcs

* file source in server side event

* time calculation based on chunks for gcs and s3

---------

Co-authored-by: kartikpersistent <[email protected]>
Co-authored-by: Prakriti Solankey <[email protected]>
Co-authored-by: abhishekkumar-27 <[email protected]>
Co-authored-by: aashipandya <[email protected]>

* fixed the layout issue

* Populate graph schema (#399)

* crreate new endpoint populate_graph_schema and update the query for getting lables from DB

* Added main.py changes

* conditionally-including-the-gcs-login-flow-in-gcs-as-source (#396)

* added the condtion

* removed llms

* Fixed issue : Remove extra unused param

* get emb only if used (#278)

* Chatbot chunks (#402)

* Added file name to the content  sent to LLM

* added chunk text in the response

* increased the docs parts sent to llm

* Modified graph query

* mardown rendering

* youtube starttime

* icons

* offset changes

* removed the files due to codespace space issue

---------

Co-authored-by: vasanthasaikalluri <[email protected]>
Co-authored-by: kartikpersistent <[email protected]>

* Settings modal to support generating the labels from the llm by using text given by user (#405)

* added the json

* added schema from text dialog

* integrated the schemaAPI

* added the alert

* resize fixes

* fixed css issue

* fixed status blank issue

* Modified response when no docs is retrived (#413)

* Fixed env/docker-compose for local deployments + README doc (#410)

* Fixed env/docker-compose for local deployments + README doc

* wrong place for ENV in README

* by default, removed langsmith + fixed knn score string to float

* by default, removed langsmith + fixed knn score string to float

* Fixed strings in docker-compose env

* Added requirements (neo4j 5.15 or later, APOC, and instructions for Neo4j Desktop)

* Missed the TIME_PER_PAGE env, was causing NaN issue in the approx time processing notification. fixed that

* Support for all unstructured files (#401)

* all unstructured files

* responsiveness

* added file type

* added the extensions

* spell mistake

* ppt file changes

---------

Co-authored-by: kartikpersistent <[email protected]>

* Settings modal to support generating the labels from the llm by using text given by user with checkbox (#415)

* added the json

* added schema from text dialog

* integrated the schemaAPI

* added the alert

* resize fixes

* Extract schema using direct ChatOpenAI API and Chain

* integrated the checkbox for schema to text dialog

* Update SettingModal.tsx

---------

Co-authored-by: Pravesh Kumar <[email protected]>

* gcs file content read via storage client (#417)

* gcs file content read via storage client

* added the access token the file state

---------

Co-authored-by: kartikpersistent <[email protected]>

* pypdf2 to read files from gcs (#420)

* 407 remove driver from frontend (#416)

* removed driver

* removed API

* connecting to database on page refresh

---------

Co-authored-by: kartikpersistent <[email protected]>

* Css handling of info modal and Tooltips (#418)

* css change

* toolTips

* Sidebar Tooltips

* copy to clip

* css change

* added image types

* added gcs

* type fix

* docker changes

* speech

* added the toolip for dropzone sources

---------

Co-authored-by: kartikpersistent <[email protected]>

* Fixed retrival bugs (#421)

* yarn format fixes

* changed the delete message

* added the cancel  button

* changed the message on tooltip

* added space

* UI fixes

* tooltip for setting

* updated req

* wikipedia URL input (#424)

* accept only wikipedia links

* added wikipedia link

* added wikilink regex

* wikipedia single url only

* changed the alert message

* wording change

* pushed validation state persist error

---------

Co-authored-by: aashipandya <[email protected]>

* speech and copy (#422)

* speech and copy

* startTime

* added chunk properties

* tooltips

---------

Co-authored-by: vasanthasaikalluri <[email protected]>
Co-authored-by: kartikpersistent <[email protected]>

* Fixed issue for out of range in KNN API

* solved conflicts

* conflict solved

* Remove logging info from update KNN API

* tooltip changes

* format and lint fixes

* responsiveness changes

* Fixed issue for total pages GCS, S3

* UI polishing (#428)

* button and tooltip changes

* checking validation on change

* settings module populate fix

* format fixes

* opening the modal after auth success

* removed the limit

* added the scrobar for dropdowns

* speech state (#426)

* speech state

* Button Details changes

* delete wording change

* Total pages in buckets (#431)

* page number NA for buckets

* added N/A for gcs and s3 pages

* total pages for gcs

* remove unwanted logger

---------

Co-authored-by: kartikpersistent <[email protected]>

* remov…
  • Loading branch information
1 parent f20edb7 commit 30f6948
Show file tree
Hide file tree
Showing 13 changed files with 54 additions and 19 deletions.
13 changes: 13 additions & 0 deletions frontend/src/App.css
Original file line number Diff line number Diff line change
Expand Up @@ -396,3 +396,16 @@
.tbody-light .ndl-data-grid-tr:hover {
--cell-background: rgb(226 227 229) !important;
}
.chatbot-deleteLoader{
position:absolute;
top:0;
bottom:0;
width:100%;
height:100%;
display: flex;
align-items: center;
justify-content: center;
z-index: 111;
opacity: 0.8;
background-color: rgb(201 201 201 / 40%);
}
7 changes: 6 additions & 1 deletion frontend/src/components/ChatBot/ChatOnlyComponent.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,8 @@ import { getIsLoading } from '../../utils/Utils';
import ThemeWrapper from '../../context/ThemeWrapper';

const ChatContent: React.FC<ChatProps> = ({ chatMessages }) => {
const { clearHistoryData, messages, setMessages, setClearHistoryData } = useMessageContext();
const { clearHistoryData, messages, setMessages, setClearHistoryData, setIsDeleteChatLoading, isDeleteChatLoading } =
useMessageContext();
const { setUserCredentials, setConnectionStatus, connectionStatus, setShowDisconnectButton } = useCredentials();
const [showBackButton, setShowBackButton] = useReducer((state) => !state, false);
const [openConnection, setOpenConnection] = useState<connectionState>({
Expand Down Expand Up @@ -71,13 +72,16 @@ const ChatContent: React.FC<ChatProps> = ({ chatMessages }) => {
const deleteOnClick = async () => {
try {
setClearHistoryData(true);
setIsDeleteChatLoading(true);
const credentials = JSON.parse(localStorage.getItem('neo4j.connection') || '{}') as UserCredentials;
const sessionId = sessionStorage.getItem('session_id') || '';
const response = await clearChatAPI(credentials, sessionId);
setIsDeleteChatLoading(false);
if (response.data.status !== 'Success') {
setClearHistoryData(false);
}
} catch (error) {
setIsDeleteChatLoading(false);
console.error('Error clearing chat history:', error);
setClearHistoryData(false);
}
Expand Down Expand Up @@ -130,6 +134,7 @@ const ChatContent: React.FC<ChatProps> = ({ chatMessages }) => {
clear={clearHistoryData}
isLoading={getIsLoading(messages)}
connectionStatus={connectionStatus}
isDeleteChatLoading={isDeleteChatLoading}
/>
</div>
</div>
Expand Down
21 changes: 14 additions & 7 deletions frontend/src/components/ChatBot/Chatbot.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -41,6 +41,7 @@ import FallBackDialog from '../UI/FallBackDialog';
import { downloadClickHandler, getDateTime } from '../../utils/Utils';
import ChatModesSwitch from './ChatModesSwitch';
import CommonActions from './CommonChatActions';
import Loader from '../../utils/Loader';
const InfoModal = lazy(() => import('./ChatInfoModal'));
if (typeof window !== 'undefined') {
if (!sessionStorage.getItem('session_id')) {
Expand All @@ -58,6 +59,7 @@ const Chatbot: FC<ChatbotProps> = (props) => {
isFullScreen,
connectionStatus,
isChatOnly,
isDeleteChatLoading,
} = props;
const [inputMessage, setInputMessage] = useState('');
const [loading, setLoading] = useState<boolean>(isLoading);
Expand Down Expand Up @@ -249,7 +251,7 @@ const Chatbot: FC<ChatbotProps> = (props) => {
} else {
setListMessages((prev) =>
prev.map((msg) =>
msg.id === chatbotMessageId ? { ...msg, modes: { ...msg.modes, [mode]: responseMode } } : msg
(msg.id === chatbotMessageId ? { ...msg, modes: { ...msg.modes, [mode]: responseMode } } : msg)
)
);
}
Expand All @@ -264,7 +266,7 @@ const Chatbot: FC<ChatbotProps> = (props) => {
} else {
setListMessages((prev) =>
prev.map((msg) =>
msg.id === chatbotMessageId ? { ...msg, modes: { ...msg.modes, [mode]: responseMode } } : msg
(msg.id === chatbotMessageId ? { ...msg, modes: { ...msg.modes, [mode]: responseMode } } : msg)
)
);
}
Expand All @@ -273,15 +275,15 @@ const Chatbot: FC<ChatbotProps> = (props) => {
console.error(`API call failed for mode ${mode}:`, result.reason);
setListMessages((prev) =>
prev.map((msg) =>
msg.id === chatbotMessageId
(msg.id === chatbotMessageId
? {
...msg,
modes: {
...msg.modes,
[mode]: { message: 'Failed to fetch response for this mode.', error: result.reason },
},
}
: msg
: msg)
)
);
}
Expand All @@ -294,7 +296,7 @@ const Chatbot: FC<ChatbotProps> = (props) => {
if (error instanceof Error) {
setListMessages((prev) =>
prev.map((msg) =>
msg.id === chatbotMessageId
(msg.id === chatbotMessageId
? {
...msg,
isLoading: false,
Expand All @@ -306,7 +308,7 @@ const Chatbot: FC<ChatbotProps> = (props) => {
},
},
}
: msg
: msg)
)
);
}
Expand Down Expand Up @@ -408,7 +410,12 @@ const Chatbot: FC<ChatbotProps> = (props) => {
}, []);

return (
<div className={'n-bg-palette-neutral-bg-weak flex flex-col justify-between min-h-full max-h-full overflow-hidden'}>
<div className='n-bg-palette-neutral-bg-weak flex flex-col justify-between min-h-full max-h-full overflow-hidden relative'>
{isDeleteChatLoading && (
<div className='chatbot-deleteLoader'>
<Loader title='Deleting...'></Loader>
</div>
)}
<div
className={`flex overflow-y-auto pb-12 min-w-full pl-5 pr-5 chatBotContainer ${
isChatOnly ? 'min-h-[calc(100dvh-114px)] max-h-[calc(100dvh-114px)]' : ''
Expand Down
1 change: 0 additions & 1 deletion frontend/src/components/ChatBot/MultiModeMetrics.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -317,7 +317,6 @@ export default function MultiModeMetrics({
);
const table = useReactTable(config);


return (
<Box>
{error?.trim() != '' ? (
Expand Down
3 changes: 2 additions & 1 deletion frontend/src/components/Layout/DrawerChatbot.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ import { useLocation } from 'react-router';
import { useEffect } from 'react';

const DrawerChatbot: React.FC<DrawerChatbotProps> = ({ isExpanded, clearHistoryData, messages, connectionStatus }) => {
const { setMessages } = useMessageContext();
const { setMessages, isDeleteChatLoading } = useMessageContext();
const location = useLocation();

useEffect(() => {
Expand All @@ -29,6 +29,7 @@ const DrawerChatbot: React.FC<DrawerChatbotProps> = ({ isExpanded, clearHistoryD
clear={clearHistoryData}
isLoading={getIsLoading(messages)}
connectionStatus={connectionStatus}
isDeleteChatLoading={isDeleteChatLoading}
/>
</Drawer.Body>
</Drawer>
Expand Down
8 changes: 5 additions & 3 deletions frontend/src/components/Layout/PageLayout.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -51,7 +51,7 @@ const PageLayout: React.FC = () => {
}
};

const { messages, setClearHistoryData, clearHistoryData, setMessages } = useMessageContext();
const { messages, setClearHistoryData, clearHistoryData, setMessages, setIsDeleteChatLoading } = useMessageContext();
const { isSchema, setIsSchema, setShowTextFromSchemaDialog, showTextFromSchemaDialog } = useFileContext();
const {
setConnectionStatus,
Expand All @@ -63,7 +63,7 @@ const PageLayout: React.FC = () => {
showDisconnectButton,
} = useCredentials();
const { cancel } = useSpeechSynthesis();

useEffect(() => {
async function initializeConnection() {
const session = localStorage.getItem('neo4j.connection');
Expand Down Expand Up @@ -193,14 +193,15 @@ const PageLayout: React.FC = () => {
const deleteOnClick = async () => {
try {
setClearHistoryData(true);
setIsDeleteChatLoading(true);
cancel();
const response = await clearChatAPI(
userCredentials as UserCredentials,
sessionStorage.getItem('session_id') ?? ''
);
setIsDeleteChatLoading(false);
if (response.data.status === 'Success') {
const date = new Date();

setMessages([
{
datetime: `${date.toLocaleDateString()} ${date.toLocaleTimeString()}`,
Expand All @@ -218,6 +219,7 @@ const PageLayout: React.FC = () => {
navigate('.', { replace: true, state: null });
}
} catch (error) {
setIsDeleteChatLoading(false);
console.log(error);
setClearHistoryData(false);
}
Expand Down
3 changes: 2 additions & 1 deletion frontend/src/components/Layout/SideNav.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -43,7 +43,7 @@ const SideNav: React.FC<SideNavProps> = ({
}) => {
const [isChatModalOpen, setIsChatModalOpen] = useState(false);
const [isFullScreen, setIsFullScreen] = useState(false);
const { setMessages } = useMessageContext();
const { setMessages, isDeleteChatLoading } = useMessageContext();
const [showChatMode, setshowChatMode] = useState<boolean>(false);
const largedesktops = useMediaQuery(`(min-width:1440px )`);
const { connectionStatus, isReadOnlyUser } = useCredentials();
Expand Down Expand Up @@ -267,6 +267,7 @@ const SideNav: React.FC<SideNavProps> = ({
setMessages={setMessages}
isLoading={getIsLoading(messages ?? [])}
connectionStatus={connectionStatus}
isDeleteChatLoading={isDeleteChatLoading}
/>
</Dialog.Content>
</Dialog>,
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -450,6 +450,7 @@ export default function ConnectionModal({
type: 'password',
'aria-label': 'Password',
placeholder: 'password',
autoComplete: 'current-password',
}}
value={password}
isDisabled={false}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -104,12 +104,12 @@ export default function DeduplicationTab() {
const onRemove = (nodeid: string, similarNodeId: string) => {
setDuplicateNodes((prev) => {
return prev.map((d) =>
d.e.elementId === nodeid
(d.e.elementId === nodeid
? {
...d,
similar: d.similar.filter((n) => n.elementId != similarNodeId),
}
: d
: d)
);
});
};
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -11,11 +11,11 @@ export default function SelectedJobList({
}) {
const ongoingPostProcessingTasks = useMemo(
() =>
isGdsActive
(isGdsActive
? postProcessingTasks.includes('enable_communities')
? postProcessingTasks
: postProcessingTasks.filter((s) => s != 'enable_communities')
: postProcessingTasks.filter((s) => s != 'enable_communities'),
: postProcessingTasks.filter((s) => s != 'enable_communities')),
[isGdsActive, postProcessingTasks]
);
return (
Expand Down
3 changes: 3 additions & 0 deletions frontend/src/context/UserMessages.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -7,12 +7,15 @@ const MessageContext = createContext<MessageContextType | undefined>(undefined);
const MessageContextWrapper: FC<MessagesContextProviderProps> = ({ children }) => {
const [messages, setMessages] = useState<Messages[] | []>(getDefaultMessage);
const [clearHistoryData, setClearHistoryData] = useState<boolean>(false);
const [isDeleteChatLoading, setIsDeleteChatLoading] = useState(false);

const value: MessageContextType = {
messages,
setMessages,
clearHistoryData,
setClearHistoryData,
isDeleteChatLoading,
setIsDeleteChatLoading,
};
return <MessageContext.Provider value={value}>{children}</MessageContext.Provider>;
};
Expand Down
3 changes: 3 additions & 0 deletions frontend/src/types.ts
Original file line number Diff line number Diff line change
Expand Up @@ -253,6 +253,7 @@ export type ChatbotProps = {
isFullScreen?: boolean;
connectionStatus: boolean;
isChatOnly?: boolean;
isDeleteChatLoading: boolean;
};
export interface WikipediaModalTypes extends Omit<S3ModalProps, ''> {}

Expand Down Expand Up @@ -763,6 +764,8 @@ export interface MessageContextType {
setMessages: Dispatch<SetStateAction<Messages[]>>;
clearHistoryData: boolean;
setClearHistoryData: Dispatch<SetStateAction<boolean>>;
isDeleteChatLoading: boolean;
setIsDeleteChatLoading: Dispatch<SetStateAction<boolean>>;
}

export interface GraphContextType {
Expand Down
2 changes: 1 addition & 1 deletion frontend/src/utils/Utils.ts
Original file line number Diff line number Diff line change
Expand Up @@ -395,7 +395,7 @@ export const isFileCompleted = (waitingFile: CustomFile, item: SourceNode) =>
waitingFile && item.status === 'Completed';

export const calculateProcessedCount = (prev: number, batchSize: number) =>
prev === batchSize ? batchSize - 1 : prev + 1;
(prev === batchSize ? batchSize - 1 : prev + 1);

export const isProcessingFileValid = (item: SourceNode, userCredentials: UserCredentials) => {
return item.status === 'Processing' && item.fileName != undefined && userCredentials && userCredentials.database;
Expand Down

0 comments on commit 30f6948

Please sign in to comment.