Merge pull request #87 from bioimage-io/cleanup-repo

Cleanup repo

docs/technical-overview.md

@@ -11,9 +11,9 @@ Users can input their profiles as depicted in Figure 2.
 ![user-profile](./screenshots/user-profile.png)
 *Figure 2. Users can personalize responses by clicking `Edit Profile` and save their settings for future conversations by clicking `Save`.*
 
-As of today, our chatbot utilizes knowledge bases from the following pivotal communities: bioimage.io [2], Imjoy [3], deepimageJ [4], ImageJ [5], bio.tools [6], and scikit-image [7]. A key feature allows users to specify a preferred channel for information retrieval, as shown in Figure 3. If a channel is designated, the chatbot sources information from that specific community. Otherwise, it uses an intelligent selection process driven by a schema-based agent to choose the most relevant channel based on the user's query.
+As of today, our chatbot integrates 6 extensions including document search in bioimage.io knowledge base, tools search on Bioimage Informatics Index (biii.eu), bioimage topics search in Bioimage Archive and Image.cs Forum, web search, and information search in Bioimage Model Zoo. The document search utilizes knowledge bases from the following pivotal communities: bioimage.io [2], Imjoy [3], deepimageJ [4], ImageJ [5], bio.tools [6], and scikit-image [7]. We also allow users to specify a preferred extension for information retrieval, as shown in Figure 3. If an extension is designated, the chatbot sources information using the specific extension and its corresponding source. Otherwise, it uses an intelligent selection process driven by a schema-based agent to choose the most relevant extension based on the user's query. 
 
-![channels](./screenshots/channels.png)
+![channels](./screenshots/extensions.png)
 *Figure 3. Users can personalize the conversation by selecting a specific channel from the ‘Knowledge Base Channel’.*
 
 ### Building the Knowledge Base
@@ -29,45 +29,46 @@ The schema-based agent operates on the function-call LLM [8], and uses input and
 ![role_create](./screenshots/role_create.png)
 *Figure 4. Creation of a chatbot role class named ‘CustomerServiceRole’ by defining fields of the role class.*
 
-## Response Modes
+## Extensions
 The BioImage.IO Chatbot employs diverse methods to generate responses, currently encompassing five distinct response modes. The response mode is chosen by the schema-based agent based on the user's query and the selected channel.
 
-### Direct Response
-This mode delivers concise, direct answers for straightforward queries, utilizing the chatbot's internal knowledge base without external data retrieval. 
+### Search BioImage Docs
+This extension allows the chatbot to search information in a community-driven bioimage related knowledge base. With a specific query, the chatbot extracts essential elements from the user's question to fetch information from the relevant documentation. 
 
-    ![direct-response](./screenshots/direct-response.png)
-    *Figure 6. Direct response example.*
+    ![direct-response](./screenshots/search-bioimage-docs.png)
+    *Figure 6. Search in Bioimage Knolwedge base documentation.*
 
-### Document Retrieval Response
-Complex or specific queries trigger this mode, where the chatbot extracts essential elements from the user's question to fetch information from the relevant documentation. 
+### Search BioImage Information Index (biii.eu)
+This extension allows the chatbot to search online software tool in biii.eu.
+    ![search-biii](./screenshots/search-biii.png)
+    *Figure 7. Search in biii.eu.*
 
-    ![retrieval-text](./screenshots/retrieval-text.png)
-    *Figure 7. Document retrieval from knowledge base.*
+The process begins with an initial response based on the user's query (`request`), which serves as a foundation for generating a new `query` for targeted information retrieval. This is combined with user profile data (`user_info`) and the query to produce a comprehensive final response.
 
-The process begins with an initial response based on the user's query (`request`), which serves as a foundation for generating a new `query` for targeted information retrieval. This is combined with user profile data (`user_info`) and the chosen retrieval channel (`channel_id`) to produce a comprehensive final response.
+### Search Bioimage Archive
+This extension allows the chatbot to search for dataset index in bioimage archive. 
+    ![search-bioimage-archive](./screenshots/search-bioimage-archive.png)
+    *Figure 8. Search in bioimage archive.*
 
-### Scripting Retrieval Response
-This mode is designed for queries requiring detailed model information or specific actions, generating and executing Python scripts for tailored solutions.
-
-    ![script-gen-exe-retrieval](./screenshots/script-gen-exe-retrieval.png)
-    *Figure 8. Scripting retrieval for complex queries.*
-
-It involves creating a `ModelZooInfoScript` schema with fields like `request`, `user info`, and `script`, where `script` is Python code for API interactions or data manipulation. The final response is formulated by integrating the script's output with the `request` and `user info`.
+### Search image.sc Forum
+This extension allows the chatbot to search bioimage related topics and software issues in the image.sc forum.
+    ![search-image-sc](./screenshots/search-image-forum.png)
+    *Figure 9. Search in image.sc forum.*
 
-### API Call Response
-This mode integrates user-provided APIs for data processing, currently implementing Cellpose[9] for image segmentation tasks on user-uploaded images.
+### Search Web
+This extension allows the chatbot to search for information from the web. This extension is triggered while the chatbot realizes it can not find relevant information from the knowledge base.
 
-    TODO: add an example screenshot.
+    ![search-web](./screenshots/search-web.png)
+    *Figure 10. Search in the web.*
 
-The chatbot translates analysis requests into a `CellposeTask` schema, detailing the segmentation task per the `TaskChoice` schema. An LLM agent identifies image axes labels using a `LabeledImage` schema, ensuring compatibility with the segmentation process. Incompatible images trigger a `CellposeHelp` schema for guidance.
 
-### Specialized Skill Response
-This mode offers enriched responses for specific user needs, activating specialized modes like `Learning` and `Coding`.
-
-    TODO: add an example screenshot.
+### BioImage Model Zoo
+This mode is designed for queries requiring detailed model information or specific actions, generating and executing Python scripts for tailored solutions.
 
-The `Learning` mode provides educational responses enriched with key terms and concepts, while the `Coding` mode generates functional code snippets for specific tasks. These specialized responses are contrasted with the more general Direct Response mode and can be extended or chosen as a channel in the channel selection process.
+    ![script-gen-exe-retrieval](./screenshots/search-model-zoo.png)
+    *Figure 11. Scripting retrieval for complex queries.*
 
+It involves creating a `ModelZooInfoScript` schema with fields like `request`, `user info`, and `script`, where `script` is Python code for API interactions or data manipulation. The final response is formulated by integrating the script's output with the `request` and `user info`.
 
 ## References
 

docs/usage-example.md

@@ -24,8 +24,7 @@ You can ask the BioImage.IO Chatbot a wide range of general and straightforward
 ### Technical Questions
 If you have more technical questions, the chatbot is equipped to assist you effectively. It retrieves the necessary information, summarizes it, and presents it in a simple and understandable manner. For instance, you can ask the chatbot about procedures like "How to segment an image using Fiji" or "How to upload a model to the bioimage.io repository."
 
-![Segment an Image using Fiji](./screenshots/chatbot-technical-question.png)
-![How to upload a model to bioimage.io](./screenshots/chatbot-model-upload.png)
+![Segment an Image using Fiji](./screenshots/search-bioimage-docs.png)
 
 
 ### Personalization: tailored Responses
@@ -36,43 +35,48 @@ To illustrate this personalized approach, we have two screenshots of the chatbot
 ![Abby, a biologist, willing to analyze biological images.](./screenshots/chatbot-biologist.png)
 ![Abby, a developer, willing to analyze biological images.](./screenshots/chatbot-developer.png)
 
+### How to switch chatbot's mode
+We have three assistants built in the BioImage.IO Chatbot to cater to the different purpose and needs of the users. BioImage Analysis Seeker (BIA Seeker) is designated for users that seek for information about bioimage analysis, it is equiped with 6 extensions including community-driven knowledge base, as well as online source like image.sc forum, web search. BIA Executor is desginated for users that want to execute model using BioEngine such as CellPose. BIA Tutor is designated for users that seek for tutoring in bioimage analysis, it is equiped with Bioimage Books extension combined with web search and Bioimage Model Zoo search. 
+![BIA Executor](./screenshots/bia-executor.png)
+![BIA Tutor](./screenshots/bia-tutor.png)
+
 ## Database Integration
 Our system harnesses knowledge from a variety of valuable sources, including established databases such as [ELIXIR bio.tools](https://bio.tools) and the [ImageJ Wiki](https://imagej.net/software/imagej2/), tool-specific documentation like [deepImageJ](https://deepimagej.github.io), [ilastik](https://www.ilastik.org), and [ImJoy](https://imjoy.io/#/), and structured databases such as the [BioImage Model Zoo repository](https://bioimage.io).
 In the chatbot's user interface, you'll find a list of the available knowledge base channels that the chatbot can access. s. You can choose to select a specific knowledge base channel or opt for the `auto` mode to query information from all integrated databases.
 
-### Knowledge Base Channel Selection
-If you are interested in retrieving information from a specific database, you can ask a question and select the "Knowledge Base Channel" that you are more interested in. This way, the information provided will be exclusively from the channel of information you have selected.
-
-For this example, you can select the `scikit-image` channel and ask the chatbot for image processing algorithms suitable for image enhancement.
-
-You will receive a list of image processing algorithms for image enhancement from the `scikit-image` database.
-![scikit-image Integration](./screenshots/chatbot-channel-scikit-image.png)
+## Extension Selection
+If you are interested in using specific extensions in the chatbot, you can click 'More Options' and select one or several extensions from the extension list. This way, the information provided will be exclusively by using the specified extensions you have selected.
 
-To see the difference between the channels, you can select the `BioImage Model Zoo` channel and ask the chatbot for models suitable for image classification.
+For this example, you can select the `Search Image.cs Forum` extension and ask the chatbot for software issues.
 
-You will receive a list of models for image classification from the `BioImage Model Zoo` database.
-![BioImage Model Zoo Integration](./screenshots/chatbot-channel-bioimageio.png)
+![Select extension](./screenshots/select-extension.png)
+![Image.sc Forum](./screenshots/search-image-forum.png)
 
+### Steps details of Chatbot's working trajectory
+In order to check the chatbot working trajectories, with every query, all steps of chatbot's working trajectory are integrated in the response. Click 'More Details' on the bottom of the response, users can see the detail of each step. 
 
+![More details](./screenshots/more-details.png)
 
-### Knowledge Base Channel: automatic model retrieval
+### Search in Knowledge Base 
 Our system harnesses knowledge from a variety of valuable sources, including established databases such as [ELIXIR bio.tools](https://bio.tools) and the [ImageJ Wiki](https://imagej.net/software/imagej2/), tool-specific documentation like [deepImageJ](https://deepimagej.github.io), [ilastik](https://www.ilastik.org), and [ImJoy](https://imjoy.io/#/), and structured databases such as the [BioImage Model Zoo repository](https://bioimage.io). This comprehensive integration enables our chatbot to not only respond to questions related to each of these individual tools but also tackle complex inquiries that demand information from multiple databases.
 
 To illustrate this capability, consider the following scenario: You have a Whole Slide Image (WSI) that you'd like to open with [QuPath](https://qupath.github.io), apply the Chatty-Frog model ([StarDist H&E Nuclei Segmentation](https://bioimage.io/#/?tags=chatty-frog&id=10.5281%2Fzenodo.6338614)) from BioImage.IO using deepImageJ within Fiji. How can this be accomplished? You can present this intricate question to the BioImage.IO Chatbot.
 
 The chatbot provides a detailed breakdown of the steps required to complete the requested pipeline. This example demonstrates how our chatbot excels at efficiently retrieving and summarizing information from various sources, offering a valuable solution for your bioimage analysis needs.
 
-![Cross-Database Query](./screenshots/chatbot-wsi-pipeline.png)
+![Cross-Database Query](./screenshots/search-bioimage-docs.png)
 
-## Code generation and execution
 ### Retrieving Models
 The BioImage.IO Chatbot is a versatile tool that can generate and execute code in response to user queries. This means that when a user asks about specific models available in bioimage.io, the chatbot can fetch this information by generating a custom script.
 
 For instance, if a user inquires about the total number of models in the BioImage Model Zoo, the chatbot can effortlessly retrieve this information using a straightforward script. However, the questions can become more intricate. Users might want to know which models in bioimage.io are suitable for a particular tag or task (*i.e., segmentation*)or similar criteria. The chatbot is adept at handling these more complex queries, too.
 
-![Model Queries](./screenshots/script-gen-exe-retrieval.png)
+![Model Queries](./screenshots/search-model-zoo.png)
+
+
+### Model execution
+TODO: Update the description for model execution and screenshot for cellpose example.
 
-### API Calls
 The BioImage.IO Chatbot can also execute functions in external APIs, analysis pipelines, plugins, and user code. Cellpose image segmentation has been added as a Chatbot function calling extension and can be used as a template for additional APIs. Users may currently use the Cellpose API function calling by uploading a file and requesting the Chatbot to segment it either from the `auto` or `cellpose` channels.
 
 ## Conclusion