Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Mermaid Chart: Inconsistent Generation #2

Open
TahaTobaili opened this issue Feb 27, 2024 · 0 comments
Open

Mermaid Chart: Inconsistent Generation #2

TahaTobaili opened this issue Feb 27, 2024 · 0 comments
Labels
bug Something isn't working help wanted Extra attention is needed

Comments

@TahaTobaili
Copy link
Owner

Following on the previous issue we now need to generate a nice mermaid chart to supplement the output summary to enhance the user's understanding of the summary.

We are building the chart in two steps:

  1. Generating it.
  2. Displaying it.

The problem is in generating it, check the generate_chart(transcript), it calls ChatGPT to generate it. The problem is that the result is not always accurate (tested on GPT-4) sometime it is a correct chart, sometimes it is not.
To test the chart, run the output on Mermaid's live editor.

Observation: GPT seems to work better with very short videos.

What needs to be done:

  1. Prompting: Play with the prompt a bit to see if you can reach consistent working charts.
  2. Different models: Test against different LLM models.
  3. Finetuning: Can you get hands on a dataset that consists of blocks of texts and their equivalent mermaid charts? Then either provide us with this dataset or finetune any LLM model and share your code if the results are better than GPT.

The chart is then displayed via st_mermaid(mermaid_code), no issues with that.

@TahaTobaili TahaTobaili added bug Something isn't working help wanted Extra attention is needed labels Feb 27, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working help wanted Extra attention is needed
Projects
None yet
Development

No branches or pull requests

1 participant