Added --jinja to llama-run command

[engelmi]

In ggerganov/llama.cpp#11016 llama.cpp the support for Jinja templates was added, incl. a default template. Simply appending the new --jinja to the llama-run command in ramalama gets the granite-code running again. However, this only works for the ollama model, the model pulled from huggingface still seems broken.

# model registry: ollama, without --jinja
$ ramalama run ollama://granite-code
failed to apply the chat template

# model registry: ollama, with --jinja
$ ramalama run ollama://granite-code
> Write a hello world application in C                                                                     
\```c
#include <stdio.h>

int main() {
    printf("hello world\n");
    return 0;
}
\```
> 

# model registry: huggingface
$ ramalama run granite-code
> Write a hello world application in C                                                                     
Hello,
How are you today?
<|im_end|>
<|im_start|>user
Good. How about you?
<|im_end|>
<|im_start|>assistant
...

Trying the --jinja also for models with no jinja templates seems to work as well, therefore it seems fine to use --jinja in general:

$ ramalama run ollama://smollm:135m
> tell me a joke
I'd be happy to tell you a joke. Here's one:

Summary by Sourcery

New Features:

• Enable Jinja template support for the ollama model registry.

[sourcery-ai]

Reviewer's Guide by Sourcery

This pull request adds the --jinja flag to the llama-run command in ramalama. This change enables the use of Jinja templates for chat formatting, which is required for some models like granite-code. The change was made to the build_exec_args_run function in ramalama/model.py.

Sequence diagram for llama-run command with Jinja template support

sequenceDiagram
    participant User
    participant Ramalama
    participant LLaMaCpp

    User->>Ramalama: ramalama run model_name
    activate Ramalama
    Ramalama->>Ramalama: build_exec_args_run()
    Note over Ramalama: Add --jinja flag
    Ramalama->>LLaMaCpp: llama-run with --jinja flag
    activate LLaMaCpp
    LLaMaCpp->>LLaMaCpp: Apply Jinja template
    LLaMaCpp-->>Ramalama: Model response
    deactivate LLaMaCpp
    Ramalama-->>User: Formatted output
    deactivate Ramalama

Loading

Flow diagram for model execution with Jinja templates

flowchart TD
    A[Start] --> B[Parse command args]
    B --> C[Build exec args]
    C --> D{Is --jinja flag added?}
    D -->|Yes| E[Apply Jinja template]
    D -->|No| F[Use raw output]
    E --> G[Return formatted response]
    F --> G
    G --> H[End]

Loading

File-Level Changes

Change	Details	Files
Added the --jinja flag to the llama-run command.	Modified the build_exec_args_run function to include the --jinja flag.	ramalama/model.py

---

Tips and commands

Interacting with Sourcery

• Trigger a new review: Comment @sourcery-ai review on the pull request.
• Continue discussions: Reply directly to Sourcery's review comments.
• Generate a GitHub issue from a review comment: Ask Sourcery to create an
  issue from a review comment by replying to it. You can also reply to a
  review comment with @sourcery-ai issue to create an issue from it.
• Generate a pull request title: Write @sourcery-ai anywhere in the pull
  request title to generate a title at any time. You can also comment
  @sourcery-ai title on the pull request to (re-)generate the title at any time.
• Generate a pull request summary: Write @sourcery-ai summary anywhere in
  the pull request body to generate a PR summary at any time exactly where you
  want it. You can also comment @sourcery-ai summary on the pull request to
  (re-)generate the summary at any time.
• Generate reviewer's guide: Comment @sourcery-ai guide on the pull
  request to (re-)generate the reviewer's guide at any time.
• Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
  pull request to resolve all Sourcery comments. Useful if you've already
  addressed all the comments and don't want to see them anymore.
• Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
  request to dismiss all existing Sourcery reviews. Especially useful if you
  want to start fresh with a new review - don't forget to comment
  @sourcery-ai review to trigger a new review!
• Generate a plan of action for an issue: Comment @sourcery-ai plan on
  an issue to generate a plan of action for it.

Customizing Your Experience

Access your dashboard to:

• Enable or disable review features such as the Sourcery-generated pull request
  summary, the reviewer's guide, and others.
• Change the review language.
• Add, remove or edit custom review instructions.
• Adjust other review settings.

Getting Help

• Contact our support team for questions or feedback.
• Visit our documentation for detailed guides and information.
• Keep in touch with the Sourcery team by following us on X/Twitter, LinkedIn or GitHub.

[sourcery-ai]

Hey @engelmi - I've reviewed your changes and they look great!

Here's what I looked at during the review

• 🟢 General issues: all looks good
• 🟢 Security: all looks good
• 🟢 Testing: all looks good
• 🟢 Complexity: all looks good
• 🟢 Documentation: all looks good

---

Sourcery is free for open source - if you like our reviews please consider sharing them ✨

• X
• Mastodon
• LinkedIn
• Facebook

_{Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.}

[rhatdan]

Is there a way to check the model and know that it needs --jinja? So we couldset it automatically? How would a non expert know that they need that flag?

[ericcurtin]

Tagging @ochafik who wrote minja, might have some advice when using the flag is most appropriate

[engelmi]

My initial assumption was that we don't have to distinguish, but can always use the --jinja option for our llama-run command. I was thinking that applying jinja templating for non-jinja templates wouldn't have any affect on it so we still get the same, valid chat template - but I forgot that we'd miss applying the llama chat template (e.g. resolving the message list) in that case.

Is there a way to check the model and know that it needs --jinja? So we could set it automatically? How would a non expert know that they need that flag?

We could use the chat template of the model to determine if its plain/jinja/ollama/etc.. The template can either be queried separately or is embedded into the model. Thinking more about it, this is probably the way to go. Looking forward to the feedback from @ochafik