Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

📝 docs: improve askAI plugin documentation #630

Open
wants to merge 10 commits into
base: main
Choose a base branch
from
Open

📝 docs: improve askAI plugin documentation #630

Show file tree
Hide file tree
Changes from 9 commits
Commits
Show all changes
10 commits
Select commit Hold shift + click to select a range
2ae58c1
📝 docs: improve askAI plugin documentation and FAQ clarity
azigler Jan 8, 2025
8123b12
Merge branch 'main' into azigler-improve-askai-docs-1
PavelLinearB Jan 8, 2025
c1440a3
📝 docs: improve clarity of AI plugin documentation and fix example sy…
azigler Jan 8, 2025
baae64c
📝 docs: improve clarity and consistency in askAI documentation
azigler Jan 8, 2025
0331f23
📝 docs: improve titles and descriptions in AI integration documentation
azigler Jan 9, 2025
05bb5f8
Merge branch 'main' into azigler-improve-askai-docs-1
azigler Jan 9, 2025
8343db2
Merge branch 'main' into azigler-improve-askai-docs-1
PavelLinearB Jan 14, 2025
cd859d0
Merge branch 'main' into azigler-improve-askai-docs-1
PavelLinearB Jan 14, 2025
3c760c9
Merge branch 'main' into azigler-improve-askai-docs-1
azigler Jan 27, 2025
cb80d1f
Update docs/integrations/askAI.md
azigler Jan 28, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 6 additions & 6 deletions docs/automations/integrations/askAI/add-tests/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,24 +1,24 @@
---
title: Automation - Ask AI to Suggest Tests
description: Use gitStream's integration with AI services to suggests additional tests
title: Automation - Ask AI to Generate Unit Tests
description: Use gitStream's integration with AI services to recommend tests for the changes in the PR.
category: [quality, genai, copilot, tests, efficiency]
starter_kits: [genai]
---
# Ask AI to Suggest Tests
# Ask AI to Generate Unit Tests

<!-- --8<-- [start:example]-->
Use AI to suggests additional test cases for uncovered or modified functions in the PR, including edge cases.
Use AI to recommend unit tests for the changes in a PR.

!!! info "Configuration Description"

Conditions (all must be true):

* A PR is created or new code has been committed to the PR.
* The PR has a label "askai tests"
* The PR has a label `askai tests`.

Automation Actions:

* Add a comment with suggested tests generated by an AI model
* Add a comment with AI-generated tests recommendations for the PR.

!!! example "Configuration Example"
```yaml+jinja
Expand Down
12 changes: 6 additions & 6 deletions docs/automations/integrations/askAI/code-review/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,24 +1,24 @@
---
title: Automation - Ask AI for a Code Review
description: Use gitStream's integration with AI services to perform a comprehensive code review to your code
title: Automation - Ask AI for a PR Review Checklist
description: Use gitStream's integration with AI services to perform a initial code review for a PR.
category: [quickstart, quality, genai, copilot, tests, efficiency]
starter_kits: [genai]
---
# Ask AI for a Code Review
# Ask AI for a PR Review Checklist

<!-- --8<-- [start:example]-->
Use AI to perform a comprehensive code review, identify bugs, security risks, performance issues, deprecated methods, and style guide violations, and suggests improvements.
Use AI to generate an initial code review checklist, identify bugs, security risks, performance issues, deprecated methods, and style guide violations, and suggests improvements. The automation provides insights and recommended next actions for a human reviewer.

!!! info "Configuration Description"

Conditions (all must be true):

* A PR is created or new code has been committed to the PR.
* The PR has a label "askai cr"
* The PR has a label `askai cr`.

Automation Actions:

* Add a comment with a code review generated by an AI model
* Add a comment with an AI-generated code review checklist for the PR.

!!! example "Configuration Example"
```yaml+jinja
Expand Down
15 changes: 8 additions & 7 deletions docs/automations/integrations/askAI/document/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,23 +1,24 @@
---
title: Automation - Ask AI to provide PR documentation
description: Use gitStream's integration with AI services to generate conscious documentation of the changes in the PR.
title: Automation - Ask AI to Generate Documentation
description: Use gitStream's integration with AI services to recommend documentation for the changes in the PR.
category: [quality, genai, copilot, tests, efficiency]
starter_kits: [genai]
---
# Ask AI to provide PR documentation
# Ask AI to Generate Documentation

<!-- --8<-- [start:example]-->
Use AI to generate documentation for the newly added features and changes based on the code diff.
Use AI to recommend new or updated documentation for changes in a PR, focusing on functionality, parameters, and expected behavior.

!!! info "Configuration Description"

Conditions (all must be true):

* A PR is created, or new code has been committed to the PR.
* The PR has a label "askai document"
* A PR is created or new code has been committed to the PR.
* The PR has a label `askai document`.

Automation Actions:

* Add a comment with a summary of the PR
* Add a comment with AI-generated documentation recommendations for the PR.

!!! example "Configuration Example"
```yaml+jinja
Expand Down
16 changes: 9 additions & 7 deletions docs/automations/integrations/askAI/improve/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,23 +1,25 @@
---
title: Automation - Ask AI for code suggestions to improve the PR
description: Use gitStream's integration with AI services to generate code suggestions to improve the PR.
title: Automation - Ask AI for PR Improvements
description: Use gitStream's integration with AI services to recommend improvements to a PR.
category: [quality, genai, copilot, tests, efficiency]
starter_kits: [genai]
---
# Ask AI to provide code suggestions to improve the PR

# Ask AI for PR Improvements

<!-- --8<-- [start:example]-->
Use AI to generate suggestions for improvement in the code changes.
Use AI to recommend improvements to a PR that enhance readability, performance, security, and maintainability.

!!! info "Configuration Description"

Conditions (all must be true):

* A PR is created, or a new code has been committed to the PR.
* The PR has a label "askai improve"
* A PR is created or new code has been committed to the PR.
* The PR has a label `askai improve`.

Automation Actions:

* Add a comment with improvement suggestions for the added code
* Add a comment with AI-generated improvement suggestions for the PR.

!!! example "Configuration Example"
```yaml+jinja
Expand Down
12 changes: 6 additions & 6 deletions docs/automations/integrations/askAI/summarize-pr/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,12 +1,12 @@
---
title: Automation - Ask AI to Summarize the Changes in a PR
description: Use gitStream's integration with AI services to generate a summary of the changes in the PR.
title: Automation - Ask AI to Summarize a PR
description: Use gitStream's integration with AI services to generate a concise bullet-point summary of a PR.
category: [quality, genai, copilot, tests, efficiency]
---
# Ask AI to Summarize the Changes in a PR
# Ask AI to Summarize a PR

<!-- --8<-- [start:example]-->
Use AI to generate a concise bullet-point summary of the changes in the pull request.
Use AI to generate a concise bullet-point summary of changes in a PR.

![summarized-pr](/automations/integrations/askAI/summarize-pr/summarized-pr.png)

Expand All @@ -15,11 +15,11 @@ Use AI to generate a concise bullet-point summary of the changes in the pull req
Conditions (all must be true):

* A PR is created or new code has been committed to the PR.
* The PR has a label "askai summarize"
* The PR has a label `askai summarize`.

Automation Actions:

* Add a comment with a summary of the PR
* Add a comment with an AI-generated summary of the PR.

!!! example "Configuration Example"
```yaml+jinja
Expand Down
12 changes: 7 additions & 5 deletions docs/downloads/automation-library/integrations/askAI/askAI_CR.cm
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,9 +14,11 @@ automations:
- action: add-comment@v1
args:
comment: |
# ✨ gitStream Review ✨
{{ source | askAI("Code reviewer",
## gitStream askAI: code review checklist for this PR ✨
{{ source | askAI("experienced software developer focused on code reviews",
"Review the PR code diff.
- Identify bugs, security risks, and performance issues
- Check for deprecated methods and style guide violations
- Provide specific improvement suggestions based on the changes", env.OPEN_AI_TOKEN) }}
- Identify bugs, security risks, and performance issues.
- Check for deprecated methods and style guide violations.
- Suggest specific improvements based on the changes.
- Provide a code review checklist that can be utilized by a human reviewer.", env.OPEN_AI_TOKEN) }}
_This response was generated by the gitStream askAI plugin._
12 changes: 7 additions & 5 deletions docs/downloads/automation-library/integrations/askAI/askAI_document.cm
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,8 +14,10 @@ automations:
- action: add-comment@v1
args:
comment: |
# 🧪 Test Suggestions 🧪
{{ source | askAI("Technical writer",
"Review the code diff and generate documentation for any newly added features or significant changes.
Focus on describing functionality, parameters, and expected behavior.
Format the output in clear, concise bullet points or paragraphs.", env.OPEN_AI_TOKEN) }}
## gitStream askAI: recommended documentation for this PR 📑
{{ source | askAI("experienced technical writer focused on documentation",
"Review the PR code diff.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm concerned about putting the prompt info into the CM file, rather than incorporating it into the plugin code because this has a risk of becoming messy if we build out additional use cases this way.

Is it possible to incorporate the prompt into the plugin? I'm imagining it as a JSON file that's pulled into the code for easy updates.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@BenLloydPearson One way we could do this is using the gitStream readFile filter in a .cm config to import *.txt files from a askAI/prompts/ folder as strings. Then we could maintain the prompts there or even use multiple readFile function calls to "build" a prompt for each use case. Something like this:

automations:
  ask_ai_summarize:
    if:
      - true
    run:
      - action: add-comment@v1
        args:
          comment: |
            {{ TITLE }}
            {{ source | askAI(ROLE, PROMPT, env.OPEN_AI_TOKEN) }}

TITLE: ## gitStream askAI: summary for this PR 📜
ROLE: {{ "./.cm/plugins/filters/askAI/prompts/PR_summary_role.txt" | readFile() }}
PROMPT: {{ "./.cm/plugins/filters/askAI/prompts/PR_summary_prompt.txt" | readFile() }}

Using variables in this approach makes it so each AI automation uses the same base configuration, and you simply adjust behavior by swapping in and out prompts. I think a small folder of combinable prompts is the good way to approach this for longevity while keeping it flat and intuitive, but open to more brainstorming here.

Alternatively, we could change the plugin's .js file to load prompts this way, but feels less composable for dev adoption in the long run. @PavelLinearB probably has an opinion on a good approach.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I love this solution.

I think your naming convention for the txt files should work, but just be wary that we'll want to make sure we follow something that's consistent, descriptive, and sorts properly alphabetically (i.e. the prompt and role should appear next to each other when viewed inside the directory.) I believe your convention should check all these boxes, and we can adjust in the future if we find too many edge cases.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Using the ReadFile filter looks great imo! It will require a bit more work from users tho, but once we'll open PRs to install automations this will be super easy and neat

- Generate documentation for any newly added features or significant changes.
- Focus on describing functionality, parameters, and expected behavior.
- Format the output in clear, concise bullet points or paragraphs.", env.OPEN_AI_TOKEN) }}
_This response was generated by the gitStream askAI plugin._
10 changes: 6 additions & 4 deletions docs/downloads/automation-library/integrations/askAI/askAI_improve.cm
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,9 @@ automations:
- action: add-comment@v1
args:
comment: |
# 🧪 Test Suggestions 🧪
{{ source | askAI("Experienced developer",
"Review the code diff and suggest improvements to enhance readability, performance, and maintainability.
Focus only on the modified code and provide specific, actionable recommendations.", env.OPEN_AI_TOKEN) }}
## gitStream askAI: recommended improvements for this PR 🔨
{{ source | askAI("experienced software developer focused on code reviews",
"Review the PR code diff.
- Suggest improvements to enhance readability, performance, security, and maintainability.
- Focus only on the modified code and provide specific, actionable recommendations.", env.OPEN_AI_TOKEN) }}
_This response was generated by the gitStream askAI plugin._
7 changes: 4 additions & 3 deletions docs/downloads/automation-library/integrations/askAI/askAI_summarize.cm
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,7 @@ automations:
- action: add-comment@v1
args:
comment: |
# 📜 PR Summary 📜
{{ source | askAI("Experienced developer",
"Summarize the changes in this PR in bullet points.", env.OPEN_AI_TOKEN) }}
## gitStream askAI: summary for this PR 📜
{{ source | askAI("experienced software developer focused on code reviews",
"Review the PR code diff.
- Summarize the changes in the PR in an unordered list.", env.OPEN_AI_TOKEN) }}
12 changes: 7 additions & 5 deletions docs/downloads/automation-library/integrations/askAI/askAI_tests.cm
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,8 +14,10 @@ automations:
- action: add-comment@v1
args:
comment: |
# 🧪 Test Suggestions 🧪
{{ source | askAI("QA tester",
"Identify any new or modified functions without test coverage in this PR.
Suggest specific test cases to add, including edge cases.
If all functions are covered, return 'No additional tests needed.'", env.OPEN_AI_TOKEN) }}
## gitStream askAI: recommended tests for this PR 🧪
{{ source | askAI("experienced software developer focused on QA testing",
"Review the PR code diff.
- Identify any new or modified functions without test coverage.
- Suggest specific test cases to add, including edge cases.
- If all functions are covered, return 'No additional tests needed.'", env.OPEN_AI_TOKEN) }}
_This response was generated by the gitStream askAI plugin._
7 changes: 4 additions & 3 deletions docs/faq.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,11 +14,12 @@ Choose which repositories are permitted to use GitHub Actions.

[x] Allow all actions and reusable workflows

## Does gitStream services have access to my code?
## Does gitStream have access to my code?

Like any other CI/CD automation, the source code is being scanned in the repo and is not shared with any external services. Only metadata related to and affecting the workflow is shared to allow rule-based automation on the repo.
Like any other CI/CD automation, the source code is being scanned in the repo and is not shared with any external services. By default, only metadata related to and affecting the workflow is shared to allow rule-based automation on the repo. Your own gitStream plugins that may connect to other services, such as using the [`askAI`](/filter-function-plugins/#askai) plugin which will provide context to the configured model provider.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we should also add that the user fully controls the data that is shared with LinearB or any other service as they have the control over the arguments for each filter


## Why does gitStream require permission to write code?

To support automations that either Approve or Merge PRs, the git providers require code write scope.

## What repos are supported?
Expand All @@ -34,7 +35,7 @@ Yes. When a merge queue is used, and gitStream is set as a required check, gitSt

The `.cm` file uses YAML with JINJA2. For your favorite editor to automatically choose the right syntax, you can use modelines.

Add the following line to the top of the `.cm` file (the default has it already):
Add the following line to the top of the `.cm` file (the default has it already):

```
# -*- mode: yaml -*-
Expand Down
26 changes: 11 additions & 15 deletions docs/integrations/askAI.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,43 +1,39 @@
---
title: Integrate gitStream with AI
description: Use gitStream to integrate with AI services for Review, describe and add tests.
description: Use gitStream to integrate with AI for different use cases.
category: [quality, genai, copilot, tests, efficiency]
---
# Integrate gitStream with AI

<!-- --8<-- [start:examples]-->
!!! warning "Required gitStream Plugins"
This example requires you to install the [`askAI`](/filter-function-plugins/#askai) plugin.
These examples requires you to install the [`askAI`](/filter-function-plugins/#askai) plugin, which will provide context to the configured model provider and may incur API costs.

[Learn more about gitStream plugins](/plugins/).

## Ask AI to Summarize the Changes in a PR
## Ask AI to Summarize a PR

--8<-- "docs/automations/integrations/askAI/summarize-pr/README.md:example"

## Ask AI to Suggest Tests

--8<-- "docs/automations/integrations/askAI/add-tests/README.md:example"

## Ask AI for a Code Review
## Ask AI for a PR Review Checklist

--8<-- "docs/automations/integrations/askAI/code-review/README.md:example"

## Ask AI for Documentation
## Ask AI for PR Improvements

--8<-- "docs/automations/integrations/askAI/document/README.md:example"
--8<-- "docs/automations/integrations/askAI/improve/README.md:example"

## Ask AI for Code Improvements
## Ask AI to Generate Documentation

--8<-- "docs/automations/integrations/askAI/improve/README.md:example"
--8<-- "docs/automations/integrations/askAI/document/README.md:example"

<!-- ## Ask AI for anything
## Ask AI to Generate Tests
azigler marked this conversation as resolved.
Show resolved Hide resolved

--8<-- "docs/automations/integrations/askAI/code-review/README.md:example" -->
--8<-- "docs/automations/integrations/askAI/add-tests/README.md:example"
<!-- --8<-- [end:examples]-->

## Additional Resources

--8<-- "docs/snippets/general.md"

--8<-- "docs/snippets/automation-footer.md"
--8<-- "docs/snippets/automation-footer.md"
14 changes: 7 additions & 7 deletions plugins/filters/askAI/index.js
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,12 +1,12 @@
/**
* @module askAI
* @description A gitStream plugin to interact with AI models. Currently works with `ChatGPR-4o-mini`.
* @param {Object} context - The context that will be attached to the prompt .
* @param {string} role - Role instructions for the conversation.
* @param {string} prompt - The prompt string.
* @param {Object} token - The token to the AI model.
* @returns {Object} Returns the response from the AI model.
* @example {{ branch | generateDescription(pr, repo, source) }}
* @description A gitStream plugin to facilitate AI workflows with OpenAI's `gpt-4o-2024-08-06` model.
PavelLinearB marked this conversation as resolved.
Show resolved Hide resolved
* @param {Object} context - The context to be sent to the AI model with the prompt.
* @param {string} role - The system role or persona for the AI to adopt while generating the response.
* @param {string} prompt - The specific request or question you want the AI to respond to, after the context has been provided.
* @param {string} token - Your OpenAI API token.
* @returns {Object} Returns the AI-generated response based on the provided context and prompt.
* @example {{ source | askAI("Experienced developer", "Summarize the changes in this PR in bullet points.", env.OPEN_AI_TOKEN) }}
PavelLinearB marked this conversation as resolved.
Show resolved Hide resolved
* @license MIT
* */

Expand Down
12 changes: 6 additions & 6 deletions plugins/filters/askAI/reference.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
<a name="module_generateDescription"></a>

## askAI

The AskAI plugin allows gitStream workflows to interact with external AI services, enabling advanced automation capabilities such as code analysis, test case generation, and PR summarization. This plugin requires a valid API token for the AI service, which must be securely provided as an environment variable.

!!! note "Security note"
Expand All @@ -15,14 +16,13 @@ The AskAI plugin allows gitStream workflows to interact with external AI service

| Param | Type | Description |
| ------- | -------- | ------------------------------------------------------------------------------------------------------------------ |
| context | `Object` | The context that needs to be sent to the AI model for analysis. |
| role | `string` | Free text. If not empty, Defines the role or persona for the AI to adopt while generating the response. |
| context | `Object` | The context to be sent to the AI model with the prompt. |
| role | `string` | The system role or persona for the AI to adopt while generating the response. |
| prompt | `string` | The specific request or question you want the AI to respond to, after the context has been provided. |
| token | `Object` | The token to the AI model |

| token | `string` | Your OpenAI API token. |

**Example**

```yaml
{{ source | askAI("QA tester", "Based on the given context, search for new functions without tests and suggest the tests to add. If all functions are covered completely, return 'no tests to suggest.'", env.OPEN_AI_TOKEN) }}
{{ source | askAI("Experienced developer", "Summarize the changes in this PR in bullet points.", env.OPEN_AI_TOKEN) }}
```