-
Notifications
You must be signed in to change notification settings - Fork 11
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
refactor: Content filter helper function #441
base: main
Are you sure you want to change the base?
Changes from all commits
e21ec35
6f31078
1b19566
6d3b410
c9008cf
1397cc1
80fbf99
b17f12b
4ee2f12
42bd8b5
23cd5aa
6e20a48
a5fee4b
07ae577
b81fd07
4dca1b9
eee41e0
d58970e
77af75e
220f395
37dc504
8a4abb3
58e4a2d
e86fb1f
ce1ac5a
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,6 @@ | ||
--- | ||
'@sap-ai-sdk/orchestration': minor | ||
--- | ||
|
||
[Compatibility Note] Deprecate `buildAzureContentFilter()` function since it restricts filtering to have only one filter. | ||
Use `buildAzureContentSafetyFilter()` function instead. |
Original file line number | Diff line number | Diff line change | ||||
---|---|---|---|---|---|---|
|
@@ -245,24 +245,35 @@ | |||||
|
||||||
This feature allows filtering both the [input](https://help.sap.com/docs/sap-ai-core/sap-ai-core-service-guide/input-filtering) and [output](https://help.sap.com/docs/sap-ai-core/sap-ai-core-service-guide/output-filtering) of a model based on content safety criteria. | ||||||
|
||||||
```ts | ||||||
import { | ||||||
OrchestrationClient, | ||||||
buildAzureContentFilter | ||||||
} from '@sap-ai-sdk/orchestration'; | ||||||
#### Azure Content Filter | ||||||
|
||||||
Use `buildAzureContentSafetyFilter()` function to build an Azure content filter. | ||||||
The Azure content filter supports four categories: `Hate`, `Violence`, `Sexual`, and `SelfHarm`. | ||||||
Each category can be configured with severity levels of 0, 2, 4, or 6. | ||||||
|
||||||
Here is a complete example of using an Azure content filter for both input and output: | ||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. [q] Is this line even needed? I feel like we're way to verbose sometimes when things are self-explanatory. We can just say (in the line above): Use |
||||||
|
||||||
const filter = buildAzureContentFilter({ Hate: 2, Violence: 4 }); | ||||||
```ts | ||||||
import { OrchestrationClient, ContentFilters } from '@sap-ai-sdk/orchestration'; | ||||||
const llm = { | ||||||
model_name: 'gpt-4o', | ||||||
model_params: { max_tokens: 50, temperature: 0.1 } | ||||||
}; | ||||||
const templating = { | ||||||
template: [{ role: 'user', content: '{{?input}}' }] | ||||||
}; | ||||||
|
||||||
const filter = buildAzureContentSafetyFilter({ Hate: 2, Violence: 4 }); | ||||||
const orchestrationClient = new OrchestrationClient({ | ||||||
llm: { | ||||||
model_name: 'gpt-4o', | ||||||
model_params: { max_tokens: 50, temperature: 0.1 } | ||||||
}, | ||||||
templating: { | ||||||
template: [{ role: 'user', content: '{{?input}}' }] | ||||||
}, | ||||||
llm, | ||||||
templating, | ||||||
filtering: { | ||||||
input: filter, | ||||||
output: filter | ||||||
input: { | ||||||
filters: [filter] | ||||||
}, | ||||||
output: { | ||||||
filters: [filter] | ||||||
} | ||||||
} | ||||||
}); | ||||||
|
||||||
|
@@ -276,23 +287,19 @@ | |||||
} | ||||||
``` | ||||||
|
||||||
#### Error Handling | ||||||
|
||||||
Both `chatCompletion()` and `getContent()` methods can throw errors. | ||||||
|
||||||
- **axios errors**: | ||||||
- **Axios Errors**: | ||||||
Check warning on line 294 in packages/orchestration/README.md GitHub Actions / grammar-check
|
||||||
When the chat completion request fails with a `400` status code, the caught error will be an `Axios` error. | ||||||
The property `error.response.data.message` may provide additional details about the failure's cause. | ||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Suggested change
|
||||||
|
||||||
- **output content filtered**: | ||||||
The method `getContent()` can throw an error if the output filter filters the model output. | ||||||
- **Output Content Filtered**: | ||||||
The `getContent()` method can throw an error if the output filter filters the model output. | ||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. [q] can or does? |
||||||
This can occur even if the chat completion request responds with a `200` HTTP status code. | ||||||
The `error.message` property indicates if the output was filtered. | ||||||
|
||||||
Therefore, handle errors appropriately to ensure meaningful feedback for both types of errors. | ||||||
|
||||||
`buildAzureContentFilter()` is a convenience function that creates an Azure content filter configuration based on the provided inputs. | ||||||
The Azure content filter supports four categories: `Hate`, `Violence`, `Sexual`, and `SelfHarm`. | ||||||
Each category can be configured with severity levels of 0, 2, 4, or 6. | ||||||
|
||||||
### Data Masking | ||||||
|
||||||
You can anonymize or pseudonomize the prompt using the data masking capabilities of the orchestration service. | ||||||
|
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,4 +1,4 @@ | ||
export * from './orchestration-client.js'; | ||
export * from './orchestration-utils.js'; | ||
export * from './util/index.js'; | ||
export * from './orchestration-types.js'; | ||
export * from './orchestration-response.js'; |
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -63,6 +63,22 @@ export interface OrchestrationModuleConfig { | |
llm: LlmModuleConfig; | ||
/** | ||
* Filtering module configuration. | ||
* @example | ||
* Configuring an input and output filter: | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. [pp] I feel this line is not need (it just repeats what is visible from the code) without giving more info |
||
* ```ts | ||
* filtering: { | ||
* input: { | ||
* filters: [ | ||
* buildAzureContentSafetyFilter({ Hate: 0, Violence: 0 }) | ||
* ] | ||
* }, | ||
* output: { | ||
* filters: [ | ||
* buildAzureContentSafetyFilter({ Hate: 0, Violence: 0 }) | ||
* ] | ||
* } | ||
* } | ||
* ``` | ||
*/ | ||
filtering?: FilteringModuleConfig; | ||
/** | ||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
[q] Are we not updating this?