Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

refactor: Content filter helper function #441

Open
wants to merge 25 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
25 commits
Select commit Hold shift + click to select a range
e21ec35
refactor: implemement ContentFilters.azure() and deprecate buildAzure…
ZhongpinWang Jan 10, 2025
6f31078
fix: Changes from lint
Jan 10, 2025
1b19566
chore: changeset
ZhongpinWang Jan 10, 2025
6d3b410
fix: vale
ZhongpinWang Jan 10, 2025
c9008cf
chore: update README
ZhongpinWang Jan 10, 2025
1397cc1
Merge branch 'main' into refactor-content-filters
ZhongpinWang Jan 20, 2025
80fbf99
review
ZhongpinWang Jan 20, 2025
b17f12b
fix: Changes from lint
Jan 20, 2025
4ee2f12
Merge branch 'main' into refactor-content-filters
ZhongpinWang Jan 20, 2025
42bd8b5
refactor: rename azure into azureContentSafety
ZhongpinWang Jan 20, 2025
23cd5aa
fix: Changes from lint
Jan 20, 2025
6e20a48
Merge branch 'main' into refactor-content-filters
KavithaSiva Jan 21, 2025
a5fee4b
Merge branch 'main' into refactor-content-filters
ZhongpinWang Jan 23, 2025
07ae577
chore: update changeset
ZhongpinWang Jan 23, 2025
b81fd07
refactor: rename azureContentSafety into buildAzureContentSafety
ZhongpinWang Jan 23, 2025
4dca1b9
fix: Changes from lint
Jan 23, 2025
eee41e0
fix: test
ZhongpinWang Jan 23, 2025
d58970e
fix: Changes from lint
Jan 23, 2025
77af75e
chore: revert behaviour of buildAzureContentFilter
KavithaSiva Jan 24, 2025
220f395
Merge branch 'main' into refactor-content-filters
KavithaSiva Jan 30, 2025
37dc504
Merge branch 'main' into refactor-content-filters
KavithaSiva Jan 31, 2025
8a4abb3
chore: changes from discussion
KavithaSiva Jan 31, 2025
58e4a2d
chore: after discussion updates
KavithaSiva Jan 31, 2025
e86fb1f
chore: fix test
KavithaSiva Jan 31, 2025
ce1ac5a
chore: adjust documentation
KavithaSiva Jan 31, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions .changeset/afraid-cooks-shave.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
---
'@sap-ai-sdk/orchestration': minor
---

[Compatibility Note] Deprecate `buildAzureContentFilter()` function since it restricts filtering to have only one filter.
Use `buildAzureContentSafetyFilter()` function instead.
55 changes: 31 additions & 24 deletions packages/orchestration/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -245,24 +245,35 @@

This feature allows filtering both the [input](https://help.sap.com/docs/sap-ai-core/sap-ai-core-service-guide/input-filtering) and [output](https://help.sap.com/docs/sap-ai-core/sap-ai-core-service-guide/output-filtering) of a model based on content safety criteria.

```ts
import {
OrchestrationClient,
buildAzureContentFilter
} from '@sap-ai-sdk/orchestration';
#### Azure Content Filter

Use `buildAzureContentSafetyFilter()` function to build an Azure content filter.
The Azure content filter supports four categories: `Hate`, `Violence`, `Sexual`, and `SelfHarm`.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[q] Are we not updating this?

Each category can be configured with severity levels of 0, 2, 4, or 6.

Here is a complete example of using an Azure content filter for both input and output:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[q] Is this line even needed? I feel like we're way to verbose sometimes when things are self-explanatory. We can just say (in the line above):

Use buildAzureContentSafetyFilter() function to build an Azure content filter for both input and output.


const filter = buildAzureContentFilter({ Hate: 2, Violence: 4 });
```ts
import { OrchestrationClient, ContentFilters } from '@sap-ai-sdk/orchestration';
const llm = {
model_name: 'gpt-4o',
model_params: { max_tokens: 50, temperature: 0.1 }
};
const templating = {
template: [{ role: 'user', content: '{{?input}}' }]
};

const filter = buildAzureContentSafetyFilter({ Hate: 2, Violence: 4 });
const orchestrationClient = new OrchestrationClient({
llm: {
model_name: 'gpt-4o',
model_params: { max_tokens: 50, temperature: 0.1 }
},
templating: {
template: [{ role: 'user', content: '{{?input}}' }]
},
llm,
templating,
filtering: {
input: filter,
output: filter
input: {
filters: [filter]
},
output: {
filters: [filter]
}
}
});

Expand All @@ -276,23 +287,19 @@
}
```

#### Error Handling

Both `chatCompletion()` and `getContent()` methods can throw errors.

- **axios errors**:
- **Axios Errors**:

Check warning on line 294 in packages/orchestration/README.md

View workflow job for this annotation

GitHub Actions / grammar-check

[vale] reported by reviewdog 🐶 [Vale.Terms] Use 'axios' instead of 'Axios'. Raw Output: {"message": "[Vale.Terms] Use 'axios' instead of 'Axios'.", "location": {"path": "packages/orchestration/README.md", "range": {"start": {"line": 294, "column": 5}}}, "severity": "WARNING"}
When the chat completion request fails with a `400` status code, the caught error will be an `Axios` error.
The property `error.response.data.message` may provide additional details about the failure's cause.
Copy link
Contributor

@deekshas8 deekshas8 Jan 31, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
The property `error.response.data.message` may provide additional details about the failure's cause.
The property `error.response.data.message` provides additional details about the failure.


- **output content filtered**:
The method `getContent()` can throw an error if the output filter filters the model output.
- **Output Content Filtered**:
The `getContent()` method can throw an error if the output filter filters the model output.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[q] can or does?

This can occur even if the chat completion request responds with a `200` HTTP status code.
The `error.message` property indicates if the output was filtered.

Therefore, handle errors appropriately to ensure meaningful feedback for both types of errors.

`buildAzureContentFilter()` is a convenience function that creates an Azure content filter configuration based on the provided inputs.
The Azure content filter supports four categories: `Hate`, `Violence`, `Sexual`, and `SelfHarm`.
Each category can be configured with severity levels of 0, 2, 4, or 6.

### Data Masking

You can anonymize or pseudonomize the prompt using the data masking capabilities of the orchestration service.
Expand Down
3 changes: 2 additions & 1 deletion packages/orchestration/src/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -21,8 +21,9 @@ export { OrchestrationClient } from './orchestration-client.js';

export {
buildAzureContentFilter,
buildAzureContentSafetyFilter,
buildDocumentGroundingConfig
} from './orchestration-utils.js';
} from './util/index.js';

export { OrchestrationResponse } from './orchestration-response.js';

Expand Down
2 changes: 1 addition & 1 deletion packages/orchestration/src/internal.ts
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
export * from './orchestration-client.js';
export * from './orchestration-utils.js';
export * from './util/index.js';
export * from './orchestration-types.js';
export * from './orchestration-response.js';
16 changes: 10 additions & 6 deletions packages/orchestration/src/orchestration-client.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -9,12 +9,12 @@ import {
parseMockResponse
} from '../../../test-util/mock-http.js';
import { OrchestrationClient } from './orchestration-client.js';
import { OrchestrationResponse } from './orchestration-response.js';
import {
buildAzureContentFilter,
constructCompletionPostRequestFromJsonModuleConfig,
constructCompletionPostRequest,
constructCompletionPostRequestFromJsonModuleConfig
} from './orchestration-utils.js';
import { OrchestrationResponse } from './orchestration-response.js';
buildAzureContentSafetyFilter
} from './util/index.js';
import type { CompletionPostResponse } from './client/api/schema/index.js';
import type {
OrchestrationModuleConfig,
Expand Down Expand Up @@ -162,8 +162,12 @@ describe('orchestration service client', () => {
]
},
filtering: {
input: buildAzureContentFilter({ Hate: 4, SelfHarm: 2 }),
output: buildAzureContentFilter({ Sexual: 0, Violence: 4 })
input: {
filters: [buildAzureContentSafetyFilter({ Hate: 4, SelfHarm: 2 })]
},
output: {
filters: [buildAzureContentSafetyFilter({ Sexual: 0, Violence: 4 })]
}
}
};
const prompt = {
Expand Down
2 changes: 1 addition & 1 deletion packages/orchestration/src/orchestration-client.ts
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ import { OrchestrationResponse } from './orchestration-response.js';
import {
constructCompletionPostRequest,
constructCompletionPostRequestFromJsonModuleConfig
} from './orchestration-utils.js';
} from './util/index.js';
import type {
HttpResponse,
CustomRequestConfig
Expand Down
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
import { constructCompletionPostRequestFromJsonModuleConfig } from './orchestration-utils.js';
import { constructCompletionPostRequestFromJsonModuleConfig } from './util/request-config.js';

describe('construct completion post request from JSON', () => {
it('should construct completion post request from JSON', () => {
Expand Down
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
import {
constructCompletionPostRequest,
buildAzureContentFilter
} from './orchestration-utils.js';
buildAzureContentSafetyFilter
} from './util/index.js';
import type { CompletionPostRequest } from './client/api/schema/index.js';
import type {
OrchestrationModuleConfig,
Expand Down Expand Up @@ -169,7 +169,9 @@ describe('construct completion post request', () => {
const config: OrchestrationModuleConfig = {
...defaultConfig,
filtering: {
input: buildAzureContentFilter({ Hate: 4, SelfHarm: 0 })
input: {
filters: [buildAzureContentSafetyFilter({ Hate: 4, SelfHarm: 0 })]
}
}
};
const expectedCompletionPostRequest: CompletionPostRequest = {
Expand Down Expand Up @@ -209,7 +211,9 @@ describe('construct completion post request', () => {
const config: OrchestrationModuleConfig = {
...defaultConfig,
filtering: {
output: buildAzureContentFilter({ Hate: 4, SelfHarm: 0 })
output: {
filters: [buildAzureContentSafetyFilter({ Hate: 4, SelfHarm: 0 })]
}
}
};

Expand Down
16 changes: 16 additions & 0 deletions packages/orchestration/src/orchestration-types.ts
Original file line number Diff line number Diff line change
Expand Up @@ -63,6 +63,22 @@ export interface OrchestrationModuleConfig {
llm: LlmModuleConfig;
/**
* Filtering module configuration.
* @example
* Configuring an input and output filter:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[pp] I feel this line is not need (it just repeats what is visible from the code) without giving more info

* ```ts
* filtering: {
* input: {
* filters: [
* buildAzureContentSafetyFilter({ Hate: 0, Violence: 0 })
* ]
* },
* output: {
* filters: [
* buildAzureContentSafetyFilter({ Hate: 0, Violence: 0 })
* ]
* }
* }
* ```
*/
filtering?: FilteringModuleConfig;
/**
Expand Down
Loading
Loading