Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(chat): increase max headers size (Issue #2852) #2871

Open
wants to merge 12 commits into
base: development
Choose a base branch
from
1 change: 1 addition & 0 deletions apps/chat/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -131,6 +131,7 @@ AI DIAL Chat uses environment variables for configuration. All environment varia
| `SHOW_TOKEN_SUB` | No | Show token sub in refresh login error logs | `true`, `false` | `false` |
| `STORAGE_TYPE` | No | Type of storage used for getting and saving information generated by a user.<br />**Note**: currently, `api` is the only supported value. | `browserStorage`, `api` | `api` |
| `KEEP_ALIVE_TIMEOUT` | No | Determines the maximum time in milliseconds for an idle connection before it is closed by the server.<br />This is needed because infrastructures usually have a default keep alive timeout 60 seconds and the Next server should have a larger value.<br />**Note**: this variable is used only when running a `dockerfile`. | Any number string | `61000` |
| `MAX_HTTP_HEADERS_SIZE` | No | Defines the maximum size of headers for Next.js server. <br />**Note**: this variable is used only when running a `dockerfile`. | `32768` |
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Need to check that for our environments (local, Review, Staging) also is used default limit with 32 KB

| `MAX_PROMPT_TOKENS_DEFAULT_PERCENT` | No | A share in percent of `MAX_PROMPT_TOKENS_DEFAULT_VALUE`.<br />If not set, the value of `MAX_PROMPT_TOKENS_DEFAULT_VALUE` applies. | Any string | `75` |
| `MAX_PROMPT_TOKENS_DEFAULT_VALUE` | No | The default maximum limit of tokens permissible for any model.<br />Applies if a limit is not defined in the [AI DIAL Core](https://github.com/epam/ai-dial-core?tab=readme-ov-file#dynamic-settings) configuration. | Any string | `2000` |
| `CUSTOM_VISUALIZERS` | No | A list of configurations of custom visualizers.<br />Refer to [Chat Visualizer](../../libs/chat-visualizer-connector/README.md) to learn more. | [{ title, description, icon, contentType, url }] | |
Expand Down
2 changes: 1 addition & 1 deletion package.json
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
"version": "0.23.0-rc",
"private": true,
"scripts": {
"nx": "nx",
"nx": "NODE_OPTIONS='--max-http-header-size=32768' nx",
"postinstall": "patch-package && node tools/patch-nextjs.js",
"graph": "nx graph",
"build": "nx run-many -t build",
Expand Down
5 changes: 4 additions & 1 deletion startup.sh
Original file line number Diff line number Diff line change
@@ -1,5 +1,8 @@
#!/bin/sh

KEEP_ALIVE_TIMEOUT="${KEEP_ALIVE_TIMEOUT:-61000}"
MAX_HTTP_HEADERS_SIZE="${MAX_HTTP_HEADERS_SIZE:-32768}"

exec npm start -- --keepAliveTimeout $KEEP_ALIVE_TIMEOUT
export NODE_OPTIONS="--max-http-header-size=$MAX_HTTP_HEADERS_SIZE ${NODE_OPTIONS}"

exec npm start -- --keepAliveTimeout $KEEP_ALIVE_TIMEOUT
Loading