Skip to content

Commit

Permalink
[doc] modify the code for output formatter schema (#2614)
Browse files Browse the repository at this point in the history
  • Loading branch information
sindhuvahinis authored Dec 9, 2024
1 parent 9fc5ad3 commit fdc8722
Showing 1 changed file with 5 additions and 2 deletions.
7 changes: 5 additions & 2 deletions serving/docs/lmi/user_guides/output_formatter_schema.md
Original file line number Diff line number Diff line change
Expand Up @@ -85,7 +85,8 @@ It's crucial to understand how your custom output formatter will be called befor
## Example
Here is an example of a custom output formatter:
```python
from djl_python.output_formatter import TextGenerationOutput, output_formatter
from djl_python.request_io import TextGenerationOutput
from djl_python.output_formatter import output_formatter
import json

@output_formatter
Expand All @@ -102,7 +103,9 @@ def custom_output_formatter(request_output: TextGenerationOutput) -> str:
"""
best_sequence = request_output.sequences[request_output.best_sequence_index]
next_token, is_first_token, is_last_token = best_sequence.get_next_token()
result = {"token_id": next_token.id, "token_text": next_token.text, "token_log_prob": next_token.log_prob}
result = {}
if next_token:
result = {"token_id": next_token.id, "token_text": next_token.text, "token_log_prob": next_token.log_prob}
if is_last_token:
result["finish_reason"] = best_sequence.finish_reason
return json.dumps(result) + "\n"
Expand Down

0 comments on commit fdc8722

Please sign in to comment.