-
Notifications
You must be signed in to change notification settings - Fork 23
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feat: enabled saving and evaluation for moderator #271
Conversation
@@ -0,0 +1,25 @@ | |||
from abc import ABC, abstractmethod |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
why not using the existing evaluators
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The existing evaluators base class can only take in message history. However, there might be cases where we need other information in EpisodeLog to generate an evaluation. So I made a new class of evaluator for future use.
feat: enabled saving and evaluation for moderator (#271)
Closes #
📑 Description
added two new features for moderator:
saving
evaluation
✅ Checks
type/descript
(e.g.feature/add-llm-agents
)ℹ Additional Information