Code

Sotopia Benchmark CLI API#69

Code

Merged

ProKil merged 48 commits intomainsotopia-lab/sotopia:mainfrom feature/benchmark_agents.pysotopia-lab/sotopia:feature/benchmark_agents.pyCopy head branch name to clipboardJun 17, 2024

+866-476

Commits

Commits on May 16, 2024

• add benchmark social agents

  XuhuiZhou

  committed
  d0811b6

  Copy full SHA for d0811b6

• add benchmark agents

  XuhuiZhou

  committed
  51a9132

  Copy full SHA for 51a9132

• Add sotopia_benchmark cli api

  ProKil

  committed
  b576be7

  Copy full SHA for b576be7

• fix pre-commit

  ProKil

  committed
  7922e16

  Copy full SHA for 7922e16

• add evaluator model argument

  ProKil

  committed
  9c13c02

  Copy full SHA for 9c13c02

Commits on May 23, 2024

• Merge branch 'main' into feature/benchmark_agents.py

  XuhuiZhou

  committed
  165c245

  Copy full SHA for 165c245

• finish benchmarking

  XuhuiZhou

  committed
  27aa565

  Copy full SHA for 27aa565

Commits on May 24, 2024

• benchmark done

  XuhuiZhou

  committed
  b625cfc

  Copy full SHA for b625cfc

Commits on May 25, 2024

• chore: Fix formatting issue in redis_stats.ipynb and cli.py

  XuhuiZhou

  committed
  7066a2a

  Copy full SHA for 7066a2a

• switch back to LLM_Name

  XuhuiZhou

  committed
  b710ea6

  Copy full SHA for b710ea6

Commits on May 26, 2024

• Merge branch 'main' into feature/benchmark_agents.py

  XuhuiZhou

  committed
  385a53c

  Copy full SHA for 385a53c

• merge main

  XuhuiZhou

  committed
  ad9e741

  Copy full SHA for ad9e741

• add together ai

  XuhuiZhou

  committed
  ee87f84

  Copy full SHA for ee87f84

• fix naming error

  XuhuiZhou

  committed
  41ad0e3

  Copy full SHA for 41ad0e3

• roll back to llama2

  XuhuiZhou

  committed
  5ae2b26

  Copy full SHA for 5ae2b26

• chore: Update langchain-together dependency to version 0.1.2

  XuhuiZhou

  committed
  af4e2ee

  Copy full SHA for af4e2ee

• use chatopenai for together models

  ProKil

  committed
  5361d0c

  Copy full SHA for 5361d0c

• Merge branch 'feature/benchmark_agents.py' of github.com:sotopia-lab/sotopia into feature/benchmark_agents.py

  ProKil

  committed
  35be2a2

  Copy full SHA for 35be2a2

• add logging

  ProKil

  committed
  7087f40

  Copy full SHA for 7087f40

• Merge remote-tracking branch 'origin/main' into feature/benchmark_agents.py

  ProKil

  committed
  01b6291

  Copy full SHA for 01b6291

• fix pre-commit

  ProKil

  committed
  4be892d

  Copy full SHA for 4be892d

Commits on May 27, 2024

• add more logging options

  ProKil

  committed
  e8a59d2

  Copy full SHA for e8a59d2

• probably fix the event loop closed error: following NVIDIA/NeMo-Guardrails#336

  ProKil

  committed
  9e15790

  Copy full SHA for 9e15790

Commits on May 28, 2024

• modify cli; fix model position bug

  XuhuiZhou

  committed
  4906efe

  Copy full SHA for 4906efe

Commits on May 29, 2024

• chore: Update benchmark tag to "benchmark_{model}_final"

  XuhuiZhou

  committed
  0026b3e

  Copy full SHA for 0026b3e

• Refactor _iterate_all_env_agent_combo_not_in_db function

  XuhuiZhou

  committed
  1d873be

  Copy full SHA for 1d873be

• chore: Update python version to 3.11.2

  XuhuiZhou

  committed
  cb49079

  Copy full SHA for cb49079

Commits on Jun 1, 2024

• change to dict comparison

  XuhuiZhou

  committed
  f983b44

  Copy full SHA for f983b44

Commits on Jun 4, 2024

• ignore jsonl

  XuhuiZhou

  committed
  dbc873c

  Copy full SHA for dbc873c

Commits on Jun 5, 2024

• ✨ finish benchmarking script

  XuhuiZhou

  committed
  6a79d7e

  Copy full SHA for 6a79d7e

• chore: Refactor server.py and redis_stats.ipynb

  XuhuiZhou

  committed
  9a38b93

  Copy full SHA for 9a38b93

• add type ignore

  XuhuiZhou

  committed
  0996090

  Copy full SHA for 0996090

• push for the eval

  XuhuiZhou

  committed
  f1a626b

  Copy full SHA for f1a626b

• Refactor run_async_benchmark_in_batch function

  XuhuiZhou

  committed
  b567224

  Copy full SHA for b567224

• Refactor run_async_benchmark_in_batch function

  XuhuiZhou

  committed
  bfadacd

  Copy full SHA for bfadacd

• add doc

  XuhuiZhou

  committed
  9a5dce9

  Copy full SHA for 9a5dce9

• precommit fix

  XuhuiZhou

  committed
  3dbde75

  Copy full SHA for 3dbde75

• pre-commit

  XuhuiZhou

  committed
  897389a

  Copy full SHA for 897389a

Commits on Jun 6, 2024

• Merge remote-tracking branch 'origin/main' into feature/benchmark_agents.py

  ProKil

  committed
  1c6b2b9

  Copy full SHA for 1c6b2b9

Commits on Jun 9, 2024

• refactor

  XuhuiZhou

  committed
  b7fd148

  Copy full SHA for b7fd148

• Merge branch 'feature/benchmark_agents.py' of github.com:sotopia-lab/sotopia into feature/benchmark_agents.py

  XuhuiZhou

  committed
  73c2c92

  Copy full SHA for 73c2c92

Commits on Jun 14, 2024

• Merge branch 'main' into feature/benchmark_agents.py

  Show description for df1bfde
  XuhuiZhou

  committed
  df1bfde

  Copy full SHA for df1bfde

• update w feedback

  XuhuiZhou

  committed
  8352ea7

  Copy full SHA for 8352ea7

• pre commit

  XuhuiZhou

  committed
  f6c86cb

  Copy full SHA for f6c86cb

• chore: Update authors in pyproject.toml and fetch benchmark_agents.json from Hugging Face API

  XuhuiZhou

  committed
  6bcb178

  Copy full SHA for 6bcb178

Commits on Jun 15, 2024

• hotfix

  XuhuiZhou

  committed
  f5c269c

  Copy full SHA for f5c269c

• Merge branch 'main' into feature/benchmark_agents.py

  XuhuiZhou

  committed
  98fe351

  Copy full SHA for 98fe351

• chore: Remove unnecessary type hint in benchmark/cli.py

  XuhuiZhou

  committed
  7165109

  Copy full SHA for 7165109