Skip to content

Issues: open-compass/opencompass

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

240 Open 352 Closed
240 Open 352 Closed
Author
Filter by author
Loading
Label
Filter by label
Loading
Use alt + click/return to exclude labels
or ⇧ + click/return for logical OR
Projects
Filter by project
Loading
Milestones
Filter by milestone
Loading
Assignee
Filter by who’s assigned
Sort
Sort by
Newest Oldest Most commented Least commented Recently updated Least recently updated Best match
Most reactions
👍 👎 😄 🎉 😕 ❤️ 🚀 👀

Issues list

[Bug] 使用leval测评时报错，求助
#1825 opened Jan 15, 2025 by fxnie
2 tasks done
@MaiziXiao
[Bug] longbench报错
#1824 opened Jan 15, 2025 by fxnie
2 tasks done
@MaiziXiao
1
[Bug] lazy module import error
#1821 opened Jan 15, 2025 by berlino
2 tasks done
@MaiziXiao
[Feature] 请问是否能有一个汇总，就是opencompass包含的所有评测数据集、这些数据集的metrics和链接(paper/github)？
#1820 opened Jan 14, 2025 by 141forever
1 task
@liushz
1
[Bug] 部分样本上opencompass的评测方法是否可能有点片面？
#1818 opened Jan 14, 2025 by laogonggong847
2 tasks done
@liushz
[Bug] Unable to load model offline
#1817 opened Jan 13, 2025 by mono-arch
2 tasks done
@bittersweet1999
3
[Bug] SuperGLUE_RTE_gen KeyError
#1816 opened Jan 10, 2025 by shedding-ash
2 tasks done
@tonysy
[Bug] 使用vllm加速时无法启动多卡
#1810 opened Jan 8, 2025 by Lichunyan3
2 tasks done
1
[Bug] 缺乏game24以及其他数据集
#1807 opened Jan 7, 2025 by nuochenpku
2 tasks done
@liushz
1
[Feature] 支持基于ray的多卡多机评测
#1804 opened Jan 4, 2025 by wenxueru
1 task
@tonysy
1
[Feature] 如何添加评价指标
#1803 opened Jan 3, 2025 by Lichunyan3
1 task
@tonysy
2
[Bug] SuperGLUE_BoolQ_ppl Unable to load file
#1799 opened Jan 2, 2025 by shutttttdown
2 tasks done
@acylam
1
[Bug] gsm8k_gen_17d0dc-BOT思考提示语错误
#1788 opened Dec 27, 2024 by TBoyBack
2 tasks done
@liushz
2
[Bug] Please make sure ./data/safety.txt is correct
#1787 opened Dec 27, 2024 by wccccp
2 tasks done
[Feature] 是否打算支持mamba或者mamba2的测评吗
#1778 opened Dec 25, 2024 by fxnie
1 task done
@MaiziXiao
2
[Bug] torch.OutOfMemoryError
#1773 opened Dec 23, 2024 by QingChengLineOne
2 tasks done
@bittersweet1999
3
[Bug] 推理完成了，但是因为插件版本，在做测评的时候报错了，但是预测的哪些文件都在，怎么重新继续做预测？还有就是运行的时候默认好像是多有的数据集都会测试
#1761 opened Dec 14, 2024 by Lelege0
2 tasks done
@tonysy
1
[Bug] commonsenseqa_gen_c946f2中的commonsenseqa_datasets数据集报错 TypeError: 'ListWrapper' object is not iterable
#1760 opened Dec 14, 2024 by hanbaoergogo
2 tasks done
@tonysy
4
[Bug] cannot use GPU when conduct evaluation
#1759 opened Dec 12, 2024 by takagi97
2 tasks done
@acylam
[Bug] 提示新写的Dataset未注册
#1756 opened Dec 11, 2024 by ezeli
2 tasks done
@MaiziXiao
[Bug] 无法多卡并行评测数据
#1755 opened Dec 11, 2024 by luhairong11
2 tasks done
@MaiziXiao
3
[Bug] 自定义数据集，模型进行评测结果为0分
#1752 opened Dec 11, 2024 by lomoonmoonbird
2 tasks done
@MaiziXiao
4
[Feature] Evaluate multiple metrics simultaneously
#1748 opened Dec 9, 2024 by ezeli
1 task
@bittersweet1999
3
[Bug] NPU 运行报错 core dump
#1747 opened Dec 9, 2024 by SefaZeng
2 tasks done
@bittersweet1999
1
[Bug] 自定义模型评测配置
#1746 opened Dec 9, 2024 by ucassqs
2 tasks done
@bittersweet1999
1
ProTip! What’s not been updated in a month: updated:<2024-12-15.