Skip to content

Pull requests: bigcode-project/bigcode-evaluation-harness

New pull request New
33 Open 116 Closed
33 Open 116 Closed
Author
Filter by author
Loading
Label
Filter by label
Loading
Use alt + click/return to exclude labels
or ⇧ + click/return for logical OR
Projects
Filter by project
Loading
Milestones
Filter by milestone
Loading
Reviews
Filter by reviews
No reviews Review required Approved review Changes requested
Assignee
Filter by who’s assigned
Sort
Sort by
Newest Oldest Most commented Least commented Recently updated Least recently updated Best match
Most reactions
👍 👎 😄 🎉 😕 ❤️ 🚀 👀

Pull requests list

use tokenizer.chat_template by default for instruction type tasks
#301 opened Jan 25, 2025 by TK-21st Loading…
Missing comma in MultiPL-E languages
#299 opened Jan 13, 2025 by hrshtv Loading…
1
[Pytest] Fix bad import to use relative instead @ module_test
#298 opened Jan 11, 2025 by ggcr Loading…
Fix the bugs in the ds1000 sample bash script; Fix typos
#295 opened Dec 10, 2024 by gameofby Loading…
Update multiple.py
#292 opened Dec 8, 2024 by ahmedashrafy Loading…
Support multiple datasets from MBPP; Fix missing commas in python list; Fix doc typos;
#291 opened Dec 4, 2024 by gameofby Loading…
add support for hpu devices
#281 opened Oct 25, 2024 by envsp Loading…
1
"," missing in LANGUAGES list
#280 opened Oct 21, 2024 by ArtemisDicoTiar Loading…
2
Speedup execute.py: Reuse same manager and dict in
#277 opened Oct 1, 2024 by michaelfeil Loading…
Basecodes
#263 opened Aug 14, 2024 by Abhineetsoccer Loading…
Add a new benchmark ENAMEL for evaluating the efficiency of LLM-generated code
#260 opened Jul 22, 2024 by q-rz Loading…
Fix Max New Tokens in HF's Generation Config
#257 opened Jul 18, 2024 by mostafaelhoushi Loading…
2
Fix unnecessary repeated overwrite
#249 opened Jun 29, 2024 by nielstron Loading…
fix: Multiple-E dataset fix go_test.go path for test execution
#225 opened Apr 20, 2024 by hitesh-1997 Loading…
1
4
Add llama3 instruction prompts
#222 opened Apr 19, 2024 by TechxGenus Loading…
1
Leaderboard README improvements
#217 opened Apr 14, 2024 by nikita1503 Loading…
4
remove pad tokens added by the accelerator.pad_across_processes
#216 opened Apr 13, 2024 by IQ17 Loading…
Ensure generations get saved in generation_only mode
#212 opened Mar 31, 2024 by Vipitis Loading…
1
fix apps evaluate error: local variable 'level' referenced before assignment
#206 opened Mar 10, 2024 by koking0 Loading…
2
Update README.md
#204 opened Mar 2, 2024 by AnitaLiu98 Loading…
4
Fix loading PAL-GSM few-shot examples
#196 opened Feb 8, 2024 by sxjscience Loading…
1
Make main.py compatible with OpenAI compatible APIs
#189 opened Jan 23, 2024 by hmellor Loading…
5
Fix typo in README.md
#177 opened Jan 2, 2024 by ab-10 Loading…
[WIP] Shadereval tasks
#173 opened Dec 16, 2023 by Vipitis • Draft
1 of 4 tasks
Add support for Ollama, Palm, Claude-2, Cohere, Replicate, Llama2 CodeLlama (100+LLMs) [LiteLLM]
#160 opened Nov 9, 2023 by ishaan-jaff Loading…
8
ProTip! Mix and match filters to narrow down what you’re looking for.