Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

fix save_ckpt bug #536

Open
wants to merge 1 commit into
base: main
Choose a base branch
from
Open

fix save_ckpt bug #536

wants to merge 1 commit into from

Conversation

HYLcool
Copy link
Collaborator

@HYLcool HYLcool commented Jan 9, 2025

  • IndexError if the number of samples in the result dataset is less than the number of workers when saving dataset to disk

@HYLcool
* fix save_ckpt bug: error if the number of samples in the result dat…
04b5ccb 
…aset is less than the number of workers when saving dataset to disk
@HYLcool HYLcool added bug Something isn't working dj:efficiency regarding to efficiency issues and enhancements labels Jan 9, 2025
@HYLcool HYLcool requested review from BeachWang and yxdyc January 9, 2025 09:00
@HYLcool HYLcool self-assigned this Jan 9, 2025
@HYLcool HYLcool linked an issue Jan 9, 2025 that may be closed by this pull request
[Bug]: 开启checkpoint，当配置的一个pipline执行到最后一个算子时，np>left samples，在开启checkpoint会报错。 #531
Open
3 tasks
@HYLcool HYLcool added dj:core issues/PRs about the core functions of Data-Juicer and removed dj:efficiency regarding to efficiency issues and enhancements labels Jan 9, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@BeachWang BeachWang Awaiting requested review from BeachWang

@yxdyc yxdyc Awaiting requested review from yxdyc

At least 1 approving review is required to merge this pull request.

Assignees

@HYLcool HYLcool

Labels
bug Something isn't working dj:core issues/PRs about the core functions of Data-Juicer
Projects
None yet
Milestone
No milestone
1 participant
@HYLcool