Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Correct bookmarked date (#9) #11

Closed
wants to merge 8 commits into from
Closed

Correct bookmarked date (#9) #11

wants to merge 8 commits into from

Conversation

flpezet
Copy link

@flpezet flpezet commented Oct 23, 2019

Misbehavior
Say I set incremental_range = 'daily'.
If the tap is run at 8:AM, all the answers and landings between midnight and 8:AM will be retrieved.
An attempt to get data after 8:AM will be made with obviously no returned data.

The next job will have its date_to_resume set to the next day at midnight. So the next jobs won't retrieve the data after 8:AM.
The resulting problem can be seen in #9 .

This PR aims at preventing date_to_resume to be higher than end_date.

@cmerrick
Copy link
Contributor

Hi @flpezet, thanks for your contribution!

In order for us to evaluate and accept your PR, we ask that you sign a contribution license agreement. It's all electronic and will take just minutes.

@cmerrick
Copy link
Contributor

You did it @flpezet!

Thank you for signing the Singer Contribution License Agreement.

@jennygibbons
Copy link

Hi @flpezet and @cmerrick - I am wondering if there is any update on this? We are running into the same problem and it seems like this would fix it.

Thanks!

@flpezet
Copy link
Author

flpezet commented Dec 5, 2019

Andy Lu has started to give me feedback on the PR via Singer Slack. I'm waiting for the next iteration.
Can't give you any ETA... :/
@luandy64 is it possible for you to review the PR? It seems to be a very eagerly awaited bug fix. If I can do anything to speed up the review process, let me know! :)

@kingfink
Copy link
Contributor

@flpezet @luandy64 any chance of a review on this or an update on the ETA?

Happy to do anything I can to accelerate the process.

@flpezet flpezet closed this Dec 13, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants