Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Display progress of a running scrape #72

Open
HorstmannA opened this issue Oct 3, 2023 · 0 comments
Open

Display progress of a running scrape #72

HorstmannA opened this issue Oct 3, 2023 · 0 comments

Comments

@HorstmannA
Copy link
Collaborator

When the 'Run scrape' button is pressed, it spins for several minutes before the 'All records scraped successfully' message pops up.
The backend logs Upserting {number_of_reports} reports and Upserting report {index} of {number_of_reports} but these aren't visible to the frontend.
These updates would be useful if displayed on the frontend, to show the user how much of the scrape has completed. It also gives evidence that the scrape is still running and hasn't got stuck.
Ideally we wouldn't be polling the backend continuously; a possible way to get this working would be using websockets.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant