Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve YouTube API error handling #1199

Conversation

shiftybitshiftr
Copy link
Contributor

This PR enhances how the bot handles YouTube API issues by adding better error handling, retries, and user feedback. The main improvements are:

  • Typed error handling with a new YouTubeError interface
  • Smart retries with exponential backoff for recoverable errors
  • Clear error messages for users when things go wrong
  • Better logging to help diagnose issues

I updated youtube-api.ts, centralizing error handling in a new executeYouTubeRequest method that all YouTube API calls use. This makes our error handling more consistent and makes the bot more resilient to API issues.

Users will now see clearer messages like:

🚫 ope: YouTube API rate limit reached. Please try again later.
🚫 ope: YouTube API quota exceeded. Please try again later.
🚫 ope: Video could not be found or is unavailable.

Testing Notes

The key things to test are:

  • Normal video fetching works as expected
  • Rate limit handling (HTTP 429) triggers backoff
  • Error messages are clear and helpful
  • Network timeouts implement retries correctly

Thanks!

Enhance YouTube API service with comprehensive error handling, retries, and
logging to improve reliability. Add typed YouTubeError interface, implement
exponential backoff with jitter, properly handle rate limits (429) and quota
exceeded (403), and improve cache key management.
@shiftybitshiftr shiftybitshiftr marked this pull request as draft January 26, 2025 01:56
@codetheweb
Copy link
Collaborator

codetheweb commented Jan 26, 2025

📦 🤖 A new release has been made for this pull request.

To play around with this PR, pull ghcr.io/museofficial/muse:pr-1199.

Images are available for x86_64 and ARM64.

Latest commit: 2f79aa6

@shiftybitshiftr
Copy link
Contributor Author

Addressed type and linting errors. All checks passing. Tested changes locally. Ready for review :)

@shiftybitshiftr shiftybitshiftr marked this pull request as ready for review January 26, 2025 02:53
const error = new Error(message) as YouTubeError;
error.code = code;
error.status = status;
error.retryable = code === 'NETWORK_ERROR' || (status ? status >= 500 : false);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we treat HTTP 403 as retry-able?

Copy link
Contributor Author

@shiftybitshiftr shiftybitshiftr Jan 28, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A 403 forbidden generally won't resolve with a retry, since it's due to invalid API key, geo restriction, or exceeding daily API quota

Copy link
Contributor

@JoeWHoward JoeWHoward left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, one question left on retries, nice PR.

Oh and if you add a line in the changelog we can push a smaller release, as there's not been a release for a while.

@shiftybitshiftr shiftybitshiftr merged commit 373e6a5 into museofficial:master Jan 28, 2025
4 checks passed
shiftybitshiftr added a commit that referenced this pull request Jan 28, 2025
shiftybitshiftr added a commit that referenced this pull request Jan 28, 2025
This reverts commit 373e6a5 so we can push a smaller release with just the dependency updates first.
@codetheweb
Copy link
Collaborator

🚀 Released in Release v2.10.1.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants