Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add sizes per location and support .local #5581

Merged
merged 1 commit into from
Jan 28, 2025
Merged

Add sizes per location and support .local #5581

merged 1 commit into from
Jan 28, 2025

Conversation

mdegat01
Copy link
Contributor

@mdegat01 mdegat01 commented Jan 27, 2025

Proposed change

Add location_attributes to API. Returns a dictionary of locations that backup can be found in. For each location says if the backup is protected and the size in bytes.

This removes the protected_locations field added in #5569 as it provides a more flexible solution. The reason it wasn't done this way to begin with is because the value used for the local /backups folder is None and None cannot be a dictionary key in JSON.

To handle this we use .local as the key in this dictionary. For consistency across the APIs I also adjusted all APIs which accept a location to allow .local and "" and treat those as the local /backups folder.

Type of change

  • Dependency upgrade
  • Bugfix (non-breaking change which fixes an issue)
  • New feature (which adds functionality to the supervisor)
  • Breaking change (fix/feature causing existing functionality to break)
  • Code quality improvements to existing code or addition of tests

Additional information

Checklist

  • The code change is tested and works locally.
  • Local tests pass. Your PR cannot be merged unless tests pass
  • There is no commented out code in this PR.
  • I have followed the development checklist
  • The code has been formatted using Ruff (ruff format supervisor tests)
  • Tests have been added to verify that the new code works.

If API endpoints or add-on configuration are added/changed:

Summary by CodeRabbit

Release Notes

  • New Features

    • Enhanced backup location handling with more detailed location attributes
    • Improved backup file size calculation and caching mechanism
  • Bug Fixes

    • Updated location processing to handle empty and special location inputs more effectively
  • Refactor

    • Replaced protected_locations with more comprehensive location_attributes
    • Optimized backup size retrieval methods
  • Tests

    • Expanded test coverage for backup location scenarios
    • Added more robust testing for backup upload, download, and restore functionality

@mdegat01 mdegat01 added missing-documentation Added to pull requests that needs a docs, but none is linked new-feature A new feature need-client-library Added to pull requests that need client library changes but none is linked labels Jan 27, 2025
@mdegat01 mdegat01 requested a review from agners January 27, 2025 22:07
@mdegat01 mdegat01 changed the title Add sizes per location and support .local Add sizes per location and support .local Jan 27, 2025
Copy link
Contributor

coderabbitai bot commented Jan 27, 2025

📝 Walkthrough

Walkthrough

The pull request introduces significant enhancements to the backup system in the supervisor API. The changes focus on improving location handling, backup file size calculation, and schema definitions. A new approach to managing backup location attributes is implemented, replacing the previous protected_locations mechanism. The modifications include adding methods for converting and processing location data, updating caching strategies for backup file sizes, and refining the test suite to cover more location-based scenarios.

Changes

File Changes
supervisor/api/backups.py - Added _convert_local_location method to handle location inputs
- Introduced _make_location_attributes method for generating location metadata
- Updated schema definitions for backup locations
- Modified download method to use new location conversion
supervisor/api/const.py - Added new constant ATTR_LOCATION_ATTRIBUTES
- Removed constant ATTR_PROTECTED_LOCATIONS
supervisor/backups/backup.py - Replaced cached_property with lru_cache for file size calculation
- Added _backup_file_size method for efficient size retrieval
- Introduced location_size method to get backup file size
- Changed size and size_bytes to regular properties
tests/api/test_backups.py - Updated test function signatures to include location parameters
- Modified assertions to check new location attributes
- Enhanced test coverage for backup location handling

Sequence Diagram

sequenceDiagram
    participant API as Backup API
    participant Backup as Backup Manager
    participant Storage as Storage System

    API->>Backup: Request backup details
    Backup->>Storage: Retrieve backup file
    Storage-->>Backup: Return file path
    Backup->>Backup: Calculate file size
    Backup-->>API: Return backup details with location attributes
Loading

This sequence diagram illustrates the new flow of retrieving backup details, highlighting the enhanced location attribute generation and file size calculation process.

✨ Finishing Touches
  • 📝 Generate Docstrings (Beta)

Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media?

❤️ Share
🪧 Tips

Chat

There are 3 ways to chat with CodeRabbit:

  • Review comments: Directly reply to a review comment made by CodeRabbit. Example:
    • I pushed a fix in commit <commit_id>, please review it.
    • Generate unit testing code for this file.
    • Open a follow-up GitHub issue for this discussion.
  • Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query. Examples:
    • @coderabbitai generate unit testing code for this file.
    • @coderabbitai modularize this function.
  • PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
    • @coderabbitai gather interesting stats about this repository and render them as a table. Additionally, render a pie chart showing the language distribution in the codebase.
    • @coderabbitai read src/utils.ts and generate unit testing code.
    • @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.
    • @coderabbitai help me debug CodeRabbit configuration file.

Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments.

CodeRabbit Commands (Invoked using PR comments)

  • @coderabbitai pause to pause the reviews on a PR.
  • @coderabbitai resume to resume the paused reviews.
  • @coderabbitai review to trigger an incremental review. This is useful when automatic reviews are disabled for the repository.
  • @coderabbitai full review to do a full review from scratch and review all the files again.
  • @coderabbitai summary to regenerate the summary of the PR.
  • @coderabbitai generate docstrings to generate docstrings for this PR. (Beta)
  • @coderabbitai resolve resolve all the CodeRabbit review comments.
  • @coderabbitai configuration to show the current CodeRabbit configuration for the repository.
  • @coderabbitai help to get help.

Other keywords and placeholders

  • Add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.
  • Add @coderabbitai summary to generate the high-level summary at a specific location in the PR description.
  • Add @coderabbitai or @coderabbitai title anywhere in the PR title to generate the title automatically.

CodeRabbit Configuration File (.coderabbit.yaml)

  • You can programmatically configure CodeRabbit by adding a .coderabbit.yaml file to the root of your repository.
  • Please see the configuration documentation for more information.
  • If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation: # yaml-language-server: $schema=https://coderabbit.ai/integrations/schema.v2.json

Documentation and Community

  • Visit our Documentation for detailed information on how to use CodeRabbit.
  • Join our Discord Community to get help, request features, and share feedback.
  • Follow us on X/Twitter for updates and announcements.

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 0

🧹 Nitpick comments (2)
supervisor/api/backups.py (1)

153-161: Consider adding docstring with example return value.

The _make_location_attributes method constructs the location attributes dictionary but could benefit from a docstring showing the structure of the returned dictionary.

Add a docstring like this:

 def _make_location_attributes(self, backup: Backup) -> dict[str, dict[str, Any]]:
-    """Make location attributes dictionary."""
+    """Make location attributes dictionary.
+    
+    Example return value:
+    {
+        ".local": {
+            "protected": True,
+            "size_bytes": 1024
+        },
+        "backup_mount": {
+            "protected": False,
+            "size_bytes": 2048
+        }
+    }
+    """
supervisor/backups/backup.py (1)

231-239: Consider adding docstring with size unit information.

The size properties could benefit from docstrings clarifying the units:

  • size returns megabytes
  • size_bytes returns bytes

Add docstrings like this:

 @property
 def size(self) -> float:
-    """Return backup size."""
+    """Return backup size in megabytes."""
     return round(self.size_bytes / 1048576, 2)  # calc mbyte

 @property
 def size_bytes(self) -> int:
-    """Return backup size in bytes."""
+    """Return backup size in bytes for the current location."""
     return self.location_size(self.location)
📜 Review details

Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 8ff9c08 and 75770cc.

📒 Files selected for processing (4)
  • supervisor/api/backups.py (10 hunks)
  • supervisor/api/const.py (1 hunks)
  • supervisor/backups/backup.py (4 hunks)
  • tests/api/test_backups.py (9 hunks)
🔇 Additional comments (9)
supervisor/api/const.py (1)

50-50: LGTM! The new constant follows the naming convention.

The addition of ATTR_LOCATION_ATTRIBUTES and removal of ATTR_PROTECTED_LOCATIONS aligns with the PR objectives to enhance backup location information.

supervisor/api/backups.py (4)

70-71: LGTM! The constant defines the key for local backups.

The introduction of LOCATION_LOCAL constant aligns with the PR objectives to use .local as the key for local backups.


87-91: LGTM! The function handles local location values consistently.

The _convert_local_location function correctly implements the requirement to treat both .local and empty string "" as references to the local /backups folder.


95-97: LGTM! The schema definitions ensure proper validation.

The schema constants provide robust validation:

  • SCHEMA_FOLDERS validates folder lists
  • SCHEMA_LOCATION handles location conversion
  • SCHEMA_LOCATION_LIST validates location lists

103-103: LGTM! The schema updates follow a logical pattern.

The schema definitions correctly handle location validation:

  • Single location for restore operations using SCHEMA_LOCATION
  • Multiple locations for backup and remove operations using SCHEMA_LOCATION_LIST

Also applies to: 121-121, 140-140

supervisor/backups/backup.py (2)

70-74: LGTM! The function efficiently caches backup file sizes.

The _backup_file_size function is well-designed:

  • Uses lru_cache for performance optimization
  • Handles non-existent files gracefully
  • Has a clear, single responsibility

263-270: LGTM! The method handles location-specific size retrieval.

The location_size method is well-implemented:

  • Returns 0 for non-existent locations
  • Uses cached file size calculation
  • Has clear error handling
tests/api/test_backups.py (2)

103-109: LGTM! The test parameters cover all location scenarios.

The test parameters effectively verify:

  • Backup to named location
  • Backup to local location using both None and empty string
  • Backup to local location using .local

1004-1009: LGTM! The assertions verify location attributes structure.

The test assertions thoroughly validate the location attributes:

  • Verify correct attribute structure
  • Test protection status
  • Validate size information

Also applies to: 1038-1043, 1051-1056

@lru_cache
def _backup_file_size(backup: Path) -> int:
"""Get backup file size."""
return backup.stat().st_size if backup.is_file() else 0
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should get the size when scanning the backup and cache the information. But it used to work like this before, so let's do one thing at a time.

Copy link
Member

@agners agners left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice! This makes .local work for all API, which is very nice. I think we should then also document that this is the preferred identifier.

Maybe we should also consider using .local throughout Supervisor. But this can be done in a follow, if we decide to go down that road.

@agners agners merged commit c8f1b22 into main Jan 28, 2025
23 checks passed
@agners agners deleted the sizes-per-location branch January 28, 2025 10:41
@mdegat01 mdegat01 removed need-client-library Added to pull requests that need client library changes but none is linked missing-documentation Added to pull requests that needs a docs, but none is linked labels Jan 29, 2025
@github-actions github-actions bot locked and limited conversation to collaborators Feb 1, 2025
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants