Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUG] Gradescope crashing causes forward crash #116

Open
3 tasks
gregbell26 opened this issue Nov 13, 2024 · 0 comments
Open
3 tasks

[BUG] Gradescope crashing causes forward crash #116

gregbell26 opened this issue Nov 13, 2024 · 0 comments
Labels
bug Something isn't working good first issue Good for newcomers Gradescope This issue only occurs on gradescope Result Generation

Comments

@gregbell26
Copy link
Member

Description

Issue

Gradescope crashes occasionally under periods of high load causing an empty entry in the submission metadata file.

When this the first entry in the list or in the submission sublist, the autograder crashes when finalizing the results metadata due to the missing score file.

Additionally, failed executions are counting against the student's submission limit.

Steps to recreate

Hard to reproduce: use example failed metadata instead

{
  "id": "REDACTED",
  "users": [
    "REDACTED"
  ],
  "created_at": "2024-11-12T22:51:05.537483-08:00",
  "assignment_id": null,
  "assignment": {
    "id": "REDACTED",
    "title": "Assessment 12: Image Compression",
    "release_date": "2024-11-04T14:00:00.000000-08:00",
    "due_date": "2024-11-12T22:59:00.000000-08:00",
    "total_points": "14.0",
    "course_id": "REDACTED",
    "group_submission": false,
    "group_size": null,
    "late_due_date": "2024-11-15T22:59:00.000000-08:00",
    "outline": [
      {
        "id": 39095172,
        "type": "ProgrammingQuestion",
        "title": "Autograder",
        "parent_id": null,
        "weight": "14.0",
        "content": [],
        "index": 1
      }
    ]
  },
  "submission_method": "upload",
  "previous_submissions": []
}

Results JSON should also be empty (or null)

Solution

In theory there are tests covering these cases, but it seems that they are missing some cases.
We also need to add an E2E test that verifies that we don't crash with a malformed results.json or submission_metadata.json.

Semi related to #87.

Acceptance Criteria

  • Autograder no longer crashes when a student resubmits a failed autograder run.
  • Autograder no longer crashes with a malformed submission_metadata.json or results.json
  • Autograder no longer counts failed runs against student when computing submission limit.
@gregbell26 gregbell26 added bug Something isn't working good first issue Good for newcomers Gradescope This issue only occurs on gradescope Result Generation labels Nov 13, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working good first issue Good for newcomers Gradescope This issue only occurs on gradescope Result Generation
Projects
None yet
Development

No branches or pull requests

1 participant