Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUG] test_metrics test failing in the Windows GitHub runner #724

Closed
keithmanville opened this issue Jan 29, 2025 · 3 comments · Fixed by #728
Closed

[BUG] test_metrics test failing in the Windows GitHub runner #724

keithmanville opened this issue Jan 29, 2025 · 3 comments · Fixed by #728
Assignees
Labels
bug Something isn't working

Comments

@keithmanville
Copy link
Collaborator

test_metrics is failing in the Windows GitHub runner.

When a second value for the accuracy metric is posted, a subsequent GET returns the first value instead of the newer value.

see:
https://github.com/usnistgov/dioptra/actions/runs/13035219363/job/36366305054?pr=722

     def assert_job_metrics_matches_expectations(
        dioptra_client: DioptraClient[DioptraResponseProtocol],
        job_id: int,
        expected: list[dict[str, Any]],
    ) -> None:
        response = dioptra_client.jobs.get_metrics_by_id(job_id=job_id)
>       assert response.status_code == HTTPStatus.OK and response.json() == expected
E       AssertionError: assert (200 == <HTTPStatus.OK: 200> and [{'name': 'ac...value': 0.99}] == [{'name': 'ac...value': 0.99}]
E        +  where 200 = <tests.unit.restapi.lib.client.DioptraTestResponse object at 0x00000183AFF19850>.status_code
E        +  and   <HTTPStatus.OK: 200> = HTTPStatus.OK
E         
E         At index 0 diff: {'name': 'accuracy', 'value': 4.1} != {'name': 'accuracy', 'value': 4.2}
E         Use -v to get more diff)
@keithmanville keithmanville added the bug Something isn't working label Jan 29, 2025
@keithmanville
Copy link
Collaborator Author

This behavior is not consistent.

Here the same test passes:
https://github.com/usnistgov/dioptra/actions/runs/13037442728/job/36371278927

@jkglasbrenner
Copy link
Collaborator

jkglasbrenner commented Jan 30, 2025

It does not seem like #728 solved the issue, this GitHub Action ran after I rebased a branch on dev to include this change: https://github.com/usnistgov/dioptra/actions/runs/13062292399/job/36447856877

Image

@keithmanville
Copy link
Collaborator Author

resolved by #728

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants