Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

tests: config validator is now tested properly #2926

Open
wants to merge 5 commits into
base: master
Choose a base branch
from

Conversation

karthik11135
Copy link

@karthik11135 karthik11135 commented Jan 6, 2025

📝 Description

Tests for the new configuration validator are written (TODO task)

@andrewfullard kindly take a look. Thanks.

@tardis-bot
Copy link
Contributor

*beep* *bop*

Hi, human.

I'm the @tardis-bot and couldn't find your records in my database. I think we don't know each other, or you changed your credentials recently.

Please add your name and email to .mailmap in your current branch and push the changes to this pull request.

In case you need to map an existing alias, follow this example.

@tardis-bot
Copy link
Contributor

tardis-bot commented Jan 6, 2025

*beep* *bop*
Hi human,
I ran ruff on the latest commit (0f7e452).
Here are the outputs produced.
Results can also be downloaded as artifacts here.
Summarised output:

10	E902  	[ ] No such file or directory (os error 2)
8	E999  	[ ] SyntaxError: Expected ',', found ':'
8	W291  	[*] Trailing whitespace
7	W293  	[*] Blank line contains whitespace
6	G004  	[ ] Logging statement uses f-string
4	RET505	[ ] Unnecessary `else` after `return` statement
4	I001  	[*] Import block is un-sorted or un-formatted
4	F401  	[ ] `tardis.plasma.equilibrium.rates.collision_strengths.UpsilonCMFGENSolver` imported but unused; consider removing, adding to `__all__`, or using a redundant alias
2	PTH117	[ ] `os.path.isabs()` should be replaced by `Path.is_absolute()`
2	E712  	[*] Avoid equality comparisons to `False`; use `if not is_quantity(None, quantity_val_2):` for false checks
2	W292  	[*] No newline at end of file
2	W605  	[*] Invalid escape sequence: `\A`
1	ANN204	[ ] Missing return type annotation for special method `__getitem__`
1	B017  	[ ] `pytest.raises(Exception)` should be considered evil
1	INP001	[ ] File `tardis/io/configuration/tests/test_config_validator.py` is part of an implicit namespace package. Add an `__init__.py`.
1	RET506	[ ] Unnecessary `else` after `raise` statement
1	D202  	[*] No blank lines allowed after function docstring (found 1)
1	D406  	[*] Section name should end with a newline ("Returns")
1	D407  	[*] Missing dashed underline after section ("Returns")
1	F541  	[*] f-string without any placeholders

Complete output(might be large):

.github/workflows/release.yml:7:4: E999 SyntaxError: Expected an expression
.github/workflows/release.yml:94:1: W293 [*] Blank line contains whitespace
.github/workflows/release.yml:97:1: W293 [*] Blank line contains whitespace
.github/workflows/release.yml:104:1: W293 [*] Blank line contains whitespace
.github/workflows/release.yml:107:1: W293 [*] Blank line contains whitespace
.github/workflows/release.yml:110:1: W293 [*] Blank line contains whitespace
.github/workflows/release.yml:115:1: W293 [*] Blank line contains whitespace
.github/workflows/release.yml:119:1: W293 [*] Blank line contains whitespace
.github/workflows/release.yml:144:23: W292 [*] No newline at end of file
.mailmap:1:38: E999 SyntaxError: Expected an expression
.mailmap:292:39: W292 [*] No newline at end of file
CHANGELOG.md:4:15: E999 SyntaxError: Expected ',', found ':'
CITATION.cff:3:1: E999 SyntaxError: Invalid annotated assignment target
README.rst:1:1: E999 SyntaxError: Expected a statement
docs/contributing/development/benchmarks.rst:1:1: E999 SyntaxError: Expected a statement
docs/io/output/how_to_rpacket_tracking.ipynb:cell 12:1:62: W291 [*] Trailing whitespace
docs/physics/plasma/equilibrium/tardis_solver_cmfgen.ipynb:1:1: E902 No such file or directory (os error 2)
docs/physics/setup/model.ipynb:cell 2:2:1: I001 [*] Import block is un-sorted or un-formatted
docs/physics/setup/model.ipynb:cell 25:19:12: F541 [*] f-string without any placeholders
docs/quickstart.ipynb:cell 12:1:39: W291 [*] Trailing whitespace
docs/quickstart.ipynb:cell 12:5:35: W291 [*] Trailing whitespace
docs/quickstart.ipynb:cell 16:10:26: W605 [*] Invalid escape sequence: `\A`
docs/quickstart.ipynb:cell 16:11:40: W605 [*] Invalid escape sequence: `\A`
docs/resources/credits.rst:1:1: E999 SyntaxError: Expected a statement
docs/workflows/v_inner_solver_workflow.ipynb:1:1: E902 No such file or directory (os error 2)
tardis/io/configuration/config_reader.py:53:29: G004 Logging statement uses f-string
tardis/io/configuration/config_reader.py:117:9: RET505 Unnecessary `else` after `return` statement
tardis/io/configuration/config_reader.py:141:13: RET505 Unnecessary `else` after `return` statement
tardis/io/configuration/config_reader.py:218:29: G004 Logging statement uses f-string
tardis/io/configuration/schemas/montecarlo_definitions.yml:1:13: E999 SyntaxError: Expected an expression
tardis/io/configuration/tests/test_config_validator.py:1:1: INP001 File `tardis/io/configuration/tests/test_config_validator.py` is part of an implicit namespace package. Add an `__init__.py`.
tardis/io/configuration/tests/test_config_validator.py:1:1: I001 [*] Import block is un-sorted or un-formatted
tardis/io/configuration/tests/test_config_validator.py:1:14: W291 [*] Trailing whitespace
tardis/io/configuration/tests/test_config_validator.py:8:19: W291 [*] Trailing whitespace
tardis/io/configuration/tests/test_config_validator.py:9:19: W291 [*] Trailing whitespace
tardis/io/configuration/tests/test_config_validator.py:10:19: W291 [*] Trailing whitespace
tardis/io/configuration/tests/test_config_validator.py:23:10: B017 `pytest.raises(Exception)` should be considered evil
tardis/io/configuration/tests/test_config_validator.py:34:12: E712 Avoid equality comparisons to `False`; use `if not is_quantity(None, quantity_val_2):` for false checks
tardis/io/configuration/tests/test_config_validator.py:35:12: E712 Avoid equality comparisons to `True`; use `if is_quantity(None, quantity_val_1):` for truth checks
tardis/io/configuration/tests/test_config_validator.py:69:77: W291 [*] Trailing whitespace
tardis/io/model/parse_geometry_configuration.py:50:12: PTH117 `os.path.isabs()` should be replaced by `Path.is_absolute()`
tardis/io/model/parse_radiation_field_configuration.py:118:13: G004 Logging statement uses f-string
tardis/model/base.py:340:12: PTH117 `os.path.isabs()` should be replaced by `Path.is_absolute()`
tardis/model/base.py:375:21: G004 Logging statement uses f-string
tardis/opacities/macro_atom/macroatom_state.py:1:1: I001 [*] Import block is un-sorted or un-formatted
tardis/opacities/opacity_state.py:210:9: ANN204 Missing return type annotation for special method `__getitem__`
tardis/opacities/opacity_state.py:216:9: D407 [*] Missing dashed underline after section ("Returns")
tardis/opacities/opacity_state.py:216:9: D406 [*] Section name should end with a newline ("Returns")
tardis/opacities/opacity_state.py:251:5: D202 [*] No blank lines allowed after function docstring (found 1)
tardis/opacities/tests/__init__.py:1:1: E902 No such file or directory (os error 2)
tardis/plasma/equilibrium/rates/__init__.py:2:5: F401 `tardis.plasma.equilibrium.rates.collision_strengths.UpsilonCMFGENSolver` imported but unused; consider removing, adding to `__all__`, or using a redundant alias
tardis/plasma/equilibrium/rates/__init__.py:3:5: F401 `tardis.plasma.equilibrium.rates.collision_strengths.UpsilonRegemorterSolver` imported but unused; consider removing, adding to `__all__`, or using a redundant alias
tardis/plasma/equilibrium/rates/__init__.py:6:5: F401 `tardis.plasma.equilibrium.rates.collisional_rates.ThermalCollisionalRateSolver` imported but unused; consider removing, adding to `__all__`, or using a redundant alias
tardis/plasma/equilibrium/rates/__init__.py:9:5: F401 `tardis.plasma.equilibrium.rates.radiative_rates.RadiativeRatesSolver` imported but unused; consider removing, adding to `__all__`, or using a redundant alias
tardis/plasma/equilibrium/rates/collisional_ionization_rates.py:1:1: E902 No such file or directory (os error 2)
tardis/plasma/equilibrium/rates/collisional_ionization_strengths.py:1:1: E902 No such file or directory (os error 2)
tardis/plasma/equilibrium/rates/photoionization_rates.py:1:1: E902 No such file or directory (os error 2)
tardis/plasma/equilibrium/rates/photoionization_strengths.py:1:1: E902 No such file or directory (os error 2)
tardis/radiation_field/validate_radiation_field.py:1:1: E902 No such file or directory (os error 2)
tardis/spectrum/formal_integral.py:1:1: I001 [*] Import block is un-sorted or un-formatted
tardis/spectrum/formal_integral.py:358:13: RET506 Unnecessary `else` after `raise` statement
tardis/spectrum/formal_integral.py:698:5: RET505 Unnecessary `else` after `return` statement
tardis/spectrum/formal_integral.py:734:5: RET505 Unnecessary `else` after `return` statement
tardis/workflows/simple_tardis_workflow.py:230:17: G004 Logging statement uses f-string
tardis/workflows/simple_tardis_workflow.py:449:17: G004 Logging statement uses f-string
tardis/workflows/util.py:1:1: E902 No such file or directory (os error 2)
tardis/workflows/v_inner_solver.py:1:1: E902 No such file or directory (os error 2)
Found 67 errors.
[*] 27 fixable with the `--fix` option (2 hidden fixes can be enabled with the `--unsafe-fixes` option).

@tardis-bot
Copy link
Contributor

tardis-bot commented Jan 6, 2025

*beep* *bop*
Hi human,
I ran benchmarks as you asked comparing master (7e37d5f) and the latest commit (a10f301).
Here are the logs produced by ASV.
Results can also be downloaded as artifacts here.

Significantly changed benchmarks:

All benchmarks:

Benchmarks that have stayed the same:

| Change   | Before [5e2d0bb3] <master>   | After [a10f3015]    | Ratio   | Benchmark (Parameter)                                                                                                               |
|----------|------------------------------|---------------------|---------|-------------------------------------------------------------------------------------------------------------------------------------|
|          | 3.46±0.2μs                   | 4.67±0.5μs          | ~1.35   | transport_montecarlo_vpacket.BenchmarkMontecarloMontecarloNumbaVpacket.time_trace_vpacket_within_shell                              |
|          | 591±200ns                    | 531±200ns           | ~0.90   | opacities_opacity.BenchmarkMontecarloMontecarloNumbaOpacities.time_pair_creation_opacity_calculation                                |
|          | 52.1±20μs                    | 46.7±20μs           | ~0.90   | transport_montecarlo_interaction.BenchmarkTransportMontecarloInteraction.time_line_emission                                         |
|          | 2.92±0.01ms                  | 2.53±0.01ms         | ~0.87   | opacities_opacity_state.BenchmarkOpacitiesOpacityState.time_opacity_state_initialize('scatter')                                     |
|          | 52.8±30μs                    | 45.6±30μs           | ~0.86   | transport_montecarlo_interaction.BenchmarkTransportMontecarloInteraction.time_line_scatter                                          |
|          | 45.5±4μs                     | 37.2±0.02μs         | ~0.82   | transport_montecarlo_packet_trackers.BenchmarkTransportMontecarloPacketTrackers.time_generate_rpacket_tracker_list                  |
|          | 1.91±0.4μs                   | 1.42±0.3μs          | ~0.74   | transport_geometry_calculate_distances.BenchmarkTransportGeometryCalculateDistances.time_calculate_distance_line                    |
|          | 7.21±2μs                     | 7.77±2μs            | 1.08    | transport_montecarlo_vpacket.BenchmarkMontecarloMontecarloNumbaVpacket.time_trace_vpacket_volley                                    |
|          | 204±0.2ns                    | 217±0.2ns           | 1.07    | spectrum_formal_integral.BenchmarkTransportMontecarloFormalIntegral.time_intensity_black_body                                       |
|          | 25.3±7μs                     | 26.6±7μs            | 1.05    | transport_montecarlo_packet_trackers.BenchmarkTransportMontecarloPacketTrackers.time_generate_rpacket_last_interaction_tracker_list |
|          | 733±0.4ns                    | 755±10ns            | 1.03    | transport_montecarlo_interaction.BenchmarkTransportMontecarloInteraction.time_thomson_scatter                                       |
|          | 2.56±0.5ms                   | 2.58±0.5ms          | 1.01    | transport_montecarlo_single_packet_loop.BenchmarkTransportMontecarloSinglePacketLoop.time_single_packet_loop                        |
|          | 631±90ns                     | 631±100ns           | 1.00    | opacities_opacity.BenchmarkMontecarloMontecarloNumbaOpacities.time_compton_opacity_calculation                                      |
|          | 2.08±0m                      | 2.08±0m             | 1.00    | spectrum_formal_integral.BenchmarkTransportMontecarloFormalIntegral.time_FormalIntegrator_functions                                 |
|          | 1.05±0m                      | 1.04±0m             | 0.99    | run_tardis.BenchmarkRunTardis.time_run_tardis_rpacket_tracking                                                                      |
|          | 3.09±0.5μs                   | 3.05±0.4μs          | 0.99    | transport_montecarlo_vpacket.BenchmarkMontecarloMontecarloNumbaVpacket.time_trace_bad_vpacket                                       |
|          | 6.11±0.8μs                   | 6.05±0.9μs          | 0.99    | transport_montecarlo_vpacket.BenchmarkMontecarloMontecarloNumbaVpacket.time_trace_vpacket                                           |
|          | 66.1±0.3ms                   | 64.7±0.02ms         | 0.98    | transport_montecarlo_packet_trackers.BenchmarkTransportMontecarloPacketTrackers.time_rpacket_trackers_to_dataframe                  |
|          | 39.0±0.07s                   | 37.8±0.1s           | 0.97    | run_tardis.BenchmarkRunTardis.time_run_tardis                                                                                       |
|          | 1.26±0μs                     | 1.23±0μs            | 0.97    | transport_geometry_calculate_distances.BenchmarkTransportGeometryCalculateDistances.time_calculate_distance_boundary                |
|          | 2.10±2μs                     | 2.03±1μs            | 0.97    | transport_montecarlo_estimators_radfield_estimator_calcs.BenchmarkMontecarloMontecarloNumbaPacket.time_update_line_estimators       |
|          | 1.78±0.01ms                  | 1.68±0.01ms         | 0.95    | transport_montecarlo_main_loop.BenchmarkTransportMontecarloMontecarloMainLoop.time_montecarlo_main_loop                             |
|          | 621±100ns                    | 582±200ns           | 0.94    | opacities_opacity.BenchmarkMontecarloMontecarloNumbaOpacities.time_photoabsorption_opacity_calculation                              |
|          | 3.87±0.04ms                  | 3.63±0.03ms         | 0.94    | opacities_opacity_state.BenchmarkOpacitiesOpacityState.time_opacity_state_initialize('macroatom')                                   |

If you want to see the graph of the results, you can check it here

@karthik11135
Copy link
Author

@atharva-2001 could you please review. Thanks.

tardis/workflows/standard_tardis_workflow.py Outdated Show resolved Hide resolved
dict_with_Default = validate_dict(tardis_config_verysimple, validator=Default_Validator)
assert dict_with_Default["plasma"]["initial_t_inner"] == "-1 K"

with pytest.raises(ValidationError):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does this with statement mean that these tests are expected to raise ValidationError?

Copy link
Author

@karthik11135 karthik11135 Jan 23, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Validation error is occurred when we use Draft4validator instead of extending it for accepting default values (Default_validator)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So are you testing the extended validator, or not?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah. I've tested both. Default validator and Draft4validator

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Status: Todo
Development

Successfully merging this pull request may close these issues.

3 participants