Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

#521: Enrich synthetic data set with shared blocks #522

Merged
merged 10 commits into from
Jul 26, 2024

Conversation

ppebay
Copy link
Contributor

@ppebay ppebay commented Jul 14, 2024

Resolves #521

**DO NOT MERGE BEFORE #509 (PR #516) **

@ppebay ppebay requested review from lifflander and nlslatt July 14, 2024 19:02
@ppebay ppebay self-assigned this Jul 14, 2024
@ppebay ppebay marked this pull request as ready for review July 23, 2024 20:53
@ppebay
Copy link
Contributor Author

ppebay commented Jul 23, 2024

Now ready for review @lifflander @cwschilly

@ppebay ppebay requested a review from cwschilly July 23, 2024 21:01
@ppebay
Copy link
Contributor Author

ppebay commented Jul 25, 2024

@tlamonthezie I rebased this PR on top of develop, following the merger of PR #519 and I am now getting the following runtime error:

$ python LBAF_app.py  -c conf.yaml 
[LBAF_app] Found configuration file at path /Users/pppebay/Documents/Git/LB-analysis-framework/config/global.yaml
[LBAF_app] Found configuration file at path /Users/pppebay/Documents/Git/LB-analysis-framework/config/conf.yaml
[LBAF_app] Logging level: info
[lbsConfigurationValidator] Skeleton schema is valid
[lbsConfigurationValidator] Reading from data was chosen
[lbsConfigurationValidator] from_data schema is valid
[lbsConfigurationValidator] Checking algorithm schema of: {'name': 'InformAndTransfer', 'phase_id': 0, 'parameters': {'n_iterations': 8, 'n_rounds': 2, 'fanout': 2, 'order_strategy': 'arbitrary', 'transfer_strategy': 'Recursive', 'criterion': 'Tempered', 'max_objects_per_transfer': 8, 'deterministic_transfer': True}}
[lbsConfigurationValidator] Algorithm: {'name': 'InformAndTransfer', 'phase_id': 0, 'parameters': {'n_iterations': 8, 'n_rounds': 2, 'fanout': 2, 'order_strategy': 'arbitrary', 'transfer_strategy': 'Recursive', 'criterion': 'Tempered', 'max_objects_per_transfer': 8, 'deterministic_transfer': True}} schema is valid
[LBAF_app] Data stem: /Users/pppebay/Documents/Git/LB-analysis-framework/data/synthetic_lb_data/data
[LBAF_app] Executing LBAF version 1.0.2
[LBAF_app] Executing with Python 3.9.19
Traceback (most recent call last):
  File "/Users/pppebay/Documents/Git/LB-analysis-framework/src/lbaf/Applications/LBAF_app.py", line 622, in <module>
    LBAFApplication().run()
  File "/Users/pppebay/Documents/Git/LB-analysis-framework/src/lbaf/Applications/LBAF_app.py", line 471, in run
    reader = LoadReader(
  File "/Users/pppebay/Documents/Git/LB-analysis-framework/src/lbaf/IO/lbsVTDataReader.py", line 59, in __init__
    from ..imported.JSON_data_files_validator import \
  File "/Users/pppebay/Documents/Git/LB-analysis-framework/src/lbaf/imported/JSON_data_files_validator.py", line 19, in <module>
    import LBDatafile_schema
ModuleNotFoundError: No module named 'LBDatafile_schema'

Do you know what is wrong here? Thanks.

@cwschilly cwschilly force-pushed the 521-enrich-synthetic-data-set-with-shared-blocks branch from 0812dad to 7fbbda1 Compare July 25, 2024 20:32
@ppebay
Copy link
Contributor Author

ppebay commented Jul 25, 2024

Update: the same error happens for me with develop, following the merger of #519

@ppebay
Copy link
Contributor Author

ppebay commented Jul 26, 2024

I confirm @cwschilly that your change fixed the problem with the LB data file schema file.

PR can be reviewed @cwschilly @lifflander, in the default setting that comes with this PR, any other load-balancer (whether it is vt's temperedLB or the MILP approach) must find a 0-imbalance as LBAF's InformAndTransfer does in a couple of iterations:

synthetic-dataset-blocks0
synthetic-dataset-blocks1

N.B.: the rendered edges in the figures above must be disregarded as they correspond to the VTK problem on certain versions of macOS.

@ppebay ppebay requested a review from cwschilly July 26, 2024 14:00
Copy link
Contributor

@cwschilly cwschilly left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@ppebay ppebay merged commit 5c66bb6 into develop Jul 26, 2024
10 checks passed
@ppebay ppebay deleted the 521-enrich-synthetic-data-set-with-shared-blocks branch July 26, 2024 15:16
github-actions bot pushed a commit that referenced this pull request Jul 26, 2024
* #521: description, constructing script, and dataset files

* #521: Configuration file to run the synthetic+blocks case through PhaseStepper and render it

* #521: modified example and specification

* #521: new test case

* #521: excluse pngs from trailing whitespace check

* #521: fix png whitespace error

* #521: pull in LBDatafile_schema.py with JSON_data_files_validator.py

* #521: finalized example and configuration with baseline load-only setting that must reach 0 imbalance

* #521: generalize whitespace check to ignore all pngs

* #521: remove unused imports

---------

Co-authored-by: Caleb Schilly <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Enrich synthetic data set with shared blocks
2 participants