Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

NaNs were messing up phase_change #98

Open
wants to merge 4 commits into
base: main
Choose a base branch
from
Open

NaNs were messing up phase_change #98

wants to merge 4 commits into from

Conversation

tjlane
Copy link
Collaborator

@tjlane tjlane commented Feb 4, 2025

Using meteor.phaseboost, it was possible to get NaNs in the output metadata:

2025-02-04 20:01:49 [info     ] convergence criteria:          max_iterations=5.0 phase_tolerance=0.001
2025-02-04 20:01:51 [info     ]   iteration 0001               negentropy=0.0034 phase_change=0.8369 tv_weight=0.01
2025-02-04 20:01:53 [info     ]   iteration 0002               negentropy=0.0073 phase_change=nan tv_weight=0.01
2025-02-04 20:01:55 [info     ]   iteration 0003               negentropy=0.0109 phase_change=nan tv_weight=0.01
2025-02-04 20:01:58 [info     ]   iteration 0004               negentropy=0.0139 phase_change=nan tv_weight=0.01
2025-02-04 20:02:00 [info     ]   iteration 0005               negentropy=0.0164 phase_change=nan tv_weight=0.01
2025-02-04 20:02:02 [info     ]   iteration 0006               negentropy=0.0184 phase_change=nan tv_weight=0.01

During iterative TV, it's possible that we compare arrays of native/derivative data without the same indices. In this case, NaNs are generated at the positions of mismatched HKL indices, but dealt with gracefully in downstream code... except for the average_phase_diff_in_degrees function in utils.

This PR updates that function to simply ignore NaN values. Based on feedback from @kmdalton, we proactively filter out non-common indices, which precludes the generation of the offending NaNs in the first place.

A regression test added to enforce this.

@tjlane
Copy link
Collaborator Author

tjlane commented Feb 4, 2025

merge #97 before merging this PR

@codecov-commenter
Copy link

codecov-commenter commented Feb 4, 2025

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 95.63%. Comparing base (8769c2f) to head (fbe5e49).
Report is 1 commits behind head on main.

Additional details and impacted files
@@           Coverage Diff           @@
##             main      #98   +/-   ##
=======================================
  Coverage   95.62%   95.63%           
=======================================
  Files          15       15           
  Lines         938      940    +2     
=======================================
+ Hits          897      899    +2     
  Misses         41       41           
Flag Coverage Δ
unittests 95.63% <100.00%> (+<0.01%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@tjlane tjlane requested a review from kmdalton February 4, 2025 19:22
meteor/utils.py Outdated Show resolved Hide resolved
@tjlane tjlane requested a review from kmdalton February 4, 2025 21:00
@@ -176,7 +182,10 @@ def _iteratively_denoise_sf_amplitudes(

# project onto the native amplitudes to obtain an "updated_derivative"
# Fh' = (D_F' + F) * [|Fh| / |D_F' + F|]
denoised_difference_sfs, native = filter_common_indices(denoised_difference_sfs, native)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can you either

  • filter outside the while loop? or
  • calculate the indices used for filtration outside the loop?

I'm sure it doesn't affect the code's correctness, but it unnecessarily costly as written. Not a big deal if it's a hassle.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants