Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

(old) Move from 2d to 3d array operations #11

Closed
wants to merge 32 commits into from

Conversation

PinkShnack
Copy link

About

We should allow stacks of 2D arrays as inputs. This will likely speed up processing of large datasets, and certainly will when we add Cupy as an FFTFilter (#10). Several things to note:

  • Allowed formats e.g. RGB, 2d, 3d (see below)
  • What should be returned to the user: currently, a 3d array with the shape (z, y, x), with z being the number of images in the stack.
    • We can provide convenience functions for users to convert between formats. I think we should limit this as a "per image" basis. In other words, the functions should handle image input, not qpretrieve classes.

To do

  • FFTFilter Base class work with 3D arrays
  • run_pipeline steps for OAH and QLSI should also work with 3D arrays.
  • Benchmark single 2d array prcessed n times against stack of n images all processed at once. Put in docs.
  • Image formats. The following formats should continue to work.
    • 2D images (y, x) should be converted internally to (z, y, x)
    • 3D images (image stacks) (z, y, x) will work with above
    • 2D RGB images (y, x, 3) or RGBA (y, x, 4). The data will be assumed to be greyscale, and each channel equal. The first channel will be taken and converted as in the 2D images case above.
      • A warning should be passed to the user.
    • The shape of the returned field will be 3D, and we will provide convenience functions for conversion back to the original format.
  • docs
    • compare old (3d input -> 3d output) and new versions (2d input -> 3d output)
    • show image format conversion
    • show benchmark for 2d vs 3d.
  • tests:
    • data input type examples taken from real examples
    • hologram fixture not inputting size argument
    • compare 2d and 3d FFT algorithms, ensure consistency
  • cicd passed
  • update changelog

Eoghan O'Connell added 30 commits August 26, 2024 16:00
@@ -230,6 +236,8 @@ def run_pipeline(self, **pipeline_kws):

self.pipeline_kws.update(pipeline_kws)

# raw_wavefront = revert_to_data_input_format(
# self.fft.data_format, raw_wavefront)
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This will be removed

@PinkShnack
Copy link
Author

Hey @paulmueller, I know you are picky with the commit history. This PR is just branched off from #8, and therefore has old commits where I worked with the Cupy code. So the history isn't perfectly "clean". I've cleaned this up, but if you really want me to create a fresh branch from main, let me know.

@paulmueller
Copy link
Member

Hey @paulmueller, I know you are picky with the commit history. This PR is just branched off from #8, and therefore has old commits where I worked with the Cupy code. So the history isn't perfectly "clean". I've cleaned this up, but if you really want me to create a fresh branch from main, let me know.

Yes, please create a fresh branch. It should be fairly simple though (create new branch and copy paste all files with override to the new working directory).

@PinkShnack
Copy link
Author

This is being continued in #12

@PinkShnack PinkShnack closed this Jan 14, 2025
@PinkShnack PinkShnack changed the title Move from 2d to 3d array operations (old) Move from 2d to 3d array operations Jan 14, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants