Add high cadence imaging worker and ability to turn frames into movies#97

Merged

landmanbester merged 86 commits intomainfrom fastim

Apr 16, 2024

+2,273-840

Commits on Mar 18, 2024

Persist mds in fastim worker
landmanbester
committed

Commits on Mar 19, 2024

Set up Hollywood shop in fastim worker
landmanbester
committed

Commits on Mar 20, 2024

don't limit OMP_NUM_THREADS. Set maxfev=2000 in fitgausspars
landmanbester
committed
no dirty in fastim
landmanbester
committed

Commits on Mar 21, 2024

add crude column arithmetic to fastim worker
landmanbester
committed
add RMS to blocker output
landmanbester
committed

Commits on Mar 25, 2024

add parallel flag to weight_data functions
landmanbester
committed
fix div by zero in fitcleanbeam
landmanbester
committed
use abs(model) for hard-thresholding in model2comps
landmanbester
committed
Don't place nan in fits header when restoring
landmanbester
committed
Merge branch 'issue94' into fastim
landmanbester
committed
typo compute_counts -> _compute_counts
landmanbester
committed
add add/subtract column functionality to init worker
landmanbester
committed

Commits on Mar 26, 2024

fix col arithmetic bug. use meta in eval_beam to avoid error message
landmanbester
committed
use client interface for dastim worker
landmanbester
committed
cast to same type
landmanbester
committed
add wait to block compute
landmanbester
committed
typo
landmanbester
committed
sum over counts axis 0, remove debug statements
landmanbester
committed
clean up fastim worker. Only allow single ms
landmanbester
committed

Commits on Mar 27, 2024

fix bug fmask logic bug in restore worker
landmanbester
committed
enforce single field selection, take first one by default
landmanbester
committed

Commits on Mar 28, 2024

use streamjoy for rendering videos
landmanbester
committed
add outname to video
landmanbester
committed
fix for empty slices
landmanbester
committed
add cmap and text-colour options
landmanbester
committed

Commits on Mar 31, 2024

rechunk when writing mds
landmanbester
committed
add missing DaskMSStore import
landmanbester
committed
use open_zarr directly
landmanbester
committed
Do not attempt to log outside of main thread
landmanbester
committed
Do not attempt to log outside of main thread
landmanbester
committed
test scattering mds
landmanbester
committed
call compute in mds inside worker
landmanbester
committed
try persist then scatter
landmanbester
committed

Commits on Apr 3, 2024

don't submit mds
landmanbester
committed
set LD_LIBRARY_PATH in set_client
landmanbester
committed
change smoovie defaults
landmanbester
committed
clone nodes before submit
landmanbester
committed
open mds in single_stokes_image
landmanbester
committed
return zarr store from single_stokes_image
landmanbester
committed
typo
landmanbester
committed
yet another typo
landmanbester
committed
don't delete mds
landmanbester
committed
persist and scatter mds
landmanbester
committed
use sync scheduler inside worker
landmanbester
committed
add spw and scan to outname
landmanbester
committed
make gridder more verbose
landmanbester
committed
set OMP_NUM_THREADS
landmanbester
committed
reinstate sync scheduler when computing on worker. Decrease gridder verbosity
landmanbester
committed
use as_completed to limit the number of tasks that get submitted
landmanbester
committed
nfutures = nworkers*nthreads_dask
landmanbester
committed
no restrictions
landmanbester
committed
wait for futures
landmanbester
committed

Commits on Apr 4, 2024

use worker_client in submit
landmanbester
committed
breakpoint at the end to keep cluster alive
landmanbester
committed
old approach with worker_client and direct_to_workers=True
landmanbester
committed
test memory footprint with direct_to_workers=False
landmanbester
committed
test memory footprint with dask.compute instead of worker_client
landmanbester
committed
as_completed with excess futures
landmanbester
committed
wait in batches
landmanbester
committed
retry with worker saturation as context manager
landmanbester
committed

Commits on Apr 5, 2024

try with semaphore
landmanbester
committed
flat list to use with as_completed
landmanbester
committed
allow dumping models to json
landmanbester
committed

Commits on Apr 6, 2024

fix nearest band logic in smoovie
landmanbester
committed

Commits on Apr 8, 2024

allow different degrid resolution in fastim worker
landmanbester
committed
begrudgingly add "progressbar" to fastim worker
landmanbester
committed

Commits on Apr 9, 2024

correct logic to combine subbands in smoovie
landmanbester
committed

Commits on Apr 10, 2024

add respect-scan-boundaries option and fix model2comps out-format bug
landmanbester
committed

Commits on Apr 11, 2024

use open_zarr to open mds in grid
landmanbester
committed
fix merge conflicts
landmanbester
committed
depend on stimela_upgrade qc branch
landmanbester
committed
typo
landmanbester
committed
typo
landmanbester
committed
add use-best-model option to grid worker and compute residual based on whether model is present or not. use open_zarr in degrid worker. check paths with fsspec
landmanbester
committed
save intermediary component models. keep track of niter to avoid overwriting previous runs
landmanbester
committed
correct column arithmetic in fastim
landmanbester
committed
try writing stream directly
landmanbester
committed
remove optimize if writing mp4
landmanbester
committed

Commits on Apr 12, 2024

don't pass freqs for null subbands into modelfit
landmanbester
committed

Commits on Apr 13, 2024

filter extremely empty cells by default
landmanbester
committed
set extremely empty cells to median
landmanbester
committed
change default filter-level
landmanbester
committed

Commits on Apr 15, 2024

use bfgs for fitting psf
landmanbester
committed
depend on codex-africanus master
landmanbester
committed
remove copyto in psf_convolve_cube
landmanbester
committed