-
Notifications
You must be signed in to change notification settings - Fork 6
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Generate margin cache for ZTF DR22 on Epyc #460
Comments
Using this script to create the margin cache: from pathlib import Path
from dask.distributed import Client
from hats_import.pipeline import pipeline_with_client
from hats_import.margin_cache.margin_cache_arguments import MarginCacheArguments
def main(main_catalog_path: Path):
main_catalog_name = main_catalog_path.name
main_catalog_dir = main_catalog_path.parent
margin_cache = f"{main_catalog_name}_10arcs"
args = MarginCacheArguments(
input_catalog_path=main_catalog_path,
output_path=main_catalog_dir,
margin_threshold=10.0,
output_artifact_name=margin_cache,
)
with Client(
n_workers=16,
memory_limit="64GB",
threads_per_worker=2,
local_directory="/epyc/data3/hats/tmp",
) as client:
pipeline_with_client(args, client)
if __name__ == "__main__":
main(
Path("/data3/epyc/data3/hats/catalogs/ztf_dr22/ztf_lc"),
) |
Getting many instances of this sort of error:
So far I have only seen it happen with the This appears to be an instance of #458 . |
With a proposed fix to #458 to convert RA and DEC to double-precision for radec2pix, the margin-cache generation succeeded:
Final margin cache is at |
@gitosaurus hats-imported ZTF DR22 on Epyc (see #452), it also would be great to have a margin cache for that catalog.
Follow
docs/guide/margin_cache.rst
withmargin_threshold=10.0
and call the output margin catalog{main_catalog_name}_10arcs
The text was updated successfully, but these errors were encountered: