partition_info ordering and partition ordering

[smcguire-cmu]

In dask dataframes, the partitions should be ordered so that the hipscat index increases between them. In some methods such as polygon and cone filtering, the partitions are ordered by the partition_info of the filtered hipscat catalog. Currently, the hipscat catalog filtering generates the partition_info from the pixel tree, so the partitions are in the correct order. However, in newly imported catalogs the partition_info is sorted first by order then by pixel number. These are inconsistent, and could cause issues in assuming ordering if we make any changes.

[hombit]

It may be related to astronomy-commons/hats#163 and astronomy-commons/hats#168

[delucchi-cmu]

@smcguire-cmu - Can you confirm that you're seeing this kind of inconsistent behavior in the wild?

We're sorting the the pixels by nested healpix ordering during the dataframe catalog loading process, and I'm not able to create a failing unit test to start development off of.

https://github.com/astronomy-commons/lsdb/blob/main/src/lsdb/loaders/dataframe/dataframe_catalog_loader.py#L140