Minimize columns used by altMTL

[sbailey]

Notes from the CAI call on 2025-01-16:

Full fiberassign calls inflate_ledger to get additional targeting columns like fluxes to propagate to the offline pipeline, but these aren't needed for the altMTL process. Investigate the minimal required set of columns and whether those can be achieved using just the MTLs without needing inflate_ledger.

From @araichoor in zoom chat:

For what is worth, I tried a call with fba_run with only those 18 columns, and it seems to run:
"TARGETID", "RA", "DEC", "REF_EPOCH", "PARALLAX", "PMRA", "PMDEC", "DESI_TARGET", 
"BGS_TARGET", "MWS_TARGET", "SUBPRIORITY", "OBSCONDITIONS", "PRIORITY_INIT", 
"NUMOBS_INIT", "SCND_TARGET", "NUMOBS_MORE", "NUMOBS", "PRIORITY”; 
it s very likely that those are in the ledgers.

To which @geordie666 replied

Yep, those 18 columns are all in the ledgers already!

@LNapolitano @ashleyjross heads up

[geordie666]

The broader point being that we may be able to completely bypass calls to inflate_ledger(), which appears to be an I/O bottleneck.

[araichoor]

for reproducibility, here s a dummy example:

first I read the ledgers, and cut on those columns:

from desitarget.io import read_targets_in_tiles

hpdir = "/global/cfs/cdirs/desi/survey/ops/surveyops/trunk/mtl/main/dark"
keys = ["TARGETID", "RA", "DEC", "REF_EPOCH", "PARALLAX", "PMRA", "PMDEC", "DESI_TARGET", "BGS_TARGET", "MWS_TARGET", "SUBPRIORITY", "OBSCONDITIONS", "PRIORITY_INIT", "NUMOBS_INIT", "SCND_TARGET", "NUMOBS_MORE", "NUMOBS", "PRIORITY"]

t = Table.read("/global/cfs/cdirs/desi/survey/fiberassign/main/001/001000-tiles.fits")
d = Table(read_targets_in_tiles(hpdir, tiles=t, mtl=True))
d = d[keys]
d.write("/global/cfs/cdirs/desi/users/raichoor/tmpdir/001000-targ.fits",overwrite=True)

then I call fba_run:

source /global/common/software/desi/desi_environment.sh main
export SKYHEALPIXS_DIR=$DESI_ROOT/target/skyhealpixs/v1
fba_run --targets /global/cfs/cdirs/desi/users/raichoor/tmpdir/001000-targ.fits --footprint /global/cfs/cdirs/desi/survey/fiberassign/main/001/001000-tiles.fits --dir /global/cfs/cdirs/desi/users/raichoor/tmpdir/

[...]
INFO: Tile 1000: Final assignments: SCIENCE: 4264, SCIENCE not STANDARD: 4145, STANDARD: 120, SKY: 434
[...]