BLD: cleanup unused build-time dependencies

[neutrinoceros]

Replaces #200

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Comparison is base (648c861) 78.83% compared to head (b6e5110) 78.83%.

Additional details and impacted files

@@           Coverage Diff           @@
##             main     #201   +/-   ##
=======================================
  Coverage   78.83%   78.83%           
=======================================
  Files          10       10           
  Lines         992      992           
=======================================
  Hits          782      782           
  Misses        210      210           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[pllim]

LGTM thanks!

devdeps failure is unrelated and since synphot is involved, I probably should open a follow-up PR to fix that.

(The infrastructure not very clean in this repo.)