How to map over a dataframe in ballista and return a dataframe

[protometa]

How can I use Ballista to call a (potentially async) function to map rows (or I guess record batches) and return a new transformed dataframe?

I understand there are ways to create UDFs in Datafusion but I don't see how to register or use them in the Ballista context. Also don't know if any UDFs can be async. I tried calling execute_stream_partitioned and mapping over those streams and wrapping them in PartitionStreams to use in a new StreamingTable to register with Ballista, but it won't compile (mainly I think due to SendableRecordBatchStream not being Sync and PartitionStream requiring it).

Is there an example of this anywhere?