You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
We already have support for conversion between mlx_rs::Array and safetensors::tensor::TensorView, so supporting this wouldn't be hard. However, there might be an asymmetry in the saving/loading API due to the lack of public API to create an SafeTensors from Array. More specifically, the only public API to create SafeTensors is SafeTensors::deserialize(buf), where buf is &[u8].
#178 provides a partial fulfillment of this feature, and both saving and loading would be dealing with a impl AsRef<Path>. The parts that is missing in #178 are
Performance of loading weights. Loading weights in the mistral example is two times slower than the original python example
We already have support for conversion between
mlx_rs::Array
andsafetensors::tensor::TensorView
, so supporting this wouldn't be hard. However, there might be anasymmetry
in the saving/loading API due to the lack of public API to create anSafeTensors
fromArray
. More specifically, the only public API to createSafeTensors
isSafeTensors::deserialize(buf)
, wherebuf
is&[u8]
.This could end up with some API look like below
where we have an asymmetry that we can only save to a file rather than a
SafeTensors
object.Or we could do something similar to
candle-nn
where both loading and saving take aPath
to a safetensors fileThe text was updated successfully, but these errors were encountered: