Standard Scaler fit-transform interface

[santiago-imelio]

There is already Preprocessing.standard_scale that performs this transformation. However, with this function we cannot standardize a test set using the train set distribution data (mean and std deviation).

Scaler.StandardScaler.fit/2 will return a struct containing the mean and standard deviation of the distribution of learned observations.

[josevalim]

@msluszniak should we generally convert our preprocessing steps into modules to mirror scikit learn?

[msluszniak]

I think we shouldn't convert preprocessing into modules. For functions that calculate some metrics and then apply them to data, it would be easy to make this interface. However, for functions like one-hot encoding or ordinal encoding, we would need to somehow map the relation key -> calculated_mapping. In Scikit they use dictionaries. One of the functions - normalize has the equivalent in Nx.Linalg.norm. If we decide to support standard_scaler, max_abs_scaler, min_max_scaler, and binarizer then the interface would be inconsistent. If we change the interface to use struct users wouldn't be able to pipe the results of standard_scale. But on the other hand, it would be cool to support both.

[msluszniak]

What we can actually do is create separate functions like standard_scale(reference_tensor, tensor_to_apply, opts) and support the application of standardization on a different tensor, WDYT?

[msluszniak]

And the standard_scale/2 is a simplified version of this because the reference tensor is the same as tensor_to_apply

[josevalim]

@msluszniak the downside of this approach is that we would not be able to fit once and apply the same fitting multiple times. Is that a concern?

[msluszniak]

You're right, hmm. In a standard use case, this function won't be called multiple times. But if so, then the benefit from modular implementation might be considerable.

[santiago-imelio]

@msluszniak From what I understand, it seems too soon to agree on an interface that fits both the language and the needs of developers, so in my opinion keeping the preprocessing API functional seems to be appropriate for now.

Closing in favor of continuing the discussion.

[josevalim]

I have reopened this PR in case you would like to address the comments in it. :)

[msluszniak]

We also need to add docs for fit, transform, and fit_transform