Support preprocess as a validation layer

[rakshith-ravi]

Feature Request

Preprocess is a crate that offers similar functionalities to validator. It also allows you to process structs to filter out data, etc.

It primarily provides two main advantages over validator, among others:

• It works with enums as well
• It allows you to manipulate data while validating them (trim usernames, lowercase emails, etc).

Would it make sense for Loco to support preprocess as a feature flag?

Disclaimer: I am the maintainer of preprocess. It's used in our company, and we use it in production for services that receive more than 20 million requests a day. So bug fixes are usually pretty quick on our end.

[kaplanelad]

Thank you for sharing your library! Preprocess looks like a powerful tool, and we appreciate you bringing it to our attention.

After a quick review, it seems straightforward for users to incorporate Preprocess directly into their applications without requiring deeper integration into the Loco framework. For that reason, adding it as a feature flag within Loco might not be necessary. Users can independently add the crate and manage it at the controller level where needed.

That said, @jondot and I have been discussing a broader vision for Loco's validation layer. If your library aligns with the following goals, there’s a potential opportunity for us to consider it as a replacement for the current validator. Here’s what we’re aiming for:

Request validation with structured error handling and logging
We want a seamless way to validate and sanitize incoming requests and return meaningful error responses while also logging validation issues. For example, given a request like:

#[derive(Debug, Deserialize, Serialize)]  
pub struct LoginParams {  
    pub email: String,  
    pub password: String,  
}  

async fn login(State(ctx): State<AppContext>, Json(params): Json<LoginParams>) -> Result<Response> {  
   ...
}  

If the parameters are invalid, we’d like to generate a JSON error response similar to this:

{  
    "errors": [  
        {  
            "field": "email",  
            "message": "Invalid email format"  
        },  
        {  
            "field": "password",  
            "message": "Password is required"  
        }  
    ]  
}  

Note: This behavior should be integrated into the Loco framework using Axum layers. Users should only need to annotate their structs with attributes to enable this functionality. Furthermore, users should have the flexibility to disable detailed error messages, opting instead for a simple status code response if preferred.

Model validation integration
Currently, we use the validator crate to validate models, providing users with an intuitive way to define and enforce validation rules directly within their data structures. You can see examples of how this works in:

• Validation module in Loco
• Demo: User model

Could Preprocess also be used effectively, not only for request scenarios like these?

If your library can address these needs while offering additional benefits, we’d be open to exploring deeper integration.

[rakshith-ravi]

Yup, indeed! This makes perfect sense. Preprocess currently returns one field's error at a time (I internally use a ? to bail out). But adding the ability to return all errors shouldn't be too difficult. Would be happy to add that functionality

[kaplanelad]

Great,
Here’s an example of a login request where we expect mandatory email and password parameters. Below is how I envision this working with preprocess:

#[preprocess::sync]
#[derive(Debug, Deserialize, Serialize)]  
pub struct LoginParams {  
    #[preprocess(trim, lowercase, email)]  
    pub email: String,  
    #[preprocess(trim, length(min = 8))]  
    pub password: String,  
}  

async fn login(State(ctx): State<AppContext>, Json(params): Json<LoginParams>) -> Result<Response> {  
  let processed_value = raw_value.preprocess()?;
   ...
}  

Currently, would I need to explicitly call raw_value.preprocess()? within the controller? Ideally, this step should be abstracted away and executed automatically within an Axum middleware or layer. This way, the framework handles validation seamlessly behind the scenes, and users only need to define how the data should be processed on the struct.

For instance, if a user configures rules directly on the struct fields and receives those fields in the controller, the validation should automatically occur without additional user intervention. This would significantly enhance the developer experience by eliminating the need for explicit validation calls in the controller.

Additionally, if a required field (e.g., email) is missing from the payload, Axum should automatically prevent execution of the controller logic. The framework should reject the request early with a proper validation response.

[rakshith-ravi]

If you use a tower service, you can automatically do this as a middleware on the service. Any struct that uses preprocess implements the Preprocessable trait and that has an associated type called Processed (Reference). You can use that to implement the layer.

Here's a simple example of how we do it:

https://github.com/patr-cloud/patr/blob/develop/api/src/utils/layers/preprocess_handler.rs#L117-L125

The way it's defined is here:

https://github.com/patr-cloud/patr/blob/develop/api/src/app.rs#L126

Does this help?

[kaplanelad]

Thank you!
We will continue using the validator crate as it is already integrated into Loco's request (custom axum extractors) and model layers and provides the same functionality. I do not see much difference between the creates

For users who wish to adopt your preprocess, I believe they can do so easily without requiring any changes to the Loco framework.

[rakshith-ravi]

Would a Axum extractor for these structs help?

I'm wondering what I can add to help make it easier for people to use

[kaplanelad]

You can find our implementation of the extractor here.

Regarding the help :), we can start by converting our validator into a trait. This approach would fully encapsulate the validator's creation process.
Once this is completed, we can provide users with an easy way to replace the default validation crate, for example, by preprocessing. and they would only need to implement the required functions from the LocoValidation trait.