Writing parsers that perform additional validation/verification of inputs

[zegelin]

I'm experimenting with writing a custom DSL for a toy project. Initially I started working with Chumsky, but then discovered winnow, and after playing around with it a bit I prefer its "toolbox" approach, and some of the usability features (ranges of chars as token sets, etc).

However, one feature I really like about Chumsky is its validate function, which can be used to add additional checks that can emit error messages. Combine that with ariadne for really nice error messages. While I could defer these checks until after the parse, having them be part of the parser makes it easy for me to save location/span info and then output nicely formatted contextual error messages.

For example, I'm trying to rewrite the following Chumsky parser in winnow:

fn prometheus_label_name() -> impl Parser<char, String, Error = Simple<char>> {
    // see https://prometheus.io/docs/concepts/data_model/#metric-names-and-labels
    // ident() matches the Prometheus metric label name regex.

    ident()
        .validate(|ident: String, span: Range<usize>, emit| {
            if ident.starts_with("__") {
                emit(Simple::custom(span.start..span.start + 2, format!(r#"label names starting with "__" are reserved for use by Prometheus"#)));
            }

            ident
        })
        .labelled("Prometheus label name")
}

(where ident() is equivalent to '[a-zA-Z_][a-zA-Z0-9_]*')

Outputs (via ariadne):

Error: label names starting with "__" are reserved for use by Prometheus
   ╭─[<unknown>:1:1]
   │
 1 │ __foo
   │ ─┬  
   │  ╰── label names starting with "__" are reserved for use by Prometheus

And here's my winnow version so far:

pub fn prometheus_label_name<'s>(input: &mut winnow::Located<&'s str>) -> winnow::PResult<&'s str> {
    (
        one_of(('a'..='z', 'A'..='Z', '_')),
        take_while(0.., ('a'..='z', 'A'..='Z', '0'..='9', '_'))
    )
    .recognize()
    .verify(|ident: &str| {
        !ident.starts_with("__")
    })
    .parse_next(input)
}

However, verify doesn't offer any way to return/emit errors.
I had a quick look at try_map, and think that might be the right path. However, this is where I paused to open this discussion thread to ask a few questions.

• What's the best way to add custom messages to the error from within verify/try_map?
• How do I track spans (start + end offsets) so I can pass this to ariadne (or miette) to highlight not just the offset but the full offending input?
• Errors can occur after parsing when the DSL gets evaluated. How do I track the spans from input so I can provide nicely formatted error messages with location info? Use with_span() and store that in my ast nodes?
  For example, I'd love to be able to print messages like this:

Error: 
   ╭─[<unknown>:1:1]
   │
 1 │ let foo = 5 / 0;
   │           ──┬──  
   │             ╰──── divide by 0 while evaluating expression
───╯

[epage]

Yes, capturing the spans during normal error paths is a weakness. The challenge is dealing with the wide variety of different needs, e.g. #180 and #231. These can be worked around by using Parser::and_then or not using combinators for all of the code. In general, Winnow's design reject's Chumsky's notion that everything should and must be done through combinators.