Question: unwrapping nested monad in Eff/Aff

[aloslider]

How do you work with different nested monads within an Eff/Aff? I've read about tranformer types and checked Transformers package, but didn't find anything that looks like it could help me to unpack A from an Eff<Option<A>> and fail if its None.
Currently, my solution looks like this:

public static Eff<Unit> doWork =>
    ...
    from opt in getContent // gets Option<A>
    from str in opt.ToEff() // unpacks option and fails if its None
    ...

static Eff<Option<string>> getContent =>...

It does what I want, but I assume maybe there's an easier built-in way to do the same in one line?

(Btw, I found mentions about language-ext v5, is there any list of upcoming changes?)

[louthy]

The way that works for all nested monads is this:

var mox = SuccessEff(Some(100));
var moy = SuccessEff(Some(200));

var mr = from ox in mox             // Extract the Some(100)
         from oy in moy             // Extract the Some(200)
         let rs = from x in ox      // Extract the 100
                  from y in oy      // Extract the 200
                  select x + y      // Repackage as Some(300)
         select rs;                 // Repackage as Eff(Some(300))

Because each LINQ block can only work with one monad at a time, you must extract the inner monads and the use them in a LINQ sub-block.

I've read about tranformer types and checked Transformers package, but didn't find anything that looks like it could help me to unpack A from an Eff<Option> and fail if its None.

The transformer generated-code system can't quite handle Aff and Eff for boring reasons that's not worth explaining, but you can support them with manually written extensions.

The critical thing is to implement Bind for the nested monads:

public static class EffOptionExt
{
    public static Eff<Option<B>> BindT<A, B>(
        this Eff<Option<A>> mma, 
        Func<A, Eff<Option<B>>> f) =>
        from mmb in mma.Map(ma => ma.Map(f))
        from mb in mmb.Match(
            Some: identity, 
            None: () => SuccessEff(Option<B>.None))
        select mb;

   ...
}

It's worth breaking it down so you understand what's happening:

from mmb in mma.Map(ma => ma.Map(f))

This line extracts (as ma) the Option<A> from the Eff<Option<A>>, then it calls Map on the Option<A> with the provided bind function f. But note, that returns Eff<Option<B>>, so the result of the call to the inner Map is Option<Eff<Option<B>> and if we returned that we'd get Eff<Option<Eff<Option<B>>>.

from mb in mmb.Match(
    Some: identity, 
    None: () => SuccessEff(Option<B>.None))

So, the next line matches on the Option<Eff<Option<B>> (which allows us to dissolve the outer Option<>. So the result of this is Eff<Option<B>>. The from will extract the value from within the Eff and so mb is Option<B>.

select mb;

That allows the next line to pack up the Option<B> into the Eff:

We could also write that like so:

    public static Eff<Option<B>> BindT<A, B>(
        this Eff<Option<A>> mma,
        Func<A, Eff<Option<B>>> f) =>
        mma.Map(ma => ma.Map(f))
           .Bind(mmb =>
               mmb.Match(
                   Some: identity,
                   None: () => SuccessEff(Option<B>.None)));

Which is a little more efficient, but maybe not as easy to read.

Once you have BindT defined, you can implement pretty much everything else in terms of BindT:

    // Alternative BindT that allows you to return an Option<B> rather 
    // than Eff<Option<B>>
    public static Eff<Option<B>> BindT<A, B>(
        this Eff<Option<A>> mma,
        Func<A, Option<B>> f) =>
        mma.BindT(a => SuccessEff(f(a)));

    public static Eff<Option<B>> MapT<A, B>(
        this Eff<Option<A>> mma, 
        Func<A, B> f) =>
        mma.BindT(a => Some(f(a)));

    public static Eff<Option<A>> FlattenT<A>(
        this Eff<Option<Eff<Option<A>>>> mma) =>
        mma.BindT(identity);

    public static Eff<Option<B>> Select<A, B>(
        this Eff<Option<A>> mma, 
        Func<A, B> f) =>
        mma.BindT(a => Some(f(a)));

    public static Eff<Option<B>> SelectMany<A, B>(
        this Eff<Option<A>> mma, 
        Func<A, Eff<Option<B>>> f) =>
        mma.BindT(f);

    public static Eff<Option<C>> SelectMany<A, B, C>(
        this Eff<Option<A>> mma, 
        Func<A, Eff<Option<B>>> bind, 
        Func<A, B, C> project) =>
        mma.BindT(x => bind(x).MapT(y => project(x, y)));

That will give you the shortcutting behaviour that you're expecting. But, it begs the question: why nest Option inside an Eff if you could instead just return FailEff(Errors.None) instead of FailEff(Option.None).

Essentially what I am saying is that the Eff monad already has optional behaviour built-in. So you may find it much easier to use that rather than use nested monads. And for the functions that return Option you can call yourOption.ToEff() to convert to an Eff (same with Aff).

Calling ToEff(Error) also allows you to give more context to the failure, which is obviously important for the wider expression. For example:

   parseInt("123").ToEff(Error.New("expected integer value"))

Calling parseInt and getting None back is fine, but for larger blocks of code you want to know why it failed, so None should often be replaced with something useful.

(Btw, I found mentions about language-ext v5, is there any list of upcoming changes?)

As you've seen nested monads are still a little awkward to work with in C#. Part of the future roadmap will be to support transformer stacks like Haskell. Essentially you create new monadic types that are nested compositions of existing monads.

This is still a little ways off. I will be making a big announcement about the future roadmap in the next few days. But, as people on social media like to say: there are big things coming! :)

[louthy]

There was an earlier version of this where I suggested that None wouldn't shortcut the transformer-monad, but I was wrong. I was just having an 'it's an hour before I go to bed and I'm tired' moment. It will shortcut as expected.

[aloslider]

Wow, thanks for such a quick and detailed response! I really appreciate your example with describing each logical step, it helps a lot.

But, it begs the question: why nest Option inside an Eff if you could instead just return FailEff(Errors.None) instead of FailEff(Option.None).

In my first example, I desided to simplify it as much as possible, but guess the context matters here: let's say I get some data from the function I can't change, and I need to check if its some important fields are not null and transform it to aproppriate form for my application to do other work with it:

record SomeData(string? A, string? B, string? C);
record TransformedData(string A, string C);

static Eff<Unit> doWork =>
    from data in getData
    from transformedData in mapData(data).ToEff() // transformedData is A
    from res in process(transformedData) // now I can use A from Option<A>
    ...

static Eff<SomeData> getData => Eff(GetData);

// Some other function I can't change, so result fields may be null in some cases (here they all are)
static SomeData GetData() => new SomeData(null, null, null);

static Option<TransformedData> mapData(SomeData d) =>
    from d in Optional(data)
    from a in Optional(d.A)
    from b in Optional(d.B)
    select new TransformedData(a, b);
    
static Eff<int> process(TransformedData d) => ...

It may look like a simple null-parsing, but again, its just another simplified example and I hope it made the situation clear. Maybe there's a simplier approach to this problem, but this is the easiest way I came up with.

Essentially what I am saying is that the Eff monad already has optional behaviour built-in.

Yes, as I understand, you're talking about sequential (chaining) computations, but there may be situatuions where you must work with, lets say, Option inside an IO and maybe want them to fail entire outer IO computation with some error, if there's some None value showed up, isn't it? Do I misconcept something or/and overenginer?
For now, I'm aware of only one way of doing that: by lifting an Option inside Eff/Aff, but eventually you get this nested monad and I thought you need some transformer one to work with. Other way, you just match it and process each case of some monad state explicitly like this:

static Eff<Unit> doWork2 =>
    from data in getData
    from optTransformedData in Eff(() => mapData(data)) // optTransformedData is Option<A>
    from transformedData in // transformedData is A
        optTransformedData.Match(
            Some: td => SuccessEff(td),
            None: FailEff<TransformedData>(DataIsNullError))
    from res in process(transformedData) // now I can use A from Option<A>
    ...

When it looks OK in a small example, I think it would be a mess to work with a large computations, even if I break them into small ones.

And for the functions that return Option you can call yourOption.ToEff() to convert to an Eff (same with Aff).

Yes, I think I'll stick with this approach for now as it solves my problem well, but will try to create some extensions based on your example.

there are big things coming! :)

That's cool, can't wait to see changes!

[louthy]

Ultimately, it's completely legitimate to work with Eff<Option<A>>, just like Option<Option<A>> is not the same as Option<A>.

Some(A) | None != Some(Some(A)) | Some(None) | None): there are more states, which may well mean something in the logic of the program. Reducing the states may make sense or it may not, it really is dependent on the code and its purpose. Only you can make that judgement. There are always ways to wrap up complex behaviour whilst maintaining the correct data structures.

Another thing to note with transformers is that, by using the LanguageExt.Transformers package, you're forced into nested monads being used as transformer-stacks in LINQ expressions. In Haskell it's an opt-in for example:

    -- | This is the transformer stack
    type Eff a = MaybeT IO a

    -- | This is just a `Maybe a` nested as the bound variable of an IO monad
    type Eff a = IO (Maybe a)

The first one (MaybeT IO a) supports lifting of IO functions into the transformer stack. Things like do notation automatically run the nested bind operators (as I showed in the code above for Eff<Option<A>>.BindT). The second one (IO (Maybe a)) will only run the outer IO monad in a do expression.

This gives the Haskell programmer control over what is stacked and what is not. That's why I'd eventually like to deprecate the LanguageExt.Transformers project for a code-gen solution where bespoke transformer-stacks can be built and the default doesn't try to automatically bind nested monads.

I slightly regret adding the LINQ operators (SelectMany, Select, Where) into the LanguageExt.Transformers project - because there's no option to not use them. Perhaps a future version could put them in their own namespace so you have to import the using manually.

[Legys]

I've stuck with a very similar question.
Started to learn conceptions and the library recently, and from what I've learned there are very neat patterns - short-circuiting.
Has it ever meant to work with LINQ Syntax?

For a given code:

var result = from _ in RegisterAsync("[email protected]", "password")
                   from user in ByEmailAsync("[email protected]")
                   select GenerateFor(user);

var finalResult = await result;

I'm getting an error at the line "select GenerateFor(user)":
Argument 1: cannot convert from 'Either<TextError,User>' to 'User'

I.e. it can't unpack Right from Either, whereas I expect it to do due to short-circuiting when if Left occurs it just returns the value as Left to result.
It kinda of works with Bind. LINQ Syntax looks more abstract, or do I not really care about it?

var result = await
    _registerUser.RegisterAsync(request.Email, request.Password)
    .BindAsync(_ => _fetchUser.ByEmailAsync(request.Email))
    .BindAsync(user => _generateToken.GenerateFor(user));
    ```
No errors in that way.
Excuse me for not opening a separate discussion, I think the context matters.
Also, don't mind the code flow, I know it's a bit messy. Just ignore it, it will have a better data model.

[louthy]

You to show the types of the functions.

[Legys]

Oh sorry, indeed forgot to attach the most important thing.

1. public async Task<Either<TextError, Task>> RegisterAsync(string email, string password)
2. public Task<Either<TextError, User>> ByEmailAsync(string email);
3. public Either<TextError, string> GenerateFor(User user);

[louthy]

You have a couple of issues:

1. You're mixing monadic types. Task<Either<L, R>> is not the same as Either<L, R>
2. select is the operator for repackaging the value, not for extracting the value (which is what from is).

So, ideally you'd do this:

var result = from _ in RegisterAsync("[email protected]", "password")
             from u in ByEmailAsync("[email protected]")
             from r in GenerateFor(u)
             select r;

But, because GenerateFor(u) returns a Either<> not a Task<Either<>> it won't work. You can convert it to a Task<Either<>> like so:

var result = from _ in RegisterAsync("[email protected]", "password")
             from u in ByEmailAsync("[email protected]")
             from r in GenerateFor(u).AsTask()
             select r;

However, because LINQ doesn't follow the exact same rules for resolving the SelectMany function as the rest of C#, it gets confused about which SelectMany extension to use. And so, to make working with nested Task<M<>> monads (where M is the inner monad), I have built *Async variants of most of the monadic types. So, EitherAsync<> is equivalent to Task<Either<>>. It's nearly always easier to use those over Task<M<>>.

You can convert a Task<Either<>> to a EitherAsync<> by calling ToAsync() on it. You can also convert an Either<> to an EitherAsync<> by calling ToAsync() on it.

So, to make your example work in LINQ, call this:

var result = from _ in RegisterAsync("[email protected]", "password").ToAsync()
             from u in ByEmailAsync("[email protected]").ToAsync()
             from r in GenerateFor(u).ToAsync()
             select r;

The first two lines convert from Task<Either<>> to EitherAsync<> and the third line converts from Either to EitherAsync (which unifies all of the from lines to the same monadic type).

If you don't want to be calling ToAsync() all the time then wrap your Task based API:

    public static EitherAsync<TextError, User> ByEmailAsync(string email)
    {
        return go().ToAsync();
        async Task<Either<TextError, User>> go()
        {
            // Just a placeholder implementation!
            await Task.Delay(100);
            return Right<TextError, User>(new User());
        };
    }

Then you can do this:

var result = from _ in RegisterAsync("[email protected]", "password")
             from u in ByEmailAsync("[email protected]")
             from r in GenerateFor(u).ToAsync()
             select r;

Usually, for any sub-system you're looking for the monadic type that captures all of the capabilities you need (in this case error handling and async behaviours). Then you build your sub-system around that type or 'smaller'. And by smaller I mean:

• EitherAsync has the maximum capability: values, errors, and async
• Either has less: values and errors
• Raw values that haven't been lifted into a monad, like int, or User - have the least capability: values.

So, it's like an onion of capability. When you write LINQ expressions you need to 'lift' values into the maximum of that expression. Luckily you get raw-values by default (by using let x = <raw value>), but for monadic values you need to make them match.

[Legys]

Thanks a lot. I did it.