Allow options via environment variables

[mrak]

testOption :: Parser String
testOption = strOption
     ( long "test"
    <> env "PROGNAME_TEST"
    <> short 't'
    <> metavar "VAL"
    <> help "For testing purposes only")
    <|> pure "default value"

I don't know if the environment variable choice should display in the help text, but it could look something like this:

Usage: progname [-t|--test VAL]

Available options:
  -h,--help                Show this help text
  -t,--test VAL            For testing purposes only

Environment variables:
  PROGNAME_TEST=VAL        For testing purposes only. Overridden by -t,--test

If you don't think this should be a responsibility of this library, what would be a good alternative to getting environment variable options that are possibly overridden by command line options?

Awesome library by the way

[pcapriotti]

I don't have a good answer for this.

One thing you can do is read your environment variables before you create the parser, and set their values as defaults. That would give you the correct behaviour, but no documentation, of course.

I've actually been working on a big refactoring to make the creations of parsers a lot more modular. That would accommodate this use case as well (it was actually one of the motivating examples).

At the moment, there is really no good way to do this, as far as I can tell.

[brendanhay]

I've been using something like this as a stopgap:

type Env = [(Text, Text)]

class FromText a where
    fromText :: Text -> Either String a

instance FromText Text where
    fromText = Right

environ :: (HasValue f, FromText a) => Text -> Env -> Mod f a
environ k env = maybe idm value . join $ parse <$> lookup k env
  where
    parse = either (const Nothing) Just . fromText

textOption :: FromText a => Mod OptionFields a -> Parser a
textOption = option (eitherReader (fromText . Text.pack))

With the usage being something like:

data Foo = Foo Text

parser :: Env -> Parser Foo
parser env = Foo
    <$> textOption
        ( long "var"
       <> metavar "VAR"
       <> environ "VAR" env
       <> help "Some variable to lookup."
        )

That way the environment can be retrieved using System.Environment.getEnvironment, massaged, and passed in when the parser is constructed.

[mrak]

It's been a few months. Has there been any movement on this?

[voidzero]

I'm not sure why a module that parses options (in the getopts sense) should do anything with the environment. Isn't it outside the scope for this module?

[gambogi]

I think that the reason this has been asked for is it feels like a very natural place to specify environment variable configuration.

Many tools have very tight correspondence between environment variables and command line arguments. Certain tools (like Python's pip) have environment variables that directly correspond to the cli options. It seems like the cleanest implementation would be to simply add environ or some other name to the cadre of annotations we have for describing the configuration of an executable.

[phadej]

After thinking about this, I'd suggest something like

testOption :: Parser String
testOption = strEnv
    ( envvar "PROGNAME_TEST"
    <> metavar "VAL"
    <> help "For testing purposes only")

i.e. introduce EnvVarFields etc.

Because even the most common case is that environment variable provides a default for the flag value, it's not always the case. Even the choice whether envvar or flag has higher precedence.

Yet doc generator could be smart about those special common cases:

testOption :: Parser String
testOption = strOption (...) <|> strEnv (...)

I can submit a patch, as it's probably going to be quite straight-forward, if the approach is ok.
Not for the fancy doc stuff though.

[HuwCampbell]

There's a few options right, my biggest concern is that we would need to change execParserPure to include an association list of environment variables.
But a lot of people use that function directly. So it's a big breaking change, and to really get on board, I would like a big step up on functionality beyond what @brendanhay's solution (and other similar I've seen) adds.

[phadej]

@HuwCampbell it's not necessary to change execParserPure type, it can assume empty enviroment. We could have

execParserPure = execParserPureWithEnv mempty

[HuwCampbell]

Yep, true, I was more thinking about the core runner not the name.

[Pitometsu]

Is there any working implementation of such feature yet?

[roman]

I implemented a library (that uses optparse-applicative internally) to allow gathering configuration values from multiple sources, you may want to take a look and see if this is what you need

[ecthiender]

What is the status on this? Is this feature going to get merged?

[alexjg]

Hey folks, is there any progress on this or any help that I could offer? I recently implemented a similar feature in http://ben.kirw.in/decline/ for Scala and have been missing it when hacking on personal projects in Haskell.

[andrewthad]

This would be rather useful. For a number of applications that I develop, I use them both at the command line (for playing around with the tool I've built interactively) and in a systemd service. The problem is that systemd services don't like arguments. They like environment variables. This feature would help improve this situation for me.

It would be great if the maintainer could offer a decision on this. If it's not going to happen, this should be closed. Otherwise, it appears that there are capable individuals willing to implement this.

[HuwCampbell]

I'm totally open to suggestions here, this issue isn't a won't fix by all means.

I personally use something similar to what @brendanhay suggested; and I could imagine providing a combinator to do something along these lines.

If that's not satisfactory, what additional things would you like to see?

[andrewthad]

I like the suggestion you referenced, although I strongly prefer that the environment become available at the time that execParser is run, not at the time that the parser is constructed. I think leaving the type of execParserPure alone (for the purpose of backwards compatibility) is probably a good way to go. And then execParserPureWithEnv could be added.

One thing that didn't get discussed at length was the question of precedence. That is, in the event that both the environment variable and the flag are set, who wins. @phadej discussed this a little. The way I see it, there are two axes:

• Decision in the presence of multiple values: prefer command-line flag, prefer environment variable, fail on conflict.
• Locality of decision-making process: Per-flag, global

I believe it is possible to accommodate all of these. ParserInfo could have a default decision-making process (a per-application global default), and this could be overridden by each option.

[andrewthad]

Any feedback on my suggestion above? Even just a yes or no. I have the availability to implement this, and at work, I keep running into situations where I work around the lack of this feature. However I end up doing this, I'd just like to make sure that it'll be accepted upstream since I'd rather not maintain a fork of optparse-applicative forever.

[HuwCampbell]

Yes, I would be quite happy to see something done here. I can't guarantee a merge, but I will work with you to try and make it happen.

[psibi]

Any updates on this issue ? I like the one suggested by Oleg, but having any kind of support for having environment variables would be nice.

[and-pete]

This feature would be a very nice to have. Roman's etc mentioned above is a nice package, but a little heavyweight for me vs just having this library read environment variables too :)

[stevladimir]

I sketched up the solution here.

It implies changes to core parser type, but tried to keep changes to API as minimal as possible. Atm it also lacks necessary changes to parser docs generation and maybe some other things.

Are you interested in these changes?

[yairchu]

For git-mediate I thought that it would be suitable to have a single GIT_MEDIATE_OPTIONS environment variable that can have options in the same format as the command line options.

It's implemented in a simple module here: https://github.com/Peaker/git-mediate/blob/20b3277bcafa0a94138468c35ee738225cf63792/src/OptUtils.hs

That would be a nice way to go IMHO. Btw my implementation currently lacks notifying which options from the environment variable weren't recognized.

[cdepillabout]

@NorfairKing recently released a library somewhat similar to optparse-applicative, that allows easily reading application configuration from the command line, environment variables, and/or files on disk:

• https://cs-syd.eu/posts/2024-07-08-announcing-opt-env-conf
• https://github.com/NorfairKing/opt-env-conf

[yairchu]

@cdepillabout Thanks for the pointer! I submitted a FR to have a single options env var like I've ended up using in git-mediate.