A tool for doing configuration nicely -- sorta.
This package has helpers for loading configuration and performing common setup tasks, e.g. for logging, stats, and datastore machinery. It provides a django-like singleton to consolidate configured values. It also provides utilities for implementing lightweight dependency injection.
Configuration can be loaded easily from from a variety of sources in a seamless, polymorphic way and is checked/cleaned by registered validation extensions (usually by section). You can load from many different sources (JSON, INI, dict, environment variable to name a few), as well as specifying a prioritized preference for loading (load from sys.argv if available, then from environment variable, then finally fall back to a certain file).
from dripconfig import config, sources
if __name__ == '__main__':
# wire up a RedisTrigger instance to configure redis based on info
config.register_trigger(RedisTrigger())
# load some config
config.merge_from(
# prefer a config loaded from filename passed as argument
sources.Argv(1),
# if we don't have an argv, load from a file
sources.EnvVar("CONF_FILENAME"),
# if the env var isn't set, fall back to a particular file
sources.Filename("some_conf.json"),
)
# specific values can be loaded very easily from environment variables
config['this']['that'] = os.environ.get('THIS_THAT_VAL', 'default_val')
# validate, run any global configuration steps, and have the triggers act
config.configure()
# on with the show...The RedisTrigger definition takes the loaded configuration info and renders
it into objects that let us use redis:
class RedisTrigger(ConfigurationTrigger):
"""
Require that redis connection information is specified.
"""
# voluptuous Schema objects are a great way of validating config.
SCHEMA = Schema({
'redis': {
Required('hostname'): basestring,
Required('realtime_updates_ss_name',
default='melrose:realtalk'): basestring,
},
}, extra=True)
def clean(self, configuration):
"""Make sure all the required values are here."""
if 'redis' not in configuration:
raise RuntimeError(
"Redis not properly configured; "
"see `melrose.config:RedisTrigger.schema`."
)
# make sure we match the schema we need
return self.SCHEMA(configuration)
def configure(self, configuration):
"""
Instantiate a redis client and any reliant objects.
"""
conf = configuration.redis
some_module.redis_client = redis.Redis(conf.hostname)Meanwhile in some_submodule, the module-level attribute redis_client is
declared to be injected by dripconfig. This allows the module to be
(blissfully) unaware of the configuration needed to instantiate its
dependencies.
# some_module.py
redis_client = dripconfig.ToBeInjected(redis.Redis)By the time dripconfig.configure() is called, redis_client will be an
object of type redis.Redis, and will be ready for use. Until that call is
made, though, any attempt to use that object will result in a RuntimeError.
Note: this does not support the python logging module 'ini' configuration format although you can load ini files. Use logging.fileConfig on that directly if desired.
The preferred format is JSON for nested data (logging configuration = yuck...), but configuration can be merged together from a variety of sources including ini files, yamls, dicts etc. A consolidated attribute-accessible object is presented for triggering further configuration actions.
from dripconfig import config
if __name__ == '__main__':
config.register_trigger(...)
# `merge` will automagically determine the type
config.merge('thisconf.yaml')
config.merge({'yo': 'bang!'})
# or you can do it explicitly
config.merge_dict({'some': 'default'})
config.merge_ini_file("/usr/local/etc/common.ini")
config.merge_json('{"foo": "bar"}')
# done loading, normalize and apply
config.configure()
# great, now get to work!
serve(port=config.port)
ConfigurationTrigger objects are tasked with validating/cleaning relevent portions of the configuration in the clean() method. After cleaning the values in the configuration are expected to be of the correct type, consumers should not worry about type checking the configuration. This is mainly a concern when loading .ini files.
The suggested method is to declare a Voluptuous Schema and use Corece() on numerical values and anything that may require additional interpretation.
ConfigurationTriggers can also apply portions of the configuration in a global fashion if necesary via the configure() method. Stats and logging are often configured by this method.
The best thing is that they're simple. The worst thing is they sort of stink for the same reason and no magic to support nesting is done. Anything in the [main] section is considered top-level, everything else is nested under a key with the name of the section.
For logging configurations that use syslog, a slightly improved handler is
provided that logs the process name with outgoing messages when used
with an appropriate formatter. The variable ident is provided, which
can be used as the prefix to the logging format to identify the process
to syslog eg %(ident)s %(message).
An example configuration might include a handler such as the following in the logging.dictConfig format:
{
"logging": {
"formatters": {
...
"syslog": {
"format": "%(ident)s [%(levelname)s %(asctime)s %(module)s] %(message)s"
}
},
...
"handlers": {
"syslog":{
"class":"dripconfig.SysLogHandler",
"level":"INFO",
"formatter": "syslog",
"address": "/dev/log"
},
},
...
}
}
- arg parse example or helper for specifying config files to load?
- some way to do email based logging without django?
- json with comments is shady, works but can't give good line-number errors, which stinks