Sensible defaults to catch bad behavior.
pyramid-sanity is a Pyramid extension that catches certain crashes caused by
badly formed requests, turning them into 400: Bad Request responses instead.
It also prevents apps from returning HTTP redirects with badly encoded locations that can crash WSGI servers.
The aim is to have sensible defaults to make it easier to write a reliable Pyramid app.
For details of all the errors and fixes, and how to reproduce them see: Error details.
with Configurator() as config:
config.add_settings({
# See the section below for all settings...
"pyramid_sanity.check_form": False,
})
# Add this as near to the end of your config as possible:
config.include("pyramid_sanity")By default all fixes are enabled. You can disable them individually with settings:
config.add_settings({
# Don't check for badly declared forms.
"pyramid_sanity.check_form": False
})You can set pyramid_sanity.disable_all to True to disable all of the fixes,
then enable only certain fixes one by one:
config.add_settings({
# Disable all fixes.
"pyramid_sanity.disable_all": True,
# Enable only the badly encoded query params fix.
"pyramid_sanity.check_params": True
})| Option | Default | Effect |
|---|---|---|
pyramid_sanity.disable_all |
False |
Disable all checks by default |
pyramid_sanity.check_form |
True |
Check for badly declared forms |
pyramid_sanity.check_params |
True |
Check for badly encoded query params |
pyramid_sanity.check_path |
True |
Check for badly encoded URL paths |
pyramid_sanity.ascii_safe_redirects |
True |
Safely encode redirect locations |
All exceptions returned by pyramid-sanity are subclasses of
pyramid_sanity.exceptions.SanityException, which is a subclass of
pyramid.httpexceptions.HTTPBadRequest.
This means all pyramid-sanity exceptions trigger 400: Bad Request responses.
Different exception subclasses are returned for different problems, so you can register custom exception views to handle them if you want:
| Exception | Returned for |
|---|---|
pyramid_sanity.exceptions.InvalidQueryString |
Badly encoded query params |
pyramid_sanity.exceptions.InvalidFormData |
Bad form posts |
pyramid_sanity.exceptions.InvalidURL |
Badly encoded URL paths |
pyramid-sanity uses a number of Pyramid tweens
to do its work. It's important that your app's tween chain has:
- Our tweens that check for errors in the request, first
- Our tweens that check for errors in the output of your app, last
The easiest way to achieve this is to include config.include("pyramid_sanity")
as late as possible in your config. This uses Pyramid's
"best effort" implicit tween ordering
to add the tweens and should work as long as your app doesn't add any
more tweens, or include any extensions that add tweens, afterwards.
You can to check the order of tweens in your app with Pyramid's
ptweens command.
As long as there are no tweens which access request.GET or request.POST
above the input checking tweens, or generate redirects below output checking
tweens, you should be fine.
You can force the order with Pyramid's explicit tween ordering if you need to.
pyramid-sanity protects against non-ASCII redirects raised by your app's
views by safely encoding them, but it can't protect against other tweens that
raise non-ASCII redirects.
For example this tween might cause a WSGI server (like Gunicorn) that's serving
your app to crash with UnicodeEncodeError:
def non_ascii_redirecting_tween_factory(handler, registry):
def non_ascii_redirecting_tween(request):
from pyramid.httpexceptions import HTTPFound
raise HTTPFound(location="http://example.com/€/☃")
return non_ascii_redirecting_tweenYou'll just have to make sure that your app doesn't have any tweens that do this! Tweens should encode any redirect locations that they generate, like this.
If you would like to reproduce the errors an example app is given at the end
of this section. All of the presented curl commands work with this app.
curl 'http://localhost:6543/foo?q=%FC'
By default
WebOb raises UnicodeDecodeError. As there is no built-in exception view for
this exception the app crashes.
With pyramid-sanity
A pyramid_sanity.exceptions.InvalidQueryString is returned which results in a
400: Bad Request response.
Related issues:
curl 'http://localhost:6543/%FC'
By default
Pyramid raises pyramid.exceptions.URLDecodeError.
As there is no built-in exception view for this exception the app crashes.
With pyramid-sanity
A pyramid_sanity.exceptions.InvalidURL is returned which results in a
400: Bad Request response.
Related issues
curl --request POST --url http://localhost:6543/foo --header 'content-type: multipart/form-data'
By default
WebOb raises an uncaught ValueError. As there is no built-in exception view
for this exception the app crashes.
With pyramid-sanity
A pyramid_sanity.exceptions.InvalidFormData is returned which results in a
400: Bad Request response.
Related issues:
curl http://localhost:6543/redirect
By default
The app will emit the redirect successfully, but the WSGI server running the app
may crash. With the example app below wsgiref.simple_server raises an
uncaught AttributeError.
With pyramid-sanity
The redirect is safely URL encoded.
from wsgiref.simple_server import make_server
from pyramid.config import Configurator
from pyramid.response import Response
from pyramid.httpexceptions import HTTPFound
def redirect(request):
# Return a redirect to a URL with a non-ASCII character in it.
return HTTPFound(location="http://example.com/☃")
def hello_world(request):
return Response(f"Hello World! Query string was: {request.GET}. Form body was: {request.POST}")
if __name__ == "__main__":
with Configurator() as config:
config.add_route("redirect", "/redirect")
config.add_route("hello", "/{anything}")
config.add_view(hello_world, route_name="hello")
config.add_view(redirect, route_name="redirect")
app = config.make_wsgi_app()
server = make_server("0.0.0.0", 6543, app)
server.serve_forever()First you'll need to install:
- Git.
On Ubuntu:
sudo apt install git, on macOS:brew install git. - GNU Make.
This is probably already installed, run
make --versionto check. - pyenv. Follow the instructions in pyenv's README to install it. The Homebrew method works best on macOS. The Basic GitHub Checkout method works best on Ubuntu. You don't need to set up pyenv's shell integration ("shims"), you can use pyenv without shims.
Then to set up your development environment:
git clone https://github.com/hypothesis/pyramid-sanity.git
cd pyramid-sanity
make help
-
First, to get PyPI publishing working you need to go to: https://github.com/organizations/hypothesis/settings/secrets/actions/PYPI_TOKEN and add pyramid-sanity to the
PYPI_TOKENsecret's selected repositories. -
Now that the pyramid-sanity project has access to the
PYPI_TOKENsecret you can release a new version by just creating a new GitHub release. Publishing a new GitHub release will automatically trigger a GitHub Actions workflow that will build the new version of your Python package and upload it to https://pypi.org/project/pyramid-sanity.
To change what versions of Python the project uses:
-
Change the Python versions in the cookiecutter.json file. For example:
"python_versions": "3.10.4, 3.9.12",
-
Re-run the cookiecutter template:
make template -
Commit everything to git and send a pull request
To change the production dependencies in the setup.cfg file:
-
Change the dependencies in the
.cookiecutter/includes/setuptools/install_requiresfile. If this file doesn't exist yet create it and add some dependencies to it. For example:pyramid sqlalchemy celery -
Re-run the cookiecutter template:
make template -
Commit everything to git and send a pull request
To change the project's formatting, linting and test dependencies:
-
Change the dependencies in the
.cookiecutter/includes/tox/depsfile. If this file doesn't exist yet create it and add some dependencies to it. Use tox's factor-conditional settings to limit which environment(s) each dependency is used in. For example:lint: flake8, format: autopep8, lint,tests: pytest-faker, -
Re-run the cookiecutter template:
make template -
Commit everything to git and send a pull request