Skip to content

Commit

Permalink
project: hook mdformat into pre-commit
Browse files Browse the repository at this point in the history
  • Loading branch information
isidentical committed Mar 20, 2021
1 parent 926c16e commit e82ce2b
Show file tree
Hide file tree
Showing 6 changed files with 61 additions and 77 deletions.
13 changes: 11 additions & 2 deletions .pre-commit-config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
# See https://pre-commit.com/hooks.html for more hooks
repos:
- repo: https://github.com/hakancelik96/unimport
rev: 0.7.3
rev: 0.8.3
hooks:
- id: unimport
args:
Expand All @@ -18,6 +18,15 @@ repos:
- id: isort
additional_dependencies: [toml]
- repo: https://github.com/asottile/setup-cfg-fmt
rev: v1.16.0
rev: v1.17.0
hooks:
- id: setup-cfg-fmt
- repo: https://github.com/executablebooks/mdformat/
rev: 0.6.1
hooks:
- id: mdformat
additional_dependencies:
- mdformat-black
args:
- --wrap
- "80"
9 changes: 5 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,8 +2,8 @@

# reiz.io

Reiz is a structural source code search engine framework,
with a built-in query language called [ReizQL](docs/reizql.md).
Reiz is a structural source code search engine framework, with a built-in query
language called [ReizQL](docs/reizql.md).

```
Warehouse Preparation (reiz.schema):
Expand Down Expand Up @@ -38,6 +38,7 @@ Data Querying [ReizQL] (reiz.reizql):
Generate IR from Reiz AST
```

For creating a new instance, please check out the "[deploy guide](docs/deploy_guide.md)". For any
other question, feel free to start up a new discussion on [#reiz channel on r/ProgrammingLanguages Discord server](https://discord.gg/r89x4EgZr4)
For creating a new instance, please check out the "[deploy guide](docs/deploy_guide.md)".
For any other question, feel free to start up a new discussion on
[#reiz channel on r/ProgrammingLanguages Discord server](https://discord.gg/r89x4EgZr4)
or on [GitHub discussions](https://github.com/reizio/reiz.io/discussions).
43 changes: 14 additions & 29 deletions docs/index.md
Original file line number Diff line number Diff line change
@@ -1,9 +1,8 @@
# reiz.io

reiz.io is a structural source code search engine for
Python. Compared to the popular alternatives (e.g Github
Code Search) it executes queries over the syntax trees
(instead of raw source code) and tries to retrive structural
reiz.io is a structural source code search engine for Python. Compared to the
popular alternatives (e.g Github Code Search) it executes queries over the
syntax trees (instead of raw source code) and tries to retrive structural
knowledge (no semantics applied).

```{toctree}
Expand All @@ -15,10 +14,9 @@ reizql

## A gentle introduction

Reiz is the code search framework that reiz.io is built
a top on. Due to it's nature, it solely works with the ASTs
and intentionally avoids doing any semantical work.

Reiz is the code search framework that reiz.io is built a top on. Due to it's
nature, it solely works with the ASTs and intentionally avoids doing any
semantical work.

```{note}
Some ASTs attach a bit of contextual knowledge (e.g `Name(ctx=...)`
Expand All @@ -27,30 +25,16 @@ reiz.io doesn't include them when comparing references (see
matchers#reference-matcher for details).
```

Here is a simple ReizQL query that searches for a function that
ends with a try statement where we return a call to a function that
has the same name as the function we are within.
Here is a simple ReizQL query that searches for a function that ends with a try
statement where we return a call to a function that has the same name as the
function we are within.

```python
FunctionDef(
~func,
body = [
*...,
Try(
body = [
Return(
Call(
Name(~func)
)
)
]
)

]
)
FunctionDef(~func, body=[*..., Try(body=[Return(Call(Name(~func)))])])
```

which would match the following;

```py
def foo(spam):
eggs = bar()
Expand All @@ -60,8 +44,9 @@ def foo(spam):
return None
```

In the very basic sense, it is generating the AST of the code above
and checks whether it fits the *pattern* (ReizQL query) or not.;
In the very basic sense, it is generating the AST of the code above and checks
whether it fits the *pattern* (ReizQL query) or not.;

```py
FunctionDef(
name='foo',
Expand Down
68 changes: 31 additions & 37 deletions docs/reizql.md
Original file line number Diff line number Diff line change
@@ -1,20 +1,18 @@
# ReizQL Language

ReizQL is a declarative query language for building AST matchers
that work on the Reiz platform.
ReizQL is a declarative query language for building AST matchers that work on
the Reiz platform.

:::{hint}
Here is an example ReizQL query that searches for an if statement
where the body consists from a single assignment statement that
assigns the result of `requests.get(...)` call's result into a
variable named `response`
:::{hint} Here is an example ReizQL query that searches for an if statement
where the body consists from a single assignment statement that assigns the
result of `requests.get(...)` call's result into a variable named `response`

```
if cache.invalidated:
response = requests.get('https://api.reiz.io/refresh')
```
:::

:::

```py
If(
Expand All @@ -34,8 +32,8 @@ If(
)
```


## Full Grammar

```bnf
start ::= match_pattern
Expand Down Expand Up @@ -76,12 +74,13 @@ NUMBER ::= INTEGER | FLOAT
match_pattern ::= NAME "(" ",".argument+ ")"
```

Match patterns are the most fundamental part of the query expression. They consist
from an identifier (matcher name) which corresponds to an AST node type, additionally
they take any number of fields to be matched (values, optionally attached with the
corresponding field names).
Match patterns are the most fundamental part of the query expression. They
consist from an identifier (matcher name) which corresponds to an AST node type,
additionally they take any number of fields to be matched (values, optionally
attached with the corresponding field names).

All node types and fields are described in the [Abstract Grammar](https://docs.python.org/3.8/library/ast.html#abstract-grammar)
All node types and fields are described in the
[Abstract Grammar](https://docs.python.org/3.8/library/ast.html#abstract-grammar)
of Python. Here are some entries from the ASDL;

```
Expand All @@ -106,33 +105,39 @@ module Python

The left hand side is the name of the base type, `stmt` would be a matcher that
could match all of the types in its right hand side (e.g `stmt()` would match
`FunctionDef()` / `While()` / `If()` / `With()`). Each element on the right
hand side are concrete matchers for that element in syntax. For example a
`BinOp()` represents a binary operation (2 operands), like `2 + 2` or `a % b()`.

Each element on the right hand side have different fields with types attached
to them. So the `BinOp()` node has 3 fields: `left`, `op`, `right` (respectively
left hand side, operator, right hand side of an arithmetic operation). `left` and
the `right` must be another matcher from the `expr` base type (`BoolOp`/`NamedExpr`,...).
The star (`*`) at the end of type implies that it requires a [list pattern](#list-pattern) that
consists from that type (e.g `stmt*` might be something like `[If(), If(), While()]`). The
question mark (`?`) indicates the value is optional and can be `None`.
`FunctionDef()` / `While()` / `If()` / `With()`). Each element on the right hand
side are concrete matchers for that element in syntax. For example a `BinOp()`
represents a binary operation (2 operands), like `2 + 2` or `a % b()`.

Each element on the right hand side have different fields with types attached to
them. So the `BinOp()` node has 3 fields: `left`, `op`, `right` (respectively
they mean left hand side, operator, right hand side of an arithmetic operation).
`left` and the `right` must be another matcher from the `expr` base type (`BoolOp`
/ `NamedExpr`, ...). The star (`*`) at the end of type implies that it requires
a [list pattern](#list-pattern) that consists from that type (e.g `stmt*` might
be something like `[If(), If(), While()]`). The question mark (`?`) indicates
the value is optional and can be `None`.

If the values are not named (e.g `BinOp(Constant())`) then the name will be
positionally given (`BinOp(Constant(), Add())` will be transformed to `BinOp(left=Constant(), op=Add()`).
positionally given (`BinOp(Constant(), Add())` will be transformed to
`BinOp(left=Constant(), op=Add()`).

#### Example Queries

- Match the `1994` literal

```py
Constant(1994)
```

- Match a binary operation where both sides are literals

```py
BinOp(left=Constant(), right=Constant())
```

- Match an (ternary) if expression that checks `a.b`'s truthness

```py
IfExp(
test = Attribute(
Expand All @@ -141,14 +146,3 @@ IfExp(
)
)
```











1 change: 0 additions & 1 deletion reiz/reizql/compiler/functions.py
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,6 @@

from reiz.ir import IR
from reiz.reizql.parser import grammar
from reiz.serialization.transformers import ast

if TYPE_CHECKING:
BuiltinFunctionType = Callable[
Expand Down
4 changes: 0 additions & 4 deletions scripts/create_db.py
Original file line number Diff line number Diff line change
@@ -1,12 +1,8 @@
#!/usr/bin/env python

import subprocess
from argparse import ArgumentParser
from contextlib import suppress
from pathlib import Path

from edgedb.errors import InvalidReferenceError

from reiz.database import get_new_connection
from reiz.utilities import logger

Expand Down

0 comments on commit e82ce2b

Please sign in to comment.