Skip to content

Commit

Permalink
lint: disabled linting and type checking
Browse files Browse the repository at this point in the history
  • Loading branch information
phitoduck committed Sep 28, 2024
1 parent b5d1c8c commit a7537e0
Show file tree
Hide file tree
Showing 32 changed files with 433 additions and 269 deletions.
2 changes: 1 addition & 1 deletion .github/workflows/build-test-publish.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ permissions:
contents: write

jobs:

check-version-txt:
runs-on: ubuntu-latest
steps:
Expand Down
52 changes: 26 additions & 26 deletions .pre-commit-config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -46,16 +46,16 @@ repos:
# Detects the presence of private keys
- id: detect-private-key

- repo: https://github.com/pre-commit/mirrors-mypy
rev: "v1.2.0"
hooks:
- id: mypy
args:
[
--no-strict-optional,
--ignore-missing-imports,
--config-file=./pyproject.toml,
]
# - repo: https://github.com/pre-commit/mirrors-mypy
# rev: "v1.2.0"
# hooks:
# - id: mypy
# args:
# [
# --no-strict-optional,
# --ignore-missing-imports,
# --config-file=./pyproject.toml,
# ]

- repo: https://github.com/psf/black
rev: 23.1.0
Expand All @@ -64,23 +64,23 @@ repos:
args:
- --config=./pyproject.toml

- repo: https://github.com/PyCQA/pylint
rev: v2.16.3
hooks:
- id: pylint
args:
- --rcfile=./pyproject.toml
# - repo: https://github.com/PyCQA/pylint
# rev: v2.16.3
# hooks:
# - id: pylint
# args:
# - --rcfile=./pyproject.toml

- repo: https://github.com/PyCQA/flake8
rev: 6.0.0
hooks:
- id: flake8
args:
- --toml-config=./pyproject.toml
additional_dependencies:
- radon
- flake8-docstrings
- Flake8-pyproject
# - repo: https://github.com/PyCQA/flake8
# rev: 6.0.0
# hooks:
# - id: flake8
# args:
# - --toml-config=./pyproject.toml
# additional_dependencies:
# - radon
# - flake8-docstrings
# - Flake8-pyproject

- repo: https://github.com/pycqa/isort
rev: 5.12.0
Expand Down
2 changes: 1 addition & 1 deletion .vscode/extensions.json
Original file line number Diff line number Diff line change
Expand Up @@ -36,4 +36,4 @@
// jump to definition of pytest fixtures, e.g. with Ctrl+Click / Cmd+Click
"nickmillerdev.pytest-fixtures",
]
}
}
2 changes: 1 addition & 1 deletion .vscode/settings.json
Original file line number Diff line number Diff line change
Expand Up @@ -46,4 +46,4 @@
"color": "green"
}
],
}
}
36 changes: 18 additions & 18 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -37,33 +37,33 @@ Project templates are great... except that they promote "code duplication at sca

## Real Examples

`pyprojen` is a minimal port of `projen`'s core functionality to Python.
`pyprojen` is a minimal port of `projen`'s core functionality to Python.

Specifically, it gets you

1. ⚡️ **Agility**
- **Examples:**

- **Examples:**
- 10x the speed of CI for all repos overnight by making CI steps run in parallel instead of serially
- Update a team linting rule in `pyproject.toml`, `ruff.toml`, etc.
- Add auth steps to publish to or install from a private PyPI server in CI or `Dockerfile`s
- Completely change CI systems with minimal disruption, e.g. switch from Bitbucket Pipelines to GitHub Actions to AWS CodeBuild and back

2. 🧱 **Modularity**
- **Examples**:
1. define an opinionated `PythonPackage` component, and
1. layer on top a
- `FastAPIApp`,
- `StreamlitApp`,
- `CdkApp`,
- `PulumiApp`,
- `AirflowDag`,
- `DagsterDag`,

- **Examples**:
1. define an opinionated `PythonPackage` component, and
1. layer on top a
- `FastAPIApp`,
- `StreamlitApp`,
- `CdkApp`,
- `PulumiApp`,
- `AirflowDag`,
- `DagsterDag`,
- `BentoMLService`,
- `AwsLambdaPythonFunction`,
- etc.
1. Add (or remove) as many of these to your repo as you like, whenever you like, and find these packages instantly set up with CI, linting, formatting, tests, packaging, publishing, deploying, etc.
1. Add (or remove) as many of these to your repo as you like, whenever you like, and find these packages instantly set up with CI, linting, formatting, tests, packaging, publishing, deploying, etc.
- For example, you might incrementally develop a "mini data science app monorepo" with
1. a `MetaflowDag` that trains a model
1. served in a `FastAPI` app
Expand Down Expand Up @@ -100,7 +100,7 @@ That said, although `cookiecutter` and `copier` are more limited, they are also
4. An opinionated "task runner" system (think `Makefile/Justfile`, `poetry` scripts, etc.) to define project-related commands.
5. A `projen new` command which creates the initial `.projenrc.py` config file for your project

#### 2. `pyprojen` implements [1] and [2] from the list above (the unopinionated parts).
#### 2. `pyprojen` implements [1] and [2] from the list above (the unopinionated parts).

It is up to you to create your own components with your own opinions on things like

Expand All @@ -127,7 +127,7 @@ If you write components in Python using `pyprojen`, it should be easy to move th

`projen` is a larger project and is primarily maintained by developers at AWS. `projen`,

But to develop with `projen`, you either need to write TypeScript, or use generated Python bindings that invoke TypeScript.
But to develop with `projen`, you either need to write TypeScript, or use generated Python bindings that invoke TypeScript.

If you are familiar with writing AWS CDK in Python, developing with `projen` in Python is a similar experience, because they both use Python bindings generated from TypeScript using the [JSII](https://github.com/aws/jsii) project.

Expand All @@ -143,7 +143,7 @@ This means:

## Quick start (TODO)

> [!NOTE]
> [!NOTE]
> Until this section is filled out, you can refer to [this repo](https://github.com/phitoduck/phito-projen) to get a sense of what projen can do. And the official [projen docs](https://projen.io/) contain many of the same concepts that this port uses.
```bash
Expand All @@ -164,7 +164,7 @@ You will need the following installed on your machine to develop on this codebas
- Python 3.7+, ideally using `pyenv` to easily change between Python versions
- `git`

###
###

```bash
# clone the repo
Expand Down
2 changes: 1 addition & 1 deletion pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,7 @@ static-code-qa = [
"Flake8-pyproject",
"radon",
]
# Installing dev depenendencies in your virtual env makes it so that during development VS Code can
# Installing dev depenendencies in your virtual env makes it so that during development VS Code can
# - surface linting errors
# - automatically apply formatting
# - show enhanced autocompletion for stubs libraries
Expand Down
16 changes: 11 additions & 5 deletions src/pyprojen/_resolve.py
Original file line number Diff line number Diff line change
@@ -1,12 +1,18 @@
from typing import Any, Dict, List
import re
from typing import (
Any,
Dict,
)


class IResolvable:
def to_json(self) -> Any:
raise NotImplementedError()


def is_resolvable(obj: Any) -> bool:
return hasattr(obj, 'to_json') and callable(getattr(obj, 'to_json'))
return hasattr(obj, "to_json") and callable(getattr(obj, "to_json"))


def resolve(value: Any, options: Dict[str, Any] = {}) -> Any:
"""
Expand All @@ -24,8 +30,8 @@ def resolve(value: Any, options: Dict[str, Any] = {}) -> Any:
Raises:
ValueError: If a regular expression with flags is encountered.
"""
args = options.get('args', [])
omit_empty = options.get('omit_empty', False)
args = options.get("args", [])
omit_empty = options.get("omit_empty", False)

match value:
case None:
Expand All @@ -51,4 +57,4 @@ def resolve(value: Any, options: Dict[str, Any] = {}) -> Any:
case _ if callable(value):
return resolve(value(*args), options)
case _:
return value
return value
17 changes: 11 additions & 6 deletions src/pyprojen/cleanup.py
Original file line number Diff line number Diff line change
@@ -1,10 +1,11 @@
import os
import json
from typing import List
import logging
import os
from typing import List

FILE_MANIFEST = ".pyprojen/files.json"


def cleanup(dir: str, new_files: List[str], exclude: List[str]):
try:
manifest_files = get_files_from_manifest(dir)
Expand All @@ -17,29 +18,33 @@ def cleanup(dir: str, new_files: List[str], exclude: List[str]):
except Exception as e:
logging.warn(f"warning: failed to clean up generated files: {str(e)}")


def remove_files(files: List[str]):
for file in files:
try:
os.remove(file)
except Exception as e:
logging.warn(f"Failed to remove file {file}: {str(e)}")


def find_orphaned_files(dir: str, old_files: List[str], new_files: List[str]) -> List[str]:
return [os.path.join(dir, f) for f in old_files if f not in new_files]


def find_generated_files(dir: str, exclude: List[str]) -> List[str]:
# Implement this function to find generated files based on a marker
# This is a placeholder and should be implemented based on your specific needs
return []


def get_files_from_manifest(dir: str) -> List[str]:
try:
manifest_path = os.path.join(dir, FILE_MANIFEST)
if os.path.exists(manifest_path):
with open(manifest_path, 'r') as f:
with open(manifest_path, "r") as f:
manifest = json.load(f)
if 'files' in manifest:
return manifest['files']
if "files" in manifest:
return manifest["files"]
except Exception as e:
logging.warn(f"warning: unable to get files to clean from file manifest: {str(e)}")
return []
return []
2 changes: 1 addition & 1 deletion src/pyprojen/common.py
Original file line number Diff line number Diff line change
Expand Up @@ -2,4 +2,4 @@
Common constants and utilities for pyprojen.
"""

FILE_MANIFEST = ".pyprojen/files.json"
FILE_MANIFEST = ".pyprojen/files.json"
17 changes: 12 additions & 5 deletions src/pyprojen/component.py
Original file line number Diff line number Diff line change
@@ -1,6 +1,16 @@
from typing import Any, Dict, Optional
from typing import (
Any,
Dict,
Optional,
)

from pyprojen.constructs import Construct
from pyprojen.util.constructs import is_component, find_closest_project, tag_as_component
from pyprojen.util.constructs import (
find_closest_project,
is_component,
tag_as_component,
)


class Component(Construct):
"""
Expand Down Expand Up @@ -48,16 +58,13 @@ def pre_synthesize(self):
"""
Called before synthesis.
"""
pass

def synthesize(self):
"""
Synthesizes files to the project output directory.
"""
pass

def post_synthesize(self):
"""
Called after synthesis.
"""
pass
14 changes: 12 additions & 2 deletions src/pyprojen/constructs/__init__.py
Original file line number Diff line number Diff line change
@@ -1,5 +1,15 @@
from .construct import Construct, IConstruct, Node, ConstructOrder, MetadataEntry
from .dependency import IDependable, DependencyGroup, Dependable
from .construct import (
Construct,
ConstructOrder,
IConstruct,
MetadataEntry,
Node,
)
from .dependency import (
Dependable,
DependencyGroup,
IDependable,
)

__all__ = [
"Construct",
Expand Down
Loading

0 comments on commit a7537e0

Please sign in to comment.