Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

start ruff github-action #180

Open
wants to merge 8 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 1 addition & 7 deletions .github/workflows/python-package.yml
Original file line number Diff line number Diff line change
Expand Up @@ -22,14 +22,8 @@ jobs:
- name: Install dependencies
run: |
python -m pip install --upgrade pip
python -m pip install flake8 pytest pytest-cov
python -m pip install pytest pytest-cov
if [ -f requirements.txt ]; then pip install -r requirements.txt; fi
- name: Lint with flake8
run: |
# stop the build if there are Python syntax errors or undefined names
flake8 spellchecker/ --count --select=E9,F63,F7,F82 --show-source --statistics
# exit-zero treats all errors as warnings. The GitHub editor is 127 chars wide
flake8 spellchecker/ --count --exit-zero --max-complexity=11 --max-line-length=127 --statistics
- name: Test with pytest
run: |
# Run tests while also generating coverage statistics
Expand Down
15 changes: 15 additions & 0 deletions .github/workflows/ruff-check.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
name: Ruff
on: [workflow_dispatch, pull_request]
jobs:
build:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Install Python
uses: actions/setup-python@v4
with:
python-version: "3.13"
- uses: astral-sh/ruff-action@v3
with:
args: "check --fix"
continue-on-error: false
95 changes: 82 additions & 13 deletions pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -24,14 +24,14 @@ classifiers = [
"License :: OSI Approved :: MIT License",
"Programming Language :: Python",
"Programming Language :: Python :: 3",
"Programming Language :: Python :: 3.7",
"Programming Language :: Python :: 3.8",
"Programming Language :: Python :: 3.9",
"Programming Language :: Python :: 3.10",
"Programming Language :: Python :: 3.11",
"Programming Language :: Python :: 3.12",
"Programming Language :: Python :: 3.13",
]
requires-python = ">=3.7"
requires-python = ">=3.8"


[tool.setuptools.dynamic]
Expand All @@ -44,16 +44,6 @@ bug-tracker = "https://github.com/barrust/pyspellchecker/issues"
documentation = "https://pyspellchecker.readthedocs.io/"


[tool.poetry]
packages = [{ include = "spellchecker" }]

[tool.poetry.dev-dependencies]
pre-commit = { version = ">=2.18.1", python = "^3.6.1" }
black = { version = "^20.8b1", python = "^3.6" }
isort = { version = "^5.6.4", python = "^3.6" }
pytest = { version = "^6.1.1", python = "^3.6" }
flake8 = { version = "^3.6.0", python = "^3.6" }

[tool.setuptools.packages.find]
include = ["spellchecker", "spellchecker.*"]

Expand All @@ -77,13 +67,92 @@ profile = "black"

[tool.black]
line-length = 120
target-version = ['py37']
target-version = ["py38", "py39", "py310", "py311"]
include = '\.pyi?$'

[tool.pylint.'MESSAGES CONTROL']
max-line-length = 120
max-args = 6

[tool.ruff]
include = ["pyproject.toml", "spellchecker/**/*.py", "scripts/**/*.py"]
exclude = [
".bzr",
".direnv",
".eggs",
".git",
".git-rewrite",
".hg",
".ipynb_checkpoints",
".mypy_cache",
".nox",
".pants.d",
".pyenv",
".pytest_cache",
".pytype",
".ruff_cache",
".svn",
".tox",
".venv",
".vscode",
"__pypackages__",
"_build",
"buck-out",
"build",
"dist",
"node_modules",
"site-packages",
"venv",
]

# Same as Black.
line-length = 120
indent-width = 4

# Assume Python 3.9
target-version = "py39"

[tool.ruff.lint]
# Enable Pyflakes (`F`) and a subset of the pycodestyle (`E`) codes by default.
# Unlike Flake8, Ruff doesn't enable pycodestyle warnings (`W`) or
# McCabe complexity (`C901`) by default.
select = ["E4", "E7", "E9", "F"]
ignore = []

# Allow fix for all enabled rules (when `--fix`) is provided.
fixable = ["ALL"]
unfixable = []

# Allow unused variables when underscore-prefixed.
dummy-variable-rgx = "^(_+|(_+[a-zA-Z0-9_]*[a-zA-Z0-9]+?))$"

[tool.ruff.format]
# Like Black, use double quotes for strings.
quote-style = "double"

# Like Black, indent with spaces, rather than tabs.
indent-style = "space"

# Like Black, respect magic trailing commas.
skip-magic-trailing-comma = false

# Like Black, automatically detect the appropriate line ending.
line-ending = "auto"

# Enable auto-formatting of code examples in docstrings. Markdown,
# reStructuredText code/literal blocks and doctests are all supported.
#
# This is currently disabled by default, but it is planned for this
# to be opt-out in the future.
docstring-code-format = false

# Set the line length limit used when formatting code snippets in
# docstrings.
#
# This only has an effect when the `docstring-code-format` setting is
# enabled.
docstring-code-line-length = "dynamic"

[build-system]
requires = ["setuptools>=61.2.0", "wheel"]
build-backend = "setuptools.build_meta"
52 changes: 26 additions & 26 deletions scripts/build_dictionary.py
Original file line number Diff line number Diff line change
@@ -1,26 +1,26 @@
""" Desc: A script to automate the building of multiple dictionaries based on
known areas of concern due to the original source of the data. The
script can be run from the source directly (-P and -p) once a
sutable text file is obtained. It can also be run on a previously
generated word frequency list to remove known problem areas.
Author: Tyler Barrus
Notes: The original inputs are from OpenSubtitles (http://opus.nlpl.eu/OpenSubtitles2018.php):
English Input: http://opus.nlpl.eu/download.php?f=OpenSubtitles/v2018/mono/OpenSubtitles.raw.en.gz
Spanish Input: http://opus.nlpl.eu/download.php?f=OpenSubtitles/v2018/mono/OpenSubtitles.raw.es.gz
German Input: http://opus.nlpl.eu/download.php?f=OpenSubtitles/v2018/mono/OpenSubtitles.raw.de.gz
French Input: http://opus.nlpl.eu/download.php?f=OpenSubtitles/v2018/mono/OpenSubtitles.raw.fr.gz
Portuguese Input: http://opus.nlpl.eu/download.php?f=OpenSubtitles/v2018/mono/OpenSubtitles.raw.pt.gz
Russian Input: http://opus.nlpl.eu/download.php?f=OpenSubtitles/v2018/mono/OpenSubtitles.raw.ru.gz
Arabic Input: http://opus.nlpl.eu/download.php?f=OpenSubtitles/v2018/mono/OpenSubtitles.raw.ar.gz
Basque Input: http://opus.nlpl.eu/download.php?f=tiOpenSubtles/v2018/mono/OpenSubtitles.raw.eu.gz
Latvian Input: https://huggingface.co/datasets/RaivisDejus/latvian-text
Dutch Input: http://opus.nlpl.eu/download.php?f=OpenSubtitles/v2018/mono/OpenSubtitles.raw.nl.gz
Italian Input: http://opus.nlpl.eu/download.php?f=OpenSubtitles/v2018/mono/OpenSubtitles.raw.it.gz
Requirements:
The script requires more than the standard library to run in its
entirety. You will also need to install the NLTK package to build a
dictionary from scratch. Otherwise, no additional packages are
required.
"""Desc: A script to automate the building of multiple dictionaries based on
known areas of concern due to the original source of the data. The
script can be run from the source directly (-P and -p) once a
sutable text file is obtained. It can also be run on a previously
generated word frequency list to remove known problem areas.
Author: Tyler Barrus
Notes: The original inputs are from OpenSubtitles (http://opus.nlpl.eu/OpenSubtitles2018.php):
English Input: http://opus.nlpl.eu/download.php?f=OpenSubtitles/v2018/mono/OpenSubtitles.raw.en.gz
Spanish Input: http://opus.nlpl.eu/download.php?f=OpenSubtitles/v2018/mono/OpenSubtitles.raw.es.gz
German Input: http://opus.nlpl.eu/download.php?f=OpenSubtitles/v2018/mono/OpenSubtitles.raw.de.gz
French Input: http://opus.nlpl.eu/download.php?f=OpenSubtitles/v2018/mono/OpenSubtitles.raw.fr.gz
Portuguese Input: http://opus.nlpl.eu/download.php?f=OpenSubtitles/v2018/mono/OpenSubtitles.raw.pt.gz
Russian Input: http://opus.nlpl.eu/download.php?f=OpenSubtitles/v2018/mono/OpenSubtitles.raw.ru.gz
Arabic Input: http://opus.nlpl.eu/download.php?f=OpenSubtitles/v2018/mono/OpenSubtitles.raw.ar.gz
Basque Input: http://opus.nlpl.eu/download.php?f=tiOpenSubtles/v2018/mono/OpenSubtitles.raw.eu.gz
Latvian Input: https://huggingface.co/datasets/RaivisDejus/latvian-text
Dutch Input: http://opus.nlpl.eu/download.php?f=OpenSubtitles/v2018/mono/OpenSubtitles.raw.nl.gz
Italian Input: http://opus.nlpl.eu/download.php?f=OpenSubtitles/v2018/mono/OpenSubtitles.raw.it.gz
Requirements:
The script requires more than the standard library to run in its
entirety. You will also need to install the NLTK package to build a
dictionary from scratch. Otherwise, no additional packages are
required.
"""

import contextlib
Expand Down Expand Up @@ -60,8 +60,8 @@ def load_include_exclude(filename, encoding="utf-8"):
if line[0] == "#":
continue
line = line.strip().split()
for l in line:
yield l.strip().lower()
for ln in line:
yield ln.strip().lower()


def export_word_frequency(filepath, word_frequency):
Expand Down Expand Up @@ -314,7 +314,7 @@ def clean_spanish(word_frequency, filepath_exclude, filepath_include, filepath_d
# NOTE: the ü must be just after a g and before an e or i only (with or without accent)!
misplaced_u = list()
for key in word_frequency:
if not "ü" in key:
if "ü" not in key:
continue
idx = key.index("ü")
if idx == 0 or idx == len(key) - 1: # first or last letter
Expand Down
20 changes: 10 additions & 10 deletions spellchecker/__init__.py
Original file line number Diff line number Diff line change
@@ -1,15 +1,15 @@
""" SpellChecker Module """
from spellchecker.spellchecker import SpellChecker, WordFrequency
from spellchecker.info import (
"""SpellChecker Module"""

from spellchecker.info import ( # noqa: F401
__author__,
__maintainer__,
__email__,
__license__, # noqa: F401
__version__,
__bugtrack_url__,
__credits__,
__email__,
__license__,
__maintainer__,
__url__,
__bugtrack_url__,
) # noqa: F401

__version__,
)
from spellchecker.spellchecker import SpellChecker, WordFrequency

__all__ = ["SpellChecker", "WordFrequency"]
2 changes: 1 addition & 1 deletion spellchecker/info.py
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
""" SpellChecker Information """
"""SpellChecker Information"""

__author__ = "Tyler Barrus"
__maintainer__ = "Tyler Barrus"
Expand Down
5 changes: 3 additions & 2 deletions spellchecker/spellchecker.py
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
""" SpellChecker Module; simple, intuitive spell checker based on the post by
Peter Norvig. See: https://norvig.com/spell-correct.html """
"""SpellChecker Module; simple, intuitive spell checker based on the post by
Peter Norvig. See: https://norvig.com/spell-correct.html"""

import gzip
import json
import pkgutil
Expand Down
3 changes: 2 additions & 1 deletion spellchecker/utils.py
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
""" Additional utility functions """
"""Additional utility functions"""

import contextlib
import functools
import gzip
Expand Down
Loading