Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add typing and drop support for EOL versions #22

Draft
wants to merge 24 commits into
base: master
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
24 commits
Select commit Hold shift + click to select a range
e261edf
Remove python 2 block
GideonBear Mar 1, 2023
5c1ce9e
Update .gitignore (https://github.com/github/gitignore/blob/main/Pyth…
GideonBear Mar 1, 2023
2321a2b
Add todo
GideonBear Mar 1, 2023
4ba7714
Run `pyupgrade`
GideonBear Mar 1, 2023
380db73
Run `autopep8`
GideonBear Mar 1, 2023
94594b7
Sort imports and add `from __future__ import annotations`
GideonBear Mar 1, 2023
55ddf74
Fix typos
GideonBear Mar 1, 2023
7225080
Replace `staticmethod` with `classmethod`
GideonBear Mar 1, 2023
065ca6f
Move to pyproject.toml from setup.py
GideonBear Mar 2, 2023
46551db
Add "Development Status :: 5 - Production/Stable" classifier
GideonBear Mar 3, 2023
70ad07e
Remove unnecessary dependencies from example
GideonBear Mar 3, 2023
5b05c10
Move to package
GideonBear Mar 3, 2023
0aacfe9
Add py.typed
GideonBear Mar 3, 2023
d0167fa
Fix pyproject.toml for package layout
GideonBear Mar 3, 2023
55ddc2a
Run `unittest2pytest`
GideonBear Mar 3, 2023
050a223
Remove deprecated aliases
GideonBear Mar 14, 2023
5516877
Remove tests for deprecated aliases
GideonBear Mar 14, 2023
cd567c4
Remove deprecated `set_data` method
GideonBear Mar 14, 2023
d8a75d4
Remove unused import
GideonBear Mar 14, 2023
ca28fe6
Update test.py
GideonBear Mar 14, 2023
dbdf95c
Resolve merge conflict
GideonBear Mar 14, 2023
5a6cd59
Merge branch 'master' into typing
GideonBear Jul 9, 2023
3dd5b90
Add typing to `_initfile` and replace datatype string with enum
GideonBear Jul 9, 2023
0b8649a
Fix typing in `_initfile`
GideonBear Jul 9, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
104 changes: 101 additions & 3 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,6 @@ __pycache__/

# Distribution / packaging
.Python
env/
build/
develop-eggs/
dist/
Expand All @@ -20,9 +19,12 @@ lib64/
parts/
sdist/
var/
wheels/
share/python-wheels/
*.egg-info/
.installed.cfg
*.egg
MANIFEST

# PyInstaller
# Usually these files are written by a python script from a template
Expand All @@ -37,26 +39,122 @@ pip-delete-this-directory.txt
# Unit test / coverage reports
htmlcov/
.tox/
.nox/
.coverage
.coverage.*
.cache
nosetests.xml
coverage.xml
*,cover
*.cover
*.py,cover
.hypothesis/
.pytest_cache/
cover/

# Translations
*.mo
*.pot

# Django stuff:
*.log
local_settings.py
db.sqlite3
db.sqlite3-journal

# Flask stuff:
instance/
.webassets-cache

# Scrapy stuff:
.scrapy

# Sphinx documentation
docs/_build/

# PyBuilder
.pybuilder/
target/

#Ipython Notebook
# Jupyter Notebook
.ipynb_checkpoints

# IPython
profile_default/
ipython_config.py

# pyenv
# For a library or package, you might want to ignore these files since the code is
# intended to run in multiple environments; otherwise, check them in:
# .python-version

# pipenv
# According to pypa/pipenv#598, it is recommended to include Pipfile.lock in version control.
# However, in case of collaboration, if having platform-specific dependencies or dependencies
# having no cross-platform support, pipenv may install dependencies that don't work, or not
# install all needed dependencies.
#Pipfile.lock

# poetry
# Similar to Pipfile.lock, it is generally recommended to include poetry.lock in version control.
# This is especially recommended for binary packages to ensure reproducibility, and is more
# commonly ignored for libraries.
# https://python-poetry.org/docs/basic-usage/#commit-your-poetrylock-file-to-version-control
#poetry.lock

# pdm
# Similar to Pipfile.lock, it is generally recommended to include pdm.lock in version control.
#pdm.lock
# pdm stores project-wide configurations in .pdm.toml, but it is recommended to not include it
# in version control.
# https://pdm.fming.dev/#use-with-ide
.pdm.toml

# PEP 582; used by e.g. github.com/David-OConnor/pyflow and github.com/pdm-project/pdm
__pypackages__/

# Celery stuff
celerybeat-schedule
celerybeat.pid

# SageMath parsed files
*.sage.py

# Environments
.env
.venv
env/
venv/
ENV/
env.bak/
venv.bak/

# Spyder project settings
.spyderproject
.spyproject

# Rope project settings
.ropeproject

# mkdocs documentation
/site

# mypy
.mypy_cache/
.dmypy.json
dmypy.json

# Pyre type checker
.pyre/

# pytype static type analyzer
.pytype/

# Cython debug symbols
cython_debug/

# PyCharm
# JetBrains specific template is maintained in a separate JetBrains.gitignore that can
# be found at https://github.com/github/gitignore/blob/main/Global/JetBrains.gitignore
# and can be added to the global gitignore or merged into this file. For a more nuclear
# option (not recommended) you can uncomment the following to ignore the entire idea folder.
#.idea/
4 changes: 2 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -27,13 +27,13 @@
- **Flexible**: `livejson` fully supports complex nestings of `list`s and `dict`s, meaning it can represent any valid JSON file.
- **Compatible**: `livejson` works on all [versions](https://devguide.python.org/versions/) of Python that are not end-of-life.
- **Lightweight**: `livejson` is a single file with no external dependencies. Just install and go!
- **Reliable**: by default, no caching is used. Every single time you access a `livejson.Database`, it's read straight from the file. And every time you write to it, the change is instant. No delays, no conflicts. However, if efficiency is important, you can use the context manager to perform "grouped writes", which allow for performing a large number of operations with only one write at the end.
- **Reliable**: by default, no caching is used. Every single time you access a `livejson.File`, it's read straight from the file. And every time you write to it, the change is instant. No delays, no conflicts. However, if efficiency is important, you can use the context manager to perform "grouped writes", which allow for performing a large number of operations with only one write at the end.
- **100% test covered** Be confident that `livejson` is working properly

`livejson` can be used for:

- **Database storage**: you can use `livejson` to easily write flexible JSON databases, without having to worry about complex `open` and `close` operations, or learning how to use the `json` module.
- **Debugging**: You can use `livejson` to “back up” your Python objects to disk. If you use a `livejson.Database` instead of a `dict` or a `list` and your script crashes, you'll still have a hard copy of your object. And you barely had to change any of your code.
- **Debugging**: You can use `livejson` to “back up” your Python objects to disk. If you use a `livejson.File` instead of a `dict` or a `list` and your script crashes, you'll still have a hard copy of your object. And you barely had to change any of your code.
- **General-purpose JSON**: If your script or application needs to interact with JSON files in any way, consider using `livejson`, for simplicity's sake. `livejson` can make your code easier to read and understand, and also save you time.

Thanks to [dgelessus](https://github.com/dgelessus) for naming this project.
Expand Down
76 changes: 32 additions & 44 deletions livejson.py → livejson/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -4,39 +4,36 @@
real-time. Magic.
"""

import os
import json
import warnings
from __future__ import annotations

# Import from collections.abc for Python 3.x but incase of ImportError
# from Python 2.x, fall back on importing from collections.
try:
from collections.abc import (
MutableMapping,
MutableSequence,
)
except ImportError:
from collections import (
MutableMapping,
MutableSequence,
)
import enum
import json
import os
from collections.abc import MutableMapping, MutableSequence
from enum import Enum
from typing import Union, Any

warnings.filterwarnings("once", category=DeprecationWarning)
PathLike = Union[str, os.PathLike]


# MISC HELPERS


def _initfile(path, data="dict"):
class _DataType(Enum):
List = enum.auto()
Dict = enum.auto()


def _initfile(path: PathLike, data_type: _DataType = _DataType.Dict) -> bool | None: # TODO: is return really neccessary? Not used anywhere
"""Initialize an empty JSON file."""
data = {} if data.lower() == "dict" else []
data: dict[Any, Any] | list[Any] = {} if data_type is _DataType.Dict else []
# The file will need to be created if it doesn't exist
if not os.path.exists(path): # The file doesn't exist
# Raise exception if the directory that should contain the file doesn't
# exist
dirname = os.path.dirname(path)
if dirname and not os.path.exists(dirname):
raise IOError(
raise IOError( # TODO: better error (IOError is deprecated)
("Could not initialize empty JSON file in non-existant "
"directory '{}'").format(os.path.dirname(path))
)
Expand All @@ -51,11 +48,12 @@ def _initfile(path, data="dict"):
return False


class _ObjectBase(object):
class _ObjectBase:
"""Class inherited by most things.

Implements the lowest common denominator for all emulating classes.
"""

def __getitem__(self, key):
out = self.data[key]

Expand Down Expand Up @@ -99,6 +97,7 @@ class _NestedBase(_ObjectBase):
object, and 'pathToThis' which specifies where in the JSON file this object
exists (as a list).
"""

def __init__(self, fileobj, pathToThis):
self.pathInData = pathToThis
self.base = fileobj
Expand Down Expand Up @@ -151,6 +150,7 @@ class _NestedDict(_NestedBase, MutableMapping):

to update the file.
"""

def __iter__(self):
return iter(self.data)

Expand All @@ -176,6 +176,7 @@ class _NestedList(_NestedBase, MutableSequence):

to update the file.
"""

def insert(self, index, value):
# See _NestedBase.__setitem__ for details on how this works
data = self.base.data
Expand All @@ -194,14 +195,15 @@ class _BaseFile(_ObjectBase):

This implements all the required methods common between
MutableMapping and MutableSequence."""

def __init__(self, path, pretty=False, sort_keys=False):
self.path = path
self.pretty = pretty
self.sort_keys = sort_keys
self.indent = 2 # Default indentation level

_initfile(self.path,
"list" if isinstance(self, ListFile) else "dict")
_DataType.List if isinstance(self, ListFile) else _DataType.Dict)

def _data(self):
"""A simpler version of data to avoid infinite recursion in some cases.
Expand All @@ -210,7 +212,7 @@ def _data(self):
"""
if self.is_caching:
return self.cache
with open(self.path, "r") as f:
with open(self.path) as f:
return json.load(f)

@property
Expand Down Expand Up @@ -266,23 +268,14 @@ def _updateType(self):

# Bonus features!

def set_data(self, data):
"""Equivalent to setting the "data" attribute. Exists for backwards
compatibility."""
warnings.warn(
"set_data is deprecated; please set .data instead.",
DeprecationWarning
)
self.data = data

def remove(self):
"""Delete the file from the disk completely."""
os.remove(self.path)

@property
def file_contents(self):
"""Get the raw file contents of the file."""
with open(self.path, "r") as f:
with open(self.path) as f:
return f.read()

# Grouped writes
Expand Down Expand Up @@ -312,6 +305,7 @@ class DictFile(_BaseFile, MutableMapping):
"""A class emulating Python's dict that will update a JSON file as it is
modified.
"""

def __iter__(self):
return iter(self.data)

Expand All @@ -328,6 +322,7 @@ class ListFile(_BaseFile, MutableSequence):
modified. Use this class directly when creating a new file if you want the
base object to be an array.
"""

def insert(self, index, value):
data = self.data
data.insert(index, value)
Expand All @@ -341,7 +336,7 @@ def clear(self):
self.data = []


class File(object):
class File:
"""The main interface of livejson. Emulates a list or a dict, updating a
JSON file in real-time as it is modified.

Expand All @@ -361,15 +356,15 @@ def __init__(self, path, pretty=False, sort_keys=True, indent=2):

_initfile(self.path)

with open(self.path, "r") as f:
with open(self.path) as f:
data = json.load(f)
if isinstance(data, dict):
self.__class__ = DictFile
elif isinstance(data, list):
self.__class__ = ListFile

@staticmethod
def with_data(path, data, *args, **kwargs):
@classmethod
def with_data(cls, path, data, *args, **kwargs):
"""Initialize a new file that starts out with some data. Pass data
as a list, dict, or JSON string.
"""
Expand All @@ -384,13 +379,6 @@ def with_data(path, data, *args, **kwargs):
"'livejson.File' instance if you really "
"want to do this.")
else:
f = File(path, *args, **kwargs)
f = cls(path, *args, **kwargs)
f.data = data
return f


# Aliases for backwards-compatibility
Database = File
ListDatabase = ListFile
DictDatabase = DictFile

Empty file added livejson/py.typed
Empty file.
Loading