Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add db specific node class registry #806

Merged
merged 3 commits into from
Jul 2, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
55 changes: 55 additions & 0 deletions doc/source/extending.rst
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,16 @@ Creating purely abstract classes is achieved using the `__abstract_node__` prope
self.balance = self.balance + int(amount)
self.save()

Custom label
------------
By default, neomodel uses the class name as the label for nodes. This can be overridden by setting the __label__ property on the class::

class PersonClass(StructuredNode):
__label__ = "Person"
name = StringProperty(unique_index=True)

Creating a PersonClass instance and saving it to the database will result in a node with the label "Person".


Optional Labels
---------------
Expand Down Expand Up @@ -131,11 +141,16 @@ Consider for example the following snippet of code::
class PilotPerson(BasePerson):
pass


class UserClass(StructuredNode):
__label__ = "User"

Once this script is executed, the *node-class registry* would contain the following entries: ::

{"BasePerson"} --> class BasePerson
{"BasePerson", "TechnicalPerson"} --> class TechnicalPerson
{"BasePerson", "PilotPerson"} --> class PilotPerson
{"User"} --> class UserClass

Therefore, a ``Node`` with labels ``"BasePerson", "TechnicalPerson"`` would lead to the instantiation of a
``TechnicalPerson`` object. This automatic resolution is **optional** and can be invoked automatically via
Expand Down Expand Up @@ -184,12 +199,52 @@ This automatic class resolution however, requires a bit of caution:
``{"BasePerson", "PilotPerson"}`` to ``PilotPerson`` **in the global scope** with a mapping of the same
set of labels but towards the class defined within the **local scope** of ``some_function``.

3. Two classes with different names but the same __label__ override will also result in a ``ClassAlreadyDefined`` exception.
This can be avoided under certain circumstances, as explained in the next section on 'Database specific labels'.

Both ``ModelDefinitionMismatch`` and ``ClassAlreadyDefined`` produce an error message that returns the labels of the
node that created the problem (either the `Node` returned from the database or the class that was attempted to be
redefined) as well as the state of the current *node-class registry*. These two pieces of information can be used to
debug the model mismatch further.


Database specific labels
------------------------
**Only for Neo4j Enterprise Edition, with multiple databases**

In some cases, it is necessary to have a class with a label that is not unique across the database.
This can be achieved by setting the `__target_databases__` property to a list of strings ::
class PatientOne(AsyncStructuredNode):
__label__ = "Patient"
__target_databases__ = ["db_one"]
name = StringProperty()

class PatientTwo(AsyncStructuredNode):
__label__ = "Patient"
__target_databases__ = ["db_two"]
identifier = StringProperty()

In this example, both `PatientOne` and `PatientTwo` have the label "Patient", but these will be mapped in a database-specific *node-class registry*.

Now, if you fetch a node with label Patient from your database with auto resolution enabled, neomodel will try to resolve it to the correct class
based on the database it was fetched from ::
db.set_connection("bolt://neo4j:password@localhost:7687/db_one")
patients = db.cypher_query("MATCH (n:Patient) RETURN n", resolve_objects=True) --> instance of PatientOne

The following will result in a ``ClassAlreadyDefined`` exception, because when retrieving from ``db_one``,
neomodel would not be able to decide which model to parse into ::
class GeneralPatient(AsyncStructuredNode):
__label__ = "Patient"
name = StringProperty()

class PatientOne(AsyncStructuredNode):
__label__ = "Patient"
__target_databases__ = ["db_one"]
name = StringProperty()

.. warning:: This does not prevent you from saving a node to the "wrong database". So you can still save an instance of PatientTwo to database "db_one".


``neomodel`` under multiple processes and threads
-------------------------------------------------
It is very important to realise that neomodel preserves a mapping of the set of labels associated with the Neo4J
Expand Down
88 changes: 57 additions & 31 deletions neomodel/async_/core.py
Original file line number Diff line number Diff line change
Expand Up @@ -91,6 +91,7 @@
"""

_NODE_CLASS_REGISTRY = {}
_DB_SPECIFIC_CLASS_REGISTRY = {}

def __init__(self):
self._active_transaction = None
Expand Down Expand Up @@ -352,13 +353,42 @@
# Consequently, the type checking was changed for both
# Node, Relationship objects
if isinstance(object_to_resolve, Node):
return self._NODE_CLASS_REGISTRY[
frozenset(object_to_resolve.labels)
].inflate(object_to_resolve)
_labels = frozenset(object_to_resolve.labels)
if _labels in self._NODE_CLASS_REGISTRY:
return self._NODE_CLASS_REGISTRY[_labels].inflate(object_to_resolve)
elif (
self._database_name is not None
and self._database_name in self._DB_SPECIFIC_CLASS_REGISTRY
and _labels in self._DB_SPECIFIC_CLASS_REGISTRY[self._database_name]
):
return self._DB_SPECIFIC_CLASS_REGISTRY[self._database_name][
_labels
].inflate(object_to_resolve)
else:
raise NodeClassNotDefined(
object_to_resolve,
self._NODE_CLASS_REGISTRY,
self._DB_SPECIFIC_CLASS_REGISTRY,
)

if isinstance(object_to_resolve, Relationship):
rel_type = frozenset([object_to_resolve.type])
return self._NODE_CLASS_REGISTRY[rel_type].inflate(object_to_resolve)
if rel_type in self._NODE_CLASS_REGISTRY:
return self._NODE_CLASS_REGISTRY[rel_type].inflate(object_to_resolve)
elif (
self._database_name is not None
and self._database_name in self._DB_SPECIFIC_CLASS_REGISTRY
and rel_type in self._DB_SPECIFIC_CLASS_REGISTRY[self._database_name]
):
return self._DB_SPECIFIC_CLASS_REGISTRY[self._database_name][

Check warning on line 383 in neomodel/async_/core.py

View check run for this annotation

Codecov / codecov/patch

neomodel/async_/core.py#L383

Added line #L383 was not covered by tests
rel_type
].inflate(object_to_resolve)
else:
raise RelationshipClassNotDefined(
object_to_resolve,
self._NODE_CLASS_REGISTRY,
self._DB_SPECIFIC_CLASS_REGISTRY,
)

if isinstance(object_to_resolve, Path):
from neomodel.async_.path import AsyncNeomodelPath
Expand Down Expand Up @@ -388,30 +418,13 @@
# Object resolution occurs in-place
for a_result_item in enumerate(result_list):
for a_result_attribute in enumerate(a_result_item[1]):
try:
# Primitive types should remain primitive types,
# Nodes to be resolved to native objects
resolved_object = a_result_attribute[1]

resolved_object = self._object_resolution(resolved_object)

result_list[a_result_item[0]][
a_result_attribute[0]
] = resolved_object

except KeyError as exc:
# Not being able to match the label set of a node with a known object results
# in a KeyError in the internal dictionary used for resolution. If it is impossible
# to match, then raise an exception with more details about the error.
if isinstance(a_result_attribute[1], Node):
raise NodeClassNotDefined(
a_result_attribute[1], self._NODE_CLASS_REGISTRY
) from exc

if isinstance(a_result_attribute[1], Relationship):
raise RelationshipClassNotDefined(
a_result_attribute[1], self._NODE_CLASS_REGISTRY
) from exc
# Primitive types should remain primitive types,
# Nodes to be resolved to native objects
resolved_object = a_result_attribute[1]

resolved_object = self._object_resolution(resolved_object)

result_list[a_result_item[0]][a_result_attribute[0]] = resolved_object

return result_list

Expand Down Expand Up @@ -1083,10 +1096,23 @@
possible_label_combinations.append(base_label_set)

for label_set in possible_label_combinations:
if label_set not in adb._NODE_CLASS_REGISTRY:
adb._NODE_CLASS_REGISTRY[label_set] = cls
if not hasattr(cls, "__target_databases__"):
if label_set not in adb._NODE_CLASS_REGISTRY:
adb._NODE_CLASS_REGISTRY[label_set] = cls
else:
raise NodeClassAlreadyDefined(
cls, adb._NODE_CLASS_REGISTRY, adb._DB_SPECIFIC_CLASS_REGISTRY
)
else:
raise NodeClassAlreadyDefined(cls, adb._NODE_CLASS_REGISTRY)
for database in cls.__target_databases__:
if database not in adb._DB_SPECIFIC_CLASS_REGISTRY:
adb._DB_SPECIFIC_CLASS_REGISTRY[database] = {}
if label_set not in adb._DB_SPECIFIC_CLASS_REGISTRY[database]:
adb._DB_SPECIFIC_CLASS_REGISTRY[database][label_set] = cls
else:
raise NodeClassAlreadyDefined(
cls, adb._NODE_CLASS_REGISTRY, adb._DB_SPECIFIC_CLASS_REGISTRY
)


NodeBase = NodeMeta("NodeBase", (AsyncPropertyManager,), {"__abstract_node__": True})
Expand Down
5 changes: 4 additions & 1 deletion neomodel/async_/relationship_manager.py
Original file line number Diff line number Diff line change
Expand Up @@ -457,7 +457,10 @@ def __init__(
is_parent = issubclass(model_from_registry, model)
if is_direct_subclass(model, AsyncStructuredRel) and not is_parent:
raise RelationshipClassRedefined(
relation_type, adb._NODE_CLASS_REGISTRY, model
relation_type,
adb._NODE_CLASS_REGISTRY,
adb._DB_SPECIFIC_CLASS_REGISTRY,
model,
)
else:
adb._NODE_CLASS_REGISTRY[label_set] = model
Expand Down
34 changes: 30 additions & 4 deletions neomodel/exceptions.py
Original file line number Diff line number Diff line change
Expand Up @@ -38,17 +38,27 @@ class ModelDefinitionException(NeomodelException):
Abstract exception to handle error conditions related to the node-to-class registry.
"""

def __init__(self, db_node_rel_class, current_node_class_registry):
def __init__(
self,
db_node_rel_class,
current_node_class_registry,
current_db_specific_node_class_registry,
):
"""
Initialises the exception with the database node that caused the missmatch.

:param db_node_rel_class: Depending on the concrete class, this is either a Neo4j driver node object
from the DBMS, or a data model class from an application's hierarchy.
:param current_node_class_registry: Dictionary that maps frozenset of
node labels to model classes
:param current_db_specific_node_class_registry: Dictionary that maps frozenset of
node labels to model classes for specific databases
"""
self.db_node_rel_class = db_node_rel_class
self.current_node_class_registry = current_node_class_registry
self.current_db_specific_node_class_registry = (
current_db_specific_node_class_registry
)

def _get_node_class_registry_formatted(self):
"""
Expand All @@ -57,13 +67,23 @@ def _get_node_class_registry_formatted(self):

:return: str
"""
ncr_items = list(
output = "\n".join(
map(
lambda x: f"{','.join(x[0])} --> {x[1]}",
self.current_node_class_registry.items(),
)
)
return "\n".join(ncr_items)
for db, db_registry in self.current_db_specific_node_class_registry.items():
output += f"\n\nDatabase-specific: {db}\n"
output += "\n".join(
list(
map(
lambda x: f"{','.join(x[0])} --> {x[1]}",
db_registry.items(),
)
)
)
return output


class NodeClassNotDefined(ModelDefinitionException):
Expand Down Expand Up @@ -102,6 +122,7 @@ def __init__(
self,
db_rel_class_type,
current_node_class_registry,
current_db_specific_node_class_registry,
remapping_to_class,
):
"""
Expand All @@ -110,11 +131,16 @@ def __init__(
:param db_rel_class_type: The type of the relationship that caused the error.
:type db_rel_class_type: str (The label of the relationship that caused the error)
:param current_node_class_registry: The current db object's node-class registry.
:param current_db_specific_node_class_registry: The current db object's node-class registry for specific databases.
:type current_node_class_registry: dict
:param remapping_to_class: The relationship class the relationship type was attempted to be redefined to.
:type remapping_to_class: class
"""
super().__init__(db_rel_class_type, current_node_class_registry)
super().__init__(
db_rel_class_type,
current_node_class_registry,
current_db_specific_node_class_registry,
)
self.remapping_to_class = remapping_to_class

def __str__(self):
Expand Down
Loading
Loading