[bug] Loading a custom controller from a different file

[thomas-whaley-poco]

Bug report checklis

• Searched the issues page for similar reports

• Read the relevant sections of the documentation

• Browse the tutorials and tests for usefull code snippets and examples of use

• Reproduced the issue after updating with pip install --upgrade pandapower (or git pull)

• Tried basic troubleshooting (if a bug/error) like restarting the interpreter and checking the pythonpath

Reproducible Example

import pandapower as pp
import pandapower.networks
from pandapower.control.controller.trafo.ContinuousTapControl import ContinuousTapControl

class CustomTapController(ContinuousTapControl):
    pass

net = pandapower.networks.example_simple()
CustomTapController(net, 0, vm_set_pu=1)

# Save the network to be loaded from a different file
pp.to_json(net, "net.json")
pp.to_pickle(net, "net.p")  # Let's try a pickle file too!

---------------------------------

# In a different file now
import pandapower as pp

# These two lines fail
pp.from_json("net.json")
pp.from_pickle(net, "net.p")

---------------------------------

Issue Description and Traceback

Loading a model with pp.from_json(...) uses a relative reference to a custom object. I am saving these models in cloud storage and download them from automated scripts. The file containing the custom object is not placed in the same module as when it was saved, meaning it cannot be loaded with the current loading system.

Traceback (json):

Traceback (most recent call last):
  File "~/test/example.py", line 3, in <module>
    pp.from_json("net.json")
  File "~/test/.venv/lib/python3.11/site-packages/pandapower/file_io.py", line 289, in from_json
    return from_json_string(
           ^^^^^^^^^^^^^^^^^
  File "~/test/.venv/lib/python3.11/site-packages/pandapower/file_io.py", line 349, in from_json_string
    net = json.loads(json_string, cls=io_utils.PPJSONDecoder,
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/json/__init__.py", line 359, in loads
    return cls(**kw).decode(s)
           ^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/json/decoder.py", line 337, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/json/decoder.py", line 353, in raw_decode
    obj, end = self.scan_once(s, idx)
               ^^^^^^^^^^^^^^^^^^^^^^
  File "~/test/.venv/lib/python3.11/site-packages/pandapower/io_utils.py", line 700, in pp_hook
    return fs.from_serializable()
           ^^^^^^^^^^^^^^^^^^^^^^
  File "~/test/.venv/lib/python3.11/site-packages/pandapower/io_utils.py", line 567, in DataFrame
    df[col] = df[col].apply(self.pp_hook)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "~/test/.venv/lib/python3.11/site-packages/pandas/core/series.py", line 4764, in apply
    ).apply()
      ^^^^^^^
  File "~/test/.venv/lib/python3.11/site-packages/pandas/core/apply.py", line 1209, in apply
    return self.apply_standard()
           ^^^^^^^^^^^^^^^^^^^^^
  File "~/test/.venv/lib/python3.11/site-packages/pandas/core/apply.py", line 1289, in apply_standard
    mapped = obj._map_values(
             ^^^^^^^^^^^^^^^^
  File "~/test/.venv/lib/python3.11/site-packages/pandas/core/base.py", line 921, in _map_values
    return algorithms.map_array(arr, mapper, na_action=na_action, convert=convert)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "~/test/.venv/lib/python3.11/site-packages/pandas/core/algorithms.py", line 1814, in map_array
    return lib.map_infer(values, mapper, convert=convert)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "lib.pyx", line 2926, in pandas._libs.lib.map_infer
  File "~/test/.venv/lib/python3.11/site-packages/pandapower/io_utils.py", line 700, in pp_hook
    return fs.from_serializable()
           ^^^^^^^^^^^^^^^^^^^^^^
  File "~/test/.venv/lib/python3.11/site-packages/pandapower/io_utils.py", line 616, in rest
    class_ = getattr(module, self.class_name)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AttributeError: module '__main__' has no attribute 'CustomTapController'

Traceback (pickle):

Traceback (most recent call last):
  File "~/ex.py", line 3, in <module>
    pp.from_pickle("net.p")
  File "~/.venv/lib/python3.11/site-packages/pandapower/file_io.py", line 167, in from_pickle
    net = pandapowerNet(io_utils.get_raw_data_from_pickle(filename))
                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "~/.venv/lib/python3.11/site-packages/pandapower/io_utils.py", line 312, in get_raw_data_from_pickle
    net = read(f)
          ^^^^^^^
  File "~/.venv/lib/python3.11/site-packages/pandapower/io_utils.py", line 304, in read
    return pd.read_pickle(f)
           ^^^^^^^^^^^^^^^^^
  File "~/.venv/lib/python3.11/site-packages/pandas/io/pickle.py", line 211, in read_pickle
    return pc.load(handles.handle, encoding=None)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "~/.venv/lib/python3.11/site-packages/pandas/compat/pickle_compat.py", line 231, in load
    return up.load()
           ^^^^^^^^^
  File "/usr/lib/python3.11/pickle.py", line 1213, in load
    dispatch[key[0]](self)
  File "/usr/lib/python3.11/pickle.py", line 1529, in load_global
    klass = self.find_class(module, name)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "~/.venv/lib/python3.11/site-packages/pandas/compat/pickle_compat.py", line 162, in find_class
    return super().find_class(module, name)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/pickle.py", line 1584, in find_class
    return getattr(sys.modules[module], name)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AttributeError: module '__main__' has no attribute 'CustomTapController'

Expected Behavior

I expect that a custom object is serialized with pickle/some other serializer and therefore doesn't need to be linked with the correct file structure.

Installed Versions

python              : 3.11.2.final.0
python-bits         : 64
OS                  : Linux
OS-release          : 5.15.167.4-microsoft-standard-WSL2
machine             : x86_64
byteorder           : little
LC_ALL              : None
LANG                : en_US.UTF-8
LOCALE              : en_US.UTF-8

pandapower          : 2.14.10
pandas              : 2.1.4
numpy               : 1.26.4
pytz                : 2024.2
dateutil            : 2.9.0.post0
setuptools          : 66.1.1
pip                 : 23.0.1
pytest              : 7.4.4
psycopg2            : 2.9.10
jinja2              : 3.1.5
IPython             : 8.31.0
bs4                 : 4.12.3
fsspec              : 2024.12.0
gcsfs               : 2024.12.0
matplotlib          : 3.10.0
openpyxl            : 3.1.5
pandas_gbq          : 0.24.0
pyarrow             : 16.1.0
scipy               : 1.13.1
sqlalchemy          : 1.4.54
tabulate            : 0.9.0
tzdata              : 2024.2

Label

• Relevant labels are selected