Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
* Add a home for the snippets * Move singleton * Move logger * Move the most basic of the test cases * Improve and test the test case * Move DotDict to snippets * Move has_post * Move SeabornColors * Format black * Standardize test import And always do that __main__ thing for Sam (and anyone else running tests without an IDE) * Fix newlines and other aesthetic stuff * Add newlines before code and list context * Raise the intuitive type of error * Move the files submodule over to snippets It only depends on pathlib and seems generically useful to me, so off to snippets it goes! * Add a shortcut for removing file(s) by name * Refactor: rename * Clean the working directory attribute * Revert the return behaviour on `DirectoryObject.delete` It was just weird. Just explicitly check for empty. Honestly, we can probably revert the `only_if_empty` kwarg too, but that can be its own PR. * Format black * Format black * Format black * add copy() * Add tests and corrections * add additional information * define __str__ * Update tests * make it possible to resolve file path * Make DirectoryObject recognize various init variables * allow absolute path and string directory in FileObject * remove resolve to allow relative path * remove resolve to allow relative path * bugfixes and add tests * restore copy * allow using same file name * add Liam's error message suggestion * Update pyiron_workflow/snippets/files.py Co-authored-by: Liam Huber <[email protected]> * add absolute path * codacy * I should have taken a look at Codacy * I hope this is the correct way to put windows * one more try * awkword mistake * codacy * black * never ending black * windows fix * And of course kind of mistake I would have noticed if I had run the test once * Use consistent and canonical cls/self for (non)classmethod Just correcting some silly little accidents that slipped into the tests. * [minor] Factories instead of meta nodes, and use them for transformer nodes (#293) * Replace evaluated code to-/from-list nodes with actual classes Leveraging custom constructors * Develop meta abstraction and make a new home for transforming meta nodes * Accept node args too * Introduce a decorator for building the class-IO * Change paradigm to whether or not the node uses __reduced__ and a constructor Instead of "Meta" nodes * Allow direct use of Constructed children * Move and update constructed stuff * Add new singleton behaviour so factory-produced classes can pass is-tests * Apply constructed and class registration updates to the transformers * Remove (now unused) meta module * PEP8 newline * Remove unnecessary __getstate__ The object isn't holding instance level state and older versions of python bork here. * Add constructed __*state__ compatibility for older versions * 🐛 add missing `return` * Format black * Introduce a new factory pattern At the cost of requiring factory functions to forgo kwargs, we get object matching for factories and classes, and picklability for instances. Still need to check some edge cases around the use of stuff with a non-trivial qualname. * Test factories as methods * Test local scoping * Add docstrings and hide a function * Simplify super * Give demo class in tests have a more generic __init__ * Test and document multiple inheritance * Stop storing __init__ args By leveraging `__new__` and `__getnewargs_ex__` * Add clearing methods In case you want to change some constructor behaviour and clear the access cache. Not sure what should be cached at the end of the day, but for now just give users a shortcut for clearing it. * Use snippets.factory.classfactory and typing.ClassVar for transformers * Revert singleton * Format black * Remove constructed It's superceded by the snippets.factory stuff * Gently rename things for when the factory comes from a decorator * Allow factory made classes to also come from decorators * Format black --------- Co-authored-by: pyiron-runner <[email protected]> * [minor] Use factories in existing nodes (#303) * Change paradigm to whether or not the node uses __reduced__ and a constructor Instead of "Meta" nodes * Allow direct use of Constructed children * Move and update constructed stuff * Add new singleton behaviour so factory-produced classes can pass is-tests * PEP8 newline * Remove unnecessary __getstate__ The object isn't holding instance level state and older versions of python bork here. * Add constructed __*state__ compatibility for older versions * 🐛 add missing `return` * Format black * Revert singleton * Remove constructed It's superceded by the snippets.factory stuff * Format black * Let the factory clear method take specific names * Don't override __module__ to the factory function If it was explicitly set downstream, leave that. But if the user left it empty, still default it back to the factory function's module * Clean up storage if job tests fail * Make tinybase the default storage backend * Switch Function and Macro over to using classfactory With this, everything is pickleable (unless you slap something unpickleable on top, or define it in a place that can't be reached by pickle like inside a local function scope). The big downside is that `h5io` storage is now basically useless, since all our nodes come from custom reconstructors. Similarly, for the node job `DataContainer` can no longer store the input node. The `tinybase` backend is still working ok, so I made it the default, and I got the node job working again by forcing it to cloudpickle the input node on saving. These are some ugly hacks, but since storage is an alpha feature right now anyhow, I'd prefer to push ahead with pickleability. * Remove unused decorator And reformat tests in the vein of usage in Function and Macro * Format black --------- Co-authored-by: pyiron-runner <[email protected]> * [patch] Make factory made objects cloudpickleable (#305) * Change paradigm to whether or not the node uses __reduced__ and a constructor Instead of "Meta" nodes * Allow direct use of Constructed children * Move and update constructed stuff * Add new singleton behaviour so factory-produced classes can pass is-tests * PEP8 newline * Remove unnecessary __getstate__ The object isn't holding instance level state and older versions of python bork here. * Add constructed __*state__ compatibility for older versions * 🐛 add missing `return` * Format black * Revert singleton * Remove constructed It's superceded by the snippets.factory stuff * Format black * Let the factory clear method take specific names * Don't override __module__ to the factory function If it was explicitly set downstream, leave that. But if the user left it empty, still default it back to the factory function's module * Clean up storage if job tests fail * Make tinybase the default storage backend * Switch Function and Macro over to using classfactory With this, everything is pickleable (unless you slap something unpickleable on top, or define it in a place that can't be reached by pickle like inside a local function scope). The big downside is that `h5io` storage is now basically useless, since all our nodes come from custom reconstructors. Similarly, for the node job `DataContainer` can no longer store the input node. The `tinybase` backend is still working ok, so I made it the default, and I got the node job working again by forcing it to cloudpickle the input node on saving. These are some ugly hacks, but since storage is an alpha feature right now anyhow, I'd prefer to push ahead with pickleability. * Remove unused decorator And reformat tests in the vein of usage in Function and Macro * Format black * Expose concurrent.futures executors on the creator * Only expose the base Executor from pympipool Doesn't hurt us now and prepares for the version bump * Extend `Runnable` to use a non-static method This is significant. `on_run` is no longer a property returning a staticmethod that will be shipped off, but we directly ship off `self.on_run` so `self` goes with it to remote processes. Similarly, `run_args` gets extended to be `tuple[tuple, dict]` so positional arguments can be sent too. Stacked on top of pickleability, this means we can now use standard `concurrent.futures.ProcessPoolExecutor` -- as long as the nodes are all defined somewhere importable, i.e. not in `__main__`. Since working in notebooks is pretty common, the more flexible `pympipool.Executor` is left as the default `Workflow.create.Executor`. This simplifies some stuff under the hood too, e.g. `Function` and `Composite` now just directly do their thing in `on_run` instead of needing the misdirection of returning their own static methods. * Format black * Expose concurrent.futures executors on the creator * Only expose the base Executor from pympipool Doesn't hurt us now and prepares for the version bump * Extend `Runnable` to use a non-static method This is significant. `on_run` is no longer a property returning a staticmethod that will be shipped off, but we directly ship off `self.on_run` so `self` goes with it to remote processes. Similarly, `run_args` gets extended to be `tuple[tuple, dict]` so positional arguments can be sent too. Stacked on top of pickleability, this means we can now use standard `concurrent.futures.ProcessPoolExecutor` -- as long as the nodes are all defined somewhere importable, i.e. not in `__main__`. Since working in notebooks is pretty common, the more flexible `pympipool.Executor` is left as the default `Workflow.create.Executor`. This simplifies some stuff under the hood too, e.g. `Function` and `Composite` now just directly do their thing in `on_run` instead of needing the misdirection of returning their own static methods. * Format black * Compute qualname if not provided * Fail early if there is a <locals> function in the factory made hierarchy * Skip the factory fanciness if you see <locals> This enables _FactoryMade objects to be cloudpickled, even when they can't be pickled, while still not letting the mere fact that they are dynamic classes stand in the way of pickling. Nicely lifts our constraint on the node job interaction with pyiron base, which was leveraging cloudpickle * Format black * Test ClassFactory this way too --------- Co-authored-by: pyiron-runner <[email protected]> * Relocate copied content And get rid of the test-case. IMO it's over-complicated and the new docstring tests are preferable. * Update dependencies and CI * Update dev status * Replace the example With something trivial but functional * [dependabot skip] Update env file * Format black * Add a trivial dependency so the merger has something to look at * [dependabot skip] Update env file * Remove benchmark directory As it has no tests in it * Disable benchmark tests --------- Co-authored-by: pyiron-runner <[email protected]> Co-authored-by: samwaseda <[email protected]> Co-authored-by: Sam Dareska <[email protected]>
- Loading branch information
4 people
authored
May 23, 2024
1 parent
569efdf
commit e778f42