You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
This requires that the user can manage the data shuffling themselves and in turn implicitly requires that both sides of the export & import exist for longer than the django command. For environments such as Kubernetes, where a pod may only exist while a command is running, this pattern is a little harder to use. It's definitely still possible though can require a bit more juggling, especially if trying to script the usage rather than use interactively.
I was wondering whether it would make sense for devdata to provide some more direct support for this sort of use-case.
Approaches I can think of which might help:
let devdata export to/import from a (compressed) tar file directly, removing the need to have tar in the target environment (as an implementation, this might e.g: use a local temporary directory rather than reading the archive for each file)
maybe allow for that archive to come from a Django storage backed location
and/or allow for the import/export folders to be universal paths
Universal paths could be used on their own, though given the number of files which devdata tends to create that's likely to be quite inefficient -- I suspect that upaths would probably be more useful as a path to an archive file rather than to a directory.
Open to other ideas too, including if there's an easy way to use devdata in kubernetes which I've missed (if so, perhaps we could document this too?)
The text was updated successfully, but these errors were encountered:
Currently the summary of usage for this package looks like:
This requires that the user can manage the data shuffling themselves and in turn implicitly requires that both sides of the export & import exist for longer than the django command. For environments such as Kubernetes, where a pod may only exist while a command is running, this pattern is a little harder to use. It's definitely still possible though can require a bit more juggling, especially if trying to script the usage rather than use interactively.
I was wondering whether it would make sense for devdata to provide some more direct support for this sort of use-case.
Approaches I can think of which might help:
tar
in the target environment (as an implementation, this might e.g: use a local temporary directory rather than reading the archive for each file)Universal paths could be used on their own, though given the number of files which devdata tends to create that's likely to be quite inefficient -- I suspect that upaths would probably be more useful as a path to an archive file rather than to a directory.
Open to other ideas too, including if there's an easy way to use devdata in kubernetes which I've missed (if so, perhaps we could document this too?)
The text was updated successfully, but these errors were encountered: