Data Migration Script Draft #19
Conversation
|
I have added the functionality to filter by |
| organization_query=SELECT [columns] FROM entity WHERE entity.id > %s and (type = 'o' or type = 'O') and entity.id in (%s) ORDER BY entity.id LIMIT %s; | ||
| planter_query=SELECT [columns] FROM planter, entity WHERE planter.id > %s and entity.id = planter.organization_id and (entity.type = 'o' or entity.type = 'O') and planter.organization_id in (%s) ORDER BY planter.id LIMIT %s; | ||
| tree_query=SELECT [columns] FROM trees, planter, entity WHERE trees.id > %s and planter_id = planter.id and entity.id = planter.organization_id and (entity.type = 'o' or entity.type = 'O') and entity.id in (%s) ORDER BY trees.id LIMIT %s; |
There was a problem hiding this comment.
I feel you don't need these settings anymore, right?
| if entity_ids: # use the specified entity id as filter condition | ||
| query = query[:-1] + " WHERE " | ||
| filter_column = "id" if table_name == "entity" else "organization_id" if table_name == "planter" else "planter_id" | ||
| entity_ids_str = ", ".join(map(str, entity_ids)) |
There was a problem hiding this comment.
@1nzexi I can not understand logic here, how we get ids here for filter? I didn't find the place to update this entity_ids, where the data from?
There was a problem hiding this comment.
The entity_id is a python list defined on line 221. Currently, it requires users to manually specify the ids in the entity_id list. While this is obviously not a robust way of specifying input, I figure it would be a great place to start testing out the functionality of this script. Please let me know if you think there needs to be a better way to specify the ids to filter that better integrate with the existing workflows. Open to all ideas!
There was a problem hiding this comment.
Also, sorry for taking so long to reply. Somehow github did not notify me of your comment.
Currently, this python script is able to migrate data from test environment's
entity,planter,treestable to dev environment'sentity,planter,trees, as well as resolving the differences of columns in tables of the two environments.However, I have not implemented the functionality which allows users to only migrate rows with specified
idin the entity table, which means the code currently migrates all rows. I'd really appreciate it if you could review the code and tell me how I should expose the interface that allows users to specify the desiredentity.id(Currently, it requires the user to manually edit theentity_idslist, which I doubt to be good enough).