Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

added a method for creating an empty table, given metadata #54

Merged
merged 4 commits into from
Jul 3, 2024

Conversation

rishsriv
Copy link
Member

@rishsriv rishsriv commented Jul 3, 2024

This is useful in situations where we have metadata that has been previously provided as - say - a CSV. With this, we can create empty tables that match the metadata, and test the model's performance.

@rishsriv rishsriv requested a review from wongjingping July 3, 2024 06:08
Copy link
Collaborator

@wongjingping wongjingping left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Have some suggestions, feel free to take it or leave it!

Comment on lines 306 to 307
table_name = table_name.split(".")[-1]
schema_name = table_name.split(".")[0]
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: we might want to limit the split to 1 in case there are multiple . present:

table_name.split(".", 1)[-1]

"""
metadata = self.get_metadata(format="json", dev=dev)
if self.db_type == "sqlserver":
ddl = create_ddl_from_metadata(metadata, if_not_exists=False)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems like if_not_exists isn't defined as an argument for create_ddl_from_metadata - would this result in an error?

else:
ddl = create_ddl_from_metadata(metadata)

if self.db_type == "postgres" or self.db_type == "redshift":
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: do we want to wrap the code from here to the end below in a try/except block in case the metadata is malformed and can't be run? We can also just bubble common things like ImportError through.

@rishsriv
Copy link
Member Author

rishsriv commented Jul 3, 2024

Thank you for the suggestions! Fixed!

@rishsriv rishsriv merged commit 39dc770 into main Jul 3, 2024
2 checks passed
@rishsriv rishsriv deleted the rishabh/create-empty-tables branch July 3, 2024 08:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants