Issue #116 Include layer information to stanford.asdb AS categories #126
Conversation
…asdb AS categories
There was a problem hiding this comment.
Pretty much good to go, functionally no problems, only two nitpicks (I would not have mentioned the linebreak, but since you'll touch the file anyways for the other comment I mentioned it).
Btw. I prefer using something like
if not category:
continue
asns.add(asn)this prevents having so many nested levels of code. But purely stylistic, you don't have to change this.
iyp/crawlers/stanford/asdb.py
Outdated
| # Remove 'Other' subcategories | ||
| # Only store their parent category | ||
| if category == 'Other' or category == 'other': | ||
| layer = 1 |
There was a problem hiding this comment.
I guess we can just continue here, since we already added this link in the previous loop iteration in the if above?
iyp/crawlers/stanford/asdb.py
Outdated
|
|
||
| asn_qid = asn_id[asn] | ||
| category_qid = category_id[category] | ||
|
|
||
| links.append({'src_id': asn_qid, 'dst_id': category_qid, 'props': [self.reference]}) # Set AS category | ||
| links.append({'src_id': asn_qid, 'dst_id': category_qid, 'props': [ |
There was a problem hiding this comment.
Can you break this line before 'props'? :)
|
As a side comment, I checked that the PART_OF links are well behaved MATCH (n: Tag)-[r:PART_OF {reference_name: "stanford.asdb"}]-(m: Tag) return n, r, m As shown in this screenshot, it's almost the case the exception (left-most graph) being caused by the error mentioned in #116 (comment) |
Description
Added a
layerproperty to links -[r:CATEGORIZED {reference_name: "stanford.asdb"}]-Added a
PART_OFrelationship (subcategory) -> (category)Removed
OtherandothersubcategoriesMotivation and Context
Issue 116
How Has This Been Tested?
Thorough check of my file parsing (noticed new errors).
Overall number of links -[r:CATEGORIZED {reference_name: "stanford.asdb"}]-:
before 394 834, now roughly twice more 592 754
Pushed and observed changes on my local iyp
Screenshots (if appropriate):
Types of changes
Checklist: