Skip to content

Commit

Permalink
add tables for data sources, node types, and relationship types
Browse files Browse the repository at this point in the history
  • Loading branch information
@romain-fontugne
romain-fontugne committed Aug 14, 2024
1 parent 70b6bd5 commit c7d86cd
Show file tree
Hide file tree
Showing 3 changed files with 102 additions and 0 deletions.
41 changes: 41 additions & 0 deletions documentation/data-sources.md
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,41 @@

# Datasets and corresponding organization providing data to IYP


| Organization | Dataset Name / Description | URL |
|-----------------------------|----------------------------------------------|-----------------------------------------------------------------------|
| Alice-LG | IXP route server looking glass snapshots | https://github.com/alice-lg/alice-lg} |
| | AMS-IX | https://lg.ams-ix.net} |
| | BCIX | https://lg.bcix.de} |
| | DE-CIX | https://lg.de-cix.net} |
| | IX.br | https://lg.ix.br} |
| | LINX | https://alice-rs.linx.net} |
| | Megaport | https://lg.megaport.com} |
| | Netnod | https://lg.netnod.se} |
| APNIC | AS population estimate | https://stats.labs.apnic.net/aspop} |
| BGPKIT | as2rel, peer-stats, pfx2as | https://data.bgpkit.com} |
| BGP.Tools | AS names, AS tags | https://bgp.tools/kb/api} |
| | Anycast prefix tags | https://github.com/bgptools/anycast-prefixes} |
| CAIDA | AS Rank | https://doi.org/10.21986/CAIDA.DATA.AS-RANK} |
| | IXPs Dataset | https://www.caida.org/catalog/datasets/ixps} |
| Cisco | Umbrella Popularity List | https://s3-us-west-1.amazonaws.com/umbrella-static/index.html} |
| Citizen Lab | URL testing lists | https://github.com/citizenlab/test-lists} |
| Cloudflare | Cloudflare Radar API endpoints | https://radar.cloudflare.com}} |
| | radar/dns/top/ases, radar/dns/top/locations, | |
| | radar/ranking/top, radar/datasets | |
| Emile Aben | AS names | https://github.com/emileaben/asnames} |
| IHR | Country Dependency, AS Hegemony, ROV | https://ihr.iijlab.net} |
| Internet Intelligence Lab | AS to Organization Mapping | https://github.com/InetIntel/Dataset-AS-to-Organization-Mapping} |
| NRO | Extended allocation and assignment reports | https://www.nro.net/about/rirs/statistics} |
| OpenINTEL | tranco1m, umbrella1m, ns | https://data.openintel.nl/data} |
| | DNS Dependency Graph | https://dnsgraph.dacs.utwente.nl} |
| Packet Clearing House | Daily routing snapshots | https://www.pch.net/resources/Routing\_Data} |
| PeeringDB | API endpoints: fac, ix, ixlan, netfac, org | https://www.peeringdb.com} |
| RIPE NCC | AS names, RPKI | https://ftp.ripe.net/ripe} |
| | RIPE Atlas measurement information | https://atlas.ripe.net} |
| SimulaMet | rDNS data | https://rir-data.org} |
| Stanford | ASdb dataset | https://asdb.stanford.edu} |
| Tranco | Tranco list | https://tranco-list.eu} |
| Virginia Tech | RoVista | https://rovista.netsecurelab.org} |
| World Bank | Indicators API: Country Population Indicator | https://www.worldbank.org} |

30 changes: 30 additions & 0 deletions documentation/node_types.md
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@

# Node types available in IYP

| Node types | Description |
|-------------------------|-----------------------------------------------------------------------------------------------------------------------------------|
| AS | Autonomous System, uniquely identified with the **asn** property. |
| AtlasMeasurement | RIPE Atlas Measurement, uniquely identified with the **id** property. |
| AtlasProbe | RIPE Atlas probe, uniquely identified with the **id** property. |
| AuthoritativeNameServer | Authoritative DNS nameserver for a set of domain names, uniquely identified with the **name** property. |
| BGPCollector | A RIPE RIS or RouteViews BGP collector, uniquely identified with the **name** property. |
| CaidaIXID | Unique identifier for IXPs from CAIDA's IXP dataset. |
| Country | Represent an economy, uniquely identified by either its two or three character code (properties **country_code** and **alpha3**). |
| DomainName | Any DNS domain name that is not a FQDN (see HostName), uniquely identified by the **name** property. |
| Estimate | Represent a report that approximate a quantity, for example the World Bank population estimate. |
| Facility | Co-location facility for IXPs and ASes, uniquely identified by the **name** property. |
| HostName | A fully qualified domain name uniquely identified by the **name** property. |
| IP | An IPv4 or IPv6 address uniquely identified by the **ip** property. The **af** property (address family) provides the IP version of the prefix.|
| IXP | An Internet Exchange Point, loosely identified by the **name** property or using related IDs (see the EXTERNAL_ID relationship). |
| Name | Represent a name that could be associated to a network resource (e.g. an AS), uniquely identified by the **name** property. |
| OpaqueID | Represent the opaque-id value found in RIR's delegated files. Resources related to the same opaque-id are registered to the same resource holder. Uniquely identified by the **id** property.|
| Organization | Represent an organization and is loosely identified by the **name** property or using related IDs (see the EXTERNAL_ID relationship).|
| PeeringdbFacID | Unique identifier for a Facility as assigned by PeeringDB. |
| PeeringdbIXID | Unique identifier for an IXP as assigned by PeeringDB. |
| PeeringdbNetID | Unique identifier for an AS as assigned by PeeringDB. |
| PeeringdbOrgID | Unique identifier for an Organization as assigned by PeeringDB. |
| Prefix | An IPv4 or IPv6 prefix uniquely identified by the **prefix** property. The **af** property (address family) provides the IP version of the prefix.|
| Ranking | Represent a specific ranking of Internet resources (e.g. CAIDA's ASRank or Tranco ranking). The rank value for each resource is given by the RANK relationship.|
| Tag | The output of a classification. A tag can be the result of a manual or automated classification. Uniquely identified by the **label** property.|
| URL | The full URL for an Internet resource, uniquely identified by the **url** property. |

31 changes: 31 additions & 0 deletions documentation/relationship_types.md
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@

# Relationships available in IYP


| Relationship | Description |
|----------------------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| ALIAS_OF | Equivalent to the CNAME record in DNS. It relates two HostNames. |
| ASSIGNED | Represent the allocation by a RIR of a network resource (AS, Prefix) to a resource holder (see OpaqueID). Or represent the assigned IP address of an AtlasProbe. |
| AVAILABLE | Relate ASes and Prefixes to RIRs (in the form of an OpaqueID) meaning that the resource is not allocated and available at the related RIR. |
| CATEGORIZED | Relate a network resource (AS, Prefix, URL) to a Tag, meaning that the resource has been classified accordingly to the Tag. The **reference_name** property provide the name of the original dataset/classifier. |
| COUNTRY | Relate any node to its corresponding country. This relation may have different meaning depending on the original dataset (e.g. geo-location or registration). |
| DEPENDS_ON | Relate an AS or Prefix to an AS, meaning the reachability of the AS/Prefix depends on a certain AS. |
| EXTERNAL_ID | Relate a node to an identifier commonly used by an organization. For example, PeeringDB assigns unique identifiers to IXPs (see PeeringdbIXID). |
| LOCATED_IN | Location of a resource at a specific geographical or topological location. For example, co-location Facility for an IXP or AS for an AtlasProbe. |
| MANAGED_BY | Entity in charge of a network resource. For example an AS is managed by an Organization, a DomainName is managed by an AuthoritativeNameServer. |
| MEMBER_OF | Represent the membership to an organization. For example, an AS is member of an IXP. |
| NAME | Relate an entity to its usual or registered name. For example, the name of an AS. |
| ORIGINATE | Relate a Prefix to an AS, meaning that the prefix is seen as being originated from that AS in BGP. |
| PARENT | Relate two DomainNames and represent a zone cut between the parent zone and the more specific zone. |
| PART_OF | Represent that one entity is a part of another. For example, an IP address is a part of an IP Prefix, a HostName is a part of a DomainName. |
| PEERS_WITH | Represent the connection between two ASes as seen in BGP. It also include peerings between ASes and BGPCollectors. |
| POPULATION | Indicate that an AS hosts a certain fraction of the population of a country. It also represent the estimated population of a country. |
| QUERIED_FROM | Relate a DomainName to an AS or Country, meaning that the AS or Country appears in the Top 100AS or Country to query the most the DomainName (as reported by Cloudflare radar). |
| RANK | Relate a resource to a Ranking, meaning that the resource appears in the Ranking. The **rank** property gives the exact rank position. |
| RESERVED | Indicate that an AS or Prefix is reserved for a certain purpose by RIRs or IANA. |
| RESOLVES_TO | Relate a HostName to an IP address, meaning that a DNS resolution resolved the corresponding IP. |
| ROUTE_ORIGIN_AUTHORIZATION | Relate an AS and a Prefix, meaning that the AS is authorized to originate the Prefix according to RPKI. |
| SIBLING_OF | Relate ASes or Organization together, meaning that they represent the same entity. |
| TARGET | Relate an AtlasMeasurement to an IP, HostName, or AS, meaning that an Atlas measurement is setup to probe that resource. |
| WEBSITE | Relate a URL to an Organization, Facility, IXP, AS, representing a common website for the resource. |

0 comments on commit c7d86cd

Please sign in to comment.