Skip to content

Commit

Permalink
Add atlas_measurements docs
Browse files Browse the repository at this point in the history
  • Loading branch information
mohamedawnallah committed Dec 27, 2023
1 parent d4f40d2 commit 938df23
Show file tree
Hide file tree
Showing 2 changed files with 13 additions and 3 deletions.
13 changes: 13 additions & 0 deletions iyp/crawlers/ripe/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -47,6 +47,19 @@ ASN(s), and country.
(:IP {ip: '202.214.97.16'})-[:ASSIGNED]->(:AtlasProbe {id: 6425})
```

### Atlas Measurements - `atlas_measurements.py`

We fetch the [list of probe measurements](https://atlas.ripe.net/api/v2/measurements) to obtain the measurement data of the `AtlasProbe`. This data is based on the probe's ID, connected through relationships defined by `PART_OF` and `TARGET`. The `TARGET` relationship encompass associations with both `DOMAIN` and `IP`.

The Cypher query for these relationships appears as follows:

```Cypher
(:AtlasProbe)-[:PART_OF]->(:AtlasMeasurement)-[:TARGET]->(:DomainName)
(:AtlasProbe)-[:PART_OF]->(:AtlasMeasurement)-[:TARGET]->(:IP)
```

This query is designed to identify `AtlasProbes` linked via the `PART_OF` relationship to `AtlasMeasurements`, which, in turn, are linked through `TARGET` to either a `DomainName` or an `IP`.

## Dependence

This crawler is not depending on other crawlers.
3 changes: 0 additions & 3 deletions iyp/crawlers/ripe/atlas_measurements.py
Original file line number Diff line number Diff line change
Expand Up @@ -142,13 +142,10 @@ def run(self):
'page_size': 500}
r = self.session.get(URL, params=params)
next_url, data = self.__process_response(r)
count = 0
while next_url:
next_url, next_data = self.__execute_query(next_url)
data += next_data
logging.info(f'Added {len(next_data)} probes. Total: {len(data)}')
if count == 5:
break
print(f'Fetched {len(data)} probe measurements', file=sys.stderr)

# Transform the data to be compatible with the flatdict format.
Expand Down

0 comments on commit 938df23

Please sign in to comment.