Add Alice-LG crawler

[m-appel]

This crawler imports memership and optionally routing information from IXPs that use the Alice-LG looking glass.

Description

The Alice-LG looking glass is used by some of the largest IXPs and provides a good opportunity to get additional membership information via a single crawler.

Motivation and Context

Currently we only get IXP membership information from PeeringDB. However, this data is usually not updated automatically and might be outdated. Getting the membership information directly from the looking glass, i.e., the route servers at the IXP, provides a data-driven view of the IXP members.

This approach might miss IXP members that do not peer with the route server.

Originally we planned to get routing information from the members as well. However, testing has shown that this takes too long for most IXPs due to the small pagination size of the API. Furthermore, we already import route collector data from PCH, which is already present at the largest IXPs.

Therefore, the functionality is included, but disabled by default.

Closes #41.

How Has This Been Tested?

Ran each crawler on its own and ran a full test including routing data for BCIX.

Types of changes

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to change)

Checklist:

• My code follows the code style of this project.
• My change requires a change to the documentation.
• I have updated the documentation accordingly.

[romain-fontugne]

Small detail, I like to keep the cloudflare crawlers at the end of the list of crawlers in the config file. These are by far the slowest crawlers we have and running them at the end has two advantages:

1. detect early if one of the other crawler crashes
2. the other crawlers can fetch data within the same few hours, and if we do cloudflare in the middle the last crawlers will fetch data on the following day.

[romain-fontugne]

thanks!!