Project dependencies may have API risk issues

[PyDeps]

Hi, In misp-warninglists, inappropriate dependency versioning constraints can cause risks.

Below are the dependencies and version constraints that the project is using

beautifulsoup4
pyOpenSSL==19.1.0
python-dateutil==2.8.1
requests
dnspython
pyasn1
pyasn1-modules

The version constraint == will introduce the risk of dependency conflicts because the scope of dependencies is too strict.
The version constraint No Upper Bound and * will introduce the risk of the missing API Error because the latest version of the dependencies may remove some APIs.

After further analysis, in this project,
The version constraint of dependency beautifulsoup4 can be changed to >=4.10.0,<=4.11.1.
The version constraint of dependency pyOpenSSL can be changed to >=0.14,<=22.0.0.
The version constraint of dependency python-dateutil can be changed to >=2.5.0,<=2.6.1.
The version constraint of dependency requests can be changed to >=0.2.1,<=0.2.3.
The version constraint of dependency requests can be changed to >=0.7.0,<=2.24.0.
The version constraint of dependency requests can be changed to ==2.26.0.
The version constraint of dependency pyasn1 can be changed to >=0.4.1,<=0.4.8.
The version constraint of dependency pyasn1-modules can be changed to >=0.0.3,<=0.2.8.

The above modification suggestions can reduce the dependency conflicts as much as possible,
and introduce the latest version as much as possible without calling Error in the projects.

The invocation of the current project includes all the following methods.

The calling methods from the beautifulsoup4

bs4.BeautifulSoup

The calling methods from the pyOpenSSL

OpenSSL.crypto.load_certificate

The calling methods from the python-dateutil

dateutil.parser.parse

The calling methods from the requests

requests.head
requests.get

The calling methods from the pyasn1

open

The calling methods from the pyasn1-modules

pyasn1_modules.rfc2459.AuthorityInfoAccessSyntax
pyasn1_modules.rfc2459.CRLDistPointsSyntax

The calling methods from the all methods

p.open
cisco_lists.open
requests.get
lipv6.append
file.get_abspath_source_file.path.getmtime.datetime.datetime.fromtimestamp.astimezone
a_tag.attrs.get
is_akamai
crl.getComponentByName
unique_sorted_warninglist
OpenSSL.crypto.load_certificate.digest
get_networks_for_asn
site.decode.split.strip
bs4.BeautifulSoup
logging.getLogger.addHandler
ipv6_networks.append
generator.download.raise_for_status
mx_ips.extend
lists.append
url.split
row.rstrip
logging.getLogger.setLevel
ip.strip
url.find
generate_irish_warninglist
tranco.readlines
map
get_domain
generate_swedish_warninglist
section.getComponentByName
tld.startswith
get_lists_dnscrypt
threads.append
lists.sort
answer.__iter__.next.to_text
url.split.split
output.append
set.append
os.path.exists
ipaddress.ip_network
urllib.parse.urlparse
mobile_numbers.items
list.split
validate_file
lips.append
part.startswith
bs4.BeautifulSoup.findAll
str
data_list.append
self.get_ip_for_domain
section.getComponentByName.getComponentByName
addresses.append
lhostname.append
isinstance
json.loads
logging.basicConfig
v.strip.replace
self.resolver.query
google_warninglist.append
bs4.BeautifulSoup.find
generator.download_to_file
urllib.request.urlopen
re.compile
site.split
dns.resolver.LRUCache
OpenSSL.crypto.load_certificate
next
line.decode
spf_ranges.extend
self.openresolvers.append
InvalidListValue
alexa_lists.open
address.split.split
os.listdir
argparse.ArgumentParser.parse_args
is_valid
generate_french_warninglist
bool
validate.extend
process
generator.Dns.get_ip_ranges_from_spf
json.loads.update
service.get
pyasn1_modules.rfc2459.AuthorityInfoAccessSyntax
validate
url.replace
obj.strip
re.findall
os.mkdir
sorted
self.get_ip_ranges_from_spf
inspect.stack
generate_uk_warninglist
netaddr.IPAddress
l.append
line.startswith
cert.get_extension.get_data
obj.strip.replace
asciidoc
sorted.append
domain.encode
ipaddress.IPv4Address
os.path.join
site.split.rstrip
logging.FileHandler.setFormatter
bs4.BeautifulSoup.find_all
f.readlines
func
csv.reader
part.split
alexa_lists.namelist
generator.download.split
domain.lstrip
actual_download_to_file
get_ips_from_domains
requests.head
gethash
int
answer.__iter__.next.to_text.strip
x.join
hostname.split
line.strip
generator.Dns.get_mx_ips_for_domain
csv.DictReader
r.text.splitlines
multiprocessing.dummy.Pool.map
tranco_lists.namelist
threading.Thread
self.get_mx_ips_for_domain
logging.FileHandler
get_crl_ocsp_domains
generator.create_resolver
time.sleep
get_abspath_source_file
asn_to_fetch.append
len
generator.consolidate_networks
logging.error
s.decode
lipv4.append
codecs.decode
self.__resolver.query
threading.Thread.start
list
header
os.path.abspath
v.rstrip.rstrip
self.errors.append
dns.resolver.query
cert.get_extension.get_short_name
urllib.parse.urljoin
argparse.ArgumentParser.add_argument
argparse.ArgumentParser
get_all_website_links
pyasn1.codec.der.decoder.decode
url.rsplit
datetime.datetime.fromtimestamp
logging.getLogger
requests.get.iter_lines
data.startswith
site.decode.split.rstrip
this.ips.pop
digest.cert.digest.decode
frame_records.getmodulename.upper
logging.warning
generator.get_abspath_source_file
datetime.datetime.utcnow
inspect.getmodulename
OpenSSL.crypto.load_certificate.get_extension_count
inspect.getframeinfo
generate_australian_warninglist.extend
list.extend
warninglist.append
generator.process_stream
json.load
logging.info
OpenResolverChecker.launch
format
generalName.getComponentByName
print
dns.resolver.Resolver
this.check
generator.download.json
line.decode.startswith
ipv4_networks.append
site.decode
csv_file.readlines
r.headers.parsedate.astimezone
row.findAll
json.dump
this.errors.append
re.compile.match
ranges.append
cisco_list.readlines
logging.Formatter
quit
university.get
range
sys.exit
get_abspath_list_file
top.readlines
generate_australian_warninglist.append
generate_australian_warninglist
url.replace.replace
urllib.request.urlopen.read
ValueError
OpenSSL.crypto.load_certificate.get_extension
url.download.json
init_logging
join
dateutil.parser.parse
ipaddress.ip_address
v.rstrip
open
OpenResolverChecker
obj.strip.replace.replace
is_valid_regexp
pathlib.Path.glob
ipaddress.IPv6Address
fd.write
spf.split
logging.exception
generator.create_resolver.query
row.findAll.find_all
cisco_lists.namelist
datetime.datetime.now
generator.download
generate
generator.get_version.append
set
tabl.findAll
url.replace.replace.replace
_f.write
os.path.realpath
inspect.currentframe
self._parse_spf
zipfile.ZipFile
set.add
search
response.read.decode
set.update
requests.get.iter_content
obj.lower
get_lists_publidns
datetime.date.today
multiprocessing.dummy.Pool
os.path.dirname
retry_link_text.find_all.get
generator.write_to_file
generate_american_warninglist
crl.getComponentByName.getComponentByName
tranco_lists.open
pathlib.Path
pyasn1_modules.rfc2459.CRLDistPointsSyntax
get_lists
soup.find.find_all
lurls.append
os.path.getmtime
all
main
domain.startswith
datetime.date.today.strftime
self.resolver.query.__iter__
get_json_url
list.append
digest.cert.digest.decode.replace.lower
ipaddress.collapse_addresses
generator.Dns
join.startswith
generator.get_version
digest.cert.digest.decode.replace
address.split.index
domain.encode.decode
ipaddress.IPv6Interface
data_file.write
site.decode.split

@developer
Could please help me check this issue?
May I pull a request to fix it?
Thank you very much.

[adulau]

Sure, if you want to make a pull-request. I'm just wondering about the No Upper Bound rule. IMHO, security wise I prefer to not have any upper bound to ensure that people are running the latest version which might include security fixes. API breaking versus security vulnerability in this project, it's clearly the security fixes which are more important are the generator scripts are only to generate the JSON available in this repository.