Using generator to get all results from a search

[tomerten]

First of all cool code!

I have been experimenting a bit and added a generator such that you can get all results from a search request(a lazy version of your _read_chunked). Would you be interested in adding this ?

[joqueka]

Thanks!

I appreciate every idea and contribution. For the search requests I already started working on implementing this for derivatives (see the other discussion) but didn't commited/pushed it yet. I totally forgot about it, but it's almost finished.

You're welcome to have a look at these and contribute your code if it adds more functionality.

[joqueka]

Hi,
new function for derivatives search is available now. Just let me know what you think about it.

[tomerten]

Hi,

I was more thinking of something like this:

def equity_search(limit: int = 25, search_params:dict = None):
    
    params = {'limit': limit, 'offset': 0}
    if search_params:
        params.update(search_params)
    
    #data = _read_chunked(_search_request, 'equity_search', params)
    data = _search_request('equity_search', params)
    print(f"Found {data['recordsTotal']} records.")
    yield data['data']
    
    for offset in range(limit,data['recordsTotal'],limit):
        params['offset'] = offset
        yield _search_request('equity_search', params)['data']

You can then use next(...) to load the next chunk (this gives the user control over how much of the data he wants), but your solution also works of course.