Skip to content

Commit

Permalink
Update get_all Method in db.py to Retrieve All Items from DynamoDB …
Browse files Browse the repository at this point in the history
…Table (#642)

This update modifies the `get_all` method to ensure that all items from the DynamoDB table are retrieved during a scan operation.


**Reason for Change:** In Lab 4, `unranked_product_ids` are pulled from the `featured` items table, subsequently used to obtain a ranked list of products from "all" products via AWS Personalize. The existing `get_all` method's use of DynamoDB's scan operation only retrieves a limited subset of items by default, thus the ranked list may contain fewer items than the unranked list depending what's available in the fraction of the products table returned by the scan. This makes side-by-side comparisons of these lists difficult. 

This modification will ensure that all items are fetched through a paginated retrieval of data.

**Updated `get_all` function:
```
def get_all(self):
    items = []
    response = self.table.scan()
    while 'LastEvaluatedKey' in response:
        items.extend(response['Items'])
        response = self.table.scan(ExclusiveStartKey=response['LastEvaluatedKey'])
    items.extend(response['Items'])
    return items
```

Thus every item in the DynamoDB table is fetched before any operations that require a complete dataset, such as generating personalized rankings.
  • Loading branch information
MustaphaU authored Oct 1, 2024
1 parent d33ea5b commit 1e4febc
Showing 1 changed file with 7 additions and 2 deletions.
9 changes: 7 additions & 2 deletions src/products/src/products_service/db.py
Original file line number Diff line number Diff line change
Expand Up @@ -112,8 +112,13 @@ def gets(self, ids: list):
return response['Responses'][self.table.name]

def get_all(self):
items = []
response = self.table.scan()
return response['Items'] if 'Items' in response else []
while 'LastEvaluatedKey' in response:
items.extend(response['Items'])
response = self.table.scan(ExclusiveStartKey=response['LastEvaluatedKey'])
items.extend(response['Items'])
return items

def upsert(self, item):
self.table.put_item(Item=item)
Expand Down Expand Up @@ -256,4 +261,4 @@ def create_table(self):
key_schema=[
{"AttributeName": "id", "KeyType": "HASH"},
]
)
)

0 comments on commit 1e4febc

Please sign in to comment.