Basket models #216

julesdesir · 2025-01-15T17:24:39Z

Description of the goal of the PR

Description: add SHOPPER model and Trip / TripDataset data structures

Changes this PR introduces (fill it before implementation)

: Trip, TripDataset and SHOPPER
: Data preprocessing for basket models
: Remove unnecessary enumerate() in for loop
: Fix weight of the loss of the last batch in the loss of the whole epoch
: Tests for basket models
: SHOPPER in readme and in docs

Checklist before requesting a review

I have commented my code, particularly in hard-to-understand areas
I have typed my code
I have created / updated the docstrings
I have updated the README, if relevant
I have updated the requirements files if a new package is used
I have tested my code
The CI pipeline passes
I have performed a self-review of my code

github-actions · 2025-01-15T17:31:40Z

Coverage Report for Python 3.9

File	Stmts	Miss	Cover	Missing
choice_learn
__init__.py	2	0	100%
tf_ops.py	48	0	100%
choice_learn/basket_models
shopper.py	280	82	71%	72, 76, 89–110, 127, 227, 295, 446, 500, 533, 686–696, 747, 964, 982, 1021, 1035, 1042, 1050–1081, 1116–1146, 1156–1178, 1195–1244
trip_dataset.py	122	34	72%	52, 79, 121, 131, 148–161, 181, 225, 247, 257, 267, 297, 391, 405–423, 528–534
choice_learn/basket_models/utils
permutation.py	22	1	95%	35
choice_learn/data
__init__.py	3	0	100%
choice_dataset.py	647	33	95%	198, 250, 283, 421, 463–464, 589, 724, 738, 840, 842, 937, 957–961, 1140, 1159–1161, 1179–1181, 1209, 1214, 1223, 1240, 1281, 1293, 1307, 1346, 1361, 1366, 1395, 1408, 1443–1444
indexer.py	239	23	90%	20, 31, 45, 60–67, 202–204, 219–230, 265, 291, 577
storage.py	161	6	96%	22, 33, 51, 56, 61, 71
store.py	72	72	0%	3–275
choice_learn/datasets
__init__.py	4	0	100%
base.py	393	5	99%	43–44, 154–155, 715
expedia.py	102	83	19%	37–301
tafeng.py	49	0	100%
choice_learn/datasets/data
__init__.py	0	0	100%
choice_learn/models
__init__.py	14	2	86%	15–16
base_model.py	254	12	95%	144, 186, 283, 302, 342, 349, 378, 397, 428–429, 438–439
baseline_models.py	49	0	100%
conditional_logit.py	236	21	91%	46, 49, 51, 82, 85, 88–92, 95–99, 133, 298, 335, 392, 467–473, 598, 632, 739, 743
halo_mnl.py	124	2	98%	186, 374
latent_class_base_model.py	286	39	86%	55–61, 273–279, 288, 325–330, 497–500, 605, 624, 665–701, 715, 720, 751–752, 774–775, 869–870, 974
latent_class_mnl.py	62	6	90%	257–261, 296
learning_mnl.py	67	3	96%	157, 182, 188
nested_logit.py	291	12	96%	55, 77, 160, 269, 351, 484, 530, 600, 679, 848, 900, 904
reslogit.py	132	6	95%	285, 360, 369, 374, 382, 432
rumnet.py	236	3	99%	748–751, 982
simple_mnl.py	139	6	96%	167, 275, 347, 355, 357, 359
tastenet.py	94	3	97%	142, 180, 188
choice_learn/toolbox
__init__.py	0	0	100%
assortment_optimizer.py	27	6	78%	28–30, 93–95, 160–162
gurobi_opt.py	236	236	0%	3–675
or_tools_opt.py	230	11	95%	103, 107, 296–305, 315, 319, 607, 611
TOTAL	4621	707	85%

Tests	Skipped	Failures	Errors	Time
144	0 💤	0 ❌	0 🔥	5m 48s ⏱️

github-actions · 2025-01-15T17:31:48Z

Coverage Report for Python 3.10

File	Stmts	Miss	Cover	Missing
choice_learn
__init__.py	2	0	100%
tf_ops.py	48	0	100%
choice_learn/basket_models
shopper.py	280	82	71%	72, 76, 89–110, 127, 227, 295, 446, 500, 533, 686–696, 747, 964, 982, 1021, 1035, 1042, 1050–1081, 1116–1146, 1156–1178, 1195–1244
trip_dataset.py	122	34	72%	52, 79, 121, 131, 148–161, 181, 225, 247, 257, 267, 297, 391, 405–423, 528–534
choice_learn/basket_models/utils
permutation.py	22	1	95%	35
choice_learn/data
__init__.py	3	0	100%
choice_dataset.py	647	33	95%	198, 250, 283, 421, 463–464, 589, 724, 738, 840, 842, 937, 957–961, 1140, 1159–1161, 1179–1181, 1209, 1214, 1223, 1240, 1281, 1293, 1307, 1346, 1361, 1366, 1395, 1408, 1443–1444
indexer.py	239	23	90%	20, 31, 45, 60–67, 202–204, 219–230, 265, 291, 577
storage.py	161	6	96%	22, 33, 51, 56, 61, 71
store.py	72	72	0%	3–275
choice_learn/datasets
__init__.py	4	0	100%
base.py	393	4	99%	39, 154–155, 715
expedia.py	102	83	19%	37–301
tafeng.py	49	0	100%
choice_learn/datasets/data
__init__.py	0	0	100%
choice_learn/models
__init__.py	14	2	86%	15–16
base_model.py	254	12	95%	144, 186, 283, 302, 342, 349, 378, 397, 428–429, 438–439
baseline_models.py	49	0	100%
conditional_logit.py	236	21	91%	46, 49, 51, 82, 85, 88–92, 95–99, 133, 298, 335, 392, 467–473, 598, 632, 739, 743
halo_mnl.py	124	2	98%	186, 374
latent_class_base_model.py	286	39	86%	55–61, 273–279, 288, 325–330, 497–500, 605, 624, 665–701, 715, 720, 751–752, 774–775, 869–870, 974
latent_class_mnl.py	62	6	90%	257–261, 296
learning_mnl.py	67	3	96%	157, 182, 188
nested_logit.py	291	12	96%	55, 77, 160, 269, 351, 484, 530, 600, 679, 848, 900, 904
reslogit.py	132	6	95%	285, 360, 369, 374, 382, 432
rumnet.py	236	3	99%	748–751, 982
simple_mnl.py	139	6	96%	167, 275, 347, 355, 357, 359
tastenet.py	94	3	97%	142, 180, 188
choice_learn/toolbox
__init__.py	0	0	100%
assortment_optimizer.py	27	6	78%	28–30, 93–95, 160–162
gurobi_opt.py	238	238	0%	3–675
or_tools_opt.py	230	11	95%	103, 107, 296–305, 315, 319, 607, 611
TOTAL	4623	708	85%

Tests	Skipped	Failures	Errors	Time
144	0 💤	0 ❌	0 🔥	6m 11s ⏱️

github-actions · 2025-01-15T17:32:08Z

Coverage Report for Python 3.11

File	Stmts	Miss	Cover	Missing
choice_learn
__init__.py	2	0	100%
tf_ops.py	48	0	100%
choice_learn/basket_models
shopper.py	280	82	71%	72, 76, 89–110, 127, 227, 295, 446, 500, 533, 686–696, 747, 964, 982, 1021, 1035, 1042, 1050–1081, 1116–1146, 1156–1178, 1195–1244
trip_dataset.py	122	34	72%	52, 79, 121, 131, 148–161, 181, 225, 247, 257, 267, 297, 391, 405–423, 528–534
choice_learn/basket_models/utils
permutation.py	22	1	95%	35
choice_learn/data
__init__.py	3	0	100%
choice_dataset.py	647	33	95%	198, 250, 283, 421, 463–464, 589, 724, 738, 840, 842, 937, 957–961, 1140, 1159–1161, 1179–1181, 1209, 1214, 1223, 1240, 1281, 1293, 1307, 1346, 1361, 1366, 1395, 1408, 1443–1444
indexer.py	239	23	90%	20, 31, 45, 60–67, 202–204, 219–230, 265, 291, 577
storage.py	161	6	96%	22, 33, 51, 56, 61, 71
store.py	72	72	0%	3–275
choice_learn/datasets
__init__.py	4	0	100%
base.py	393	4	99%	39, 154–155, 715
expedia.py	102	83	19%	37–301
tafeng.py	49	0	100%
choice_learn/datasets/data
__init__.py	0	0	100%
choice_learn/models
__init__.py	14	2	86%	15–16
base_model.py	254	12	95%	144, 186, 283, 302, 342, 349, 378, 397, 428–429, 438–439
baseline_models.py	49	0	100%
conditional_logit.py	236	21	91%	46, 49, 51, 82, 85, 88–92, 95–99, 133, 298, 335, 392, 467–473, 598, 632, 739, 743
halo_mnl.py	124	2	98%	186, 374
latent_class_base_model.py	286	39	86%	55–61, 273–279, 288, 325–330, 497–500, 605, 624, 665–701, 715, 720, 751–752, 774–775, 869–870, 974
latent_class_mnl.py	62	6	90%	257–261, 296
learning_mnl.py	67	3	96%	157, 182, 188
nested_logit.py	291	12	96%	55, 77, 160, 269, 351, 484, 530, 600, 679, 848, 900, 904
reslogit.py	132	6	95%	285, 360, 369, 374, 382, 432
rumnet.py	236	3	99%	748–751, 982
simple_mnl.py	139	6	96%	167, 275, 347, 355, 357, 359
tastenet.py	94	3	97%	142, 180, 188
choice_learn/toolbox
__init__.py	0	0	100%
assortment_optimizer.py	27	6	78%	28–30, 93–95, 160–162
gurobi_opt.py	238	238	0%	3–675
or_tools_opt.py	230	11	95%	103, 107, 296–305, 315, 319, 607, 611
TOTAL	4623	708	85%

Tests	Skipped	Failures	Errors	Time
144	0 💤	0 ❌	0 🔥	6m 14s ⏱️

github-actions · 2025-01-15T17:33:07Z

Coverage Report for Python 3.12

File	Stmts	Miss	Cover	Missing
choice_learn
__init__.py	2	0	100%
tf_ops.py	48	0	100%
choice_learn/basket_models
shopper.py	280	82	71%	72, 76, 89–110, 127, 227, 295, 446, 500, 533, 686–696, 747, 964, 982, 1021, 1035, 1042, 1050–1081, 1116–1146, 1156–1178, 1195–1244
trip_dataset.py	122	34	72%	52, 79, 121, 131, 148–161, 181, 225, 247, 257, 267, 297, 391, 405–423, 528–534
choice_learn/basket_models/utils
permutation.py	22	1	95%	35
choice_learn/data
__init__.py	3	0	100%
choice_dataset.py	647	33	95%	198, 250, 283, 421, 463–464, 589, 724, 738, 840, 842, 937, 957–961, 1140, 1159–1161, 1179–1181, 1209, 1214, 1223, 1240, 1281, 1293, 1307, 1346, 1361, 1366, 1395, 1408, 1443–1444
indexer.py	239	23	90%	20, 31, 45, 60–67, 202–204, 219–230, 265, 291, 577
storage.py	161	6	96%	22, 33, 51, 56, 61, 71
store.py	72	72	0%	3–275
choice_learn/datasets
__init__.py	4	0	100%
base.py	393	4	99%	39, 154–155, 715
expedia.py	102	83	19%	37–301
tafeng.py	49	0	100%
choice_learn/datasets/data
__init__.py	0	0	100%
choice_learn/models
__init__.py	14	2	86%	15–16
base_model.py	254	12	95%	144, 186, 283, 302, 342, 349, 378, 397, 428–429, 438–439
baseline_models.py	49	0	100%
conditional_logit.py	236	21	91%	46, 49, 51, 82, 85, 88–92, 95–99, 133, 298, 335, 392, 467–473, 598, 632, 739, 743
halo_mnl.py	124	2	98%	186, 374
latent_class_base_model.py	286	39	86%	55–61, 273–279, 288, 325–330, 497–500, 605, 624, 665–701, 715, 720, 751–752, 774–775, 869–870, 974
latent_class_mnl.py	62	6	90%	257–261, 296
learning_mnl.py	67	3	96%	157, 182, 188
nested_logit.py	291	12	96%	55, 77, 160, 269, 351, 484, 530, 600, 679, 848, 900, 904
reslogit.py	132	6	95%	285, 360, 369, 374, 382, 432
rumnet.py	236	3	99%	748–751, 982
simple_mnl.py	139	6	96%	167, 275, 347, 355, 357, 359
tastenet.py	94	3	97%	142, 180, 188
choice_learn/toolbox
__init__.py	0	0	100%
assortment_optimizer.py	27	6	78%	28–30, 93–95, 160–162
gurobi_opt.py	238	238	0%	3–675
or_tools_opt.py	230	11	95%	103, 107, 296–305, 315, 319, 607, 611
TOTAL	4623	708	85%

Tests	Skipped	Failures	Errors	Time
144	0 💤	0 ❌	0 🔥	7m 39s ⏱️

VincentAuriau

First round of comments, to be discussed :)

choice_learn/basket_models/base_basket.py

choice_learn/basket_models/shopper.py

choice_learn/basket_models/trip_dataset.py

choice_learn/basket_models/utils/permutation.py

choice_learn/models/base_model.py

…t when computing item utilities

choice_learn/basket_models/shopper.py

VincentAuriau · 2025-01-21T18:23:44Z

choice_learn/basket_models/shopper.py

+            Shape must be (n_items,)
+        """
+        # Prevent unintended side effects from in-place modifications
+        availability_matrix_copy = availability_matrix.copy()


Je comprends que ce soit là, si on arrive à l'enlever c'est mieux =)

choice_learn/basket_models/shopper.py

choice_learn/basket_models/trip_dataset.py

VincentAuriau · 2025-01-21T21:33:36Z

choice_learn/basket_models/trip_dataset.py

+class TripDataset:
+    """Class for a dataset of trips."""
+
+    def __init__(self, trips: list[Trip], assortments: dict[int, np.ndarray]) -> None:


les assortments pourraient être des FeaturesStorage -> parlons en :)

choice_learn/basket_models/trip_dataset.py

choice_learn/basket_models/trip_preprocessing.py

VincentAuriau · 2025-01-21T22:08:51Z

choice_learn/basket_models/trip_preprocessing.py

+
+
+def from_csv(
+    train_file: str,


could it be a method of the TripDataset class ?

Then trip_preprocessing.py should be deleted?

There are only 3 functions in this file:

from_csv

csv_to_df and map_indexes (2 short functions which don't justify on their own the existence of trip_preprocesing.py and could be moved to the folder choice_learn/basket_models/utils if from_csv is moved to trip_dataset.py)

…/test in one shot

julesdesir added 10 commits January 15, 2025 12:03

ADD: Trip, TripDataset and Shopper

311a423

MAINT: fix typos in docstrings

7b6fc4b

ADD: data preprocessing for basket models

834757b

MAINT: fix typos in docstrings

1560348

ENH: remove unnecessary enumerate() in for loop

bf71ef5

MAINT: fix typos in docstrings and variable names

1909362

FIX: weight of the loss of the last batch in the loss of the whole epoch

2ca140c

TST: tests for basket models

235f7d7

DOC: add SHOPPER in readme and in docs

f7e06de

DOC: fix typos in readme

aa000ab

julesdesir requested a review from VincentAuriau as a code owner January 15, 2025 17:24

julesdesir self-assigned this Jan 15, 2025

julesdesir added the new feature Additional feature to be added label Jan 15, 2025

DOC: add SHOPPER references in documentation index

23c426a

VincentAuriau reviewed Jan 16, 2025

View reviewed changes

VincentAuriau linked an issue Jan 16, 2025 that may be closed by this pull request

ADD: SHOPPER model #217

Open

julesdesir added 10 commits January 17, 2025 10:44

FIX: path to data for basket models

dc6347b

ENH: first part of changes following review

0581efe

ENH: use 5 variables for batch instead of only 1

7d75651

ENH: second part of changes following review

1613925

ENH: lazy instantiation in .fit + some attributes moved to __init__

7171cbe

TST: update Shopper tests with the new attributes in __init__

380333b

ENH: add assortment attribute to Trip and take into account assortmen…

e0b266a

…t when computing item utilities

TST: update Shopper tests with assortment handling

92a07e4

FIX: handling assortments when stage=3

0dd49dc

MAINT: rename base_basket.py

76281d9

MAINT: clean useless comment lines

bd41337

VincentAuriau reviewed Jan 21, 2025

View reviewed changes

choice_learn/basket_models/shopper.py Outdated Show resolved Hide resolved

choice_learn/basket_models/shopper.py Outdated Show resolved Hide resolved

choice_learn/basket_models/shopper.py Show resolved Hide resolved

choice_learn/basket_models/shopper.py Outdated Show resolved Hide resolved

VincentAuriau reviewed Jan 21, 2025

View reviewed changes

julesdesir added 9 commits January 22, 2025 11:12

ENH: first part of changes following 2nd review round

354e925

ENH: second part of changes following 2nd review round

9fc78f0

ADD: __getitem__ method for the TripDataset class

094c133

DEP: np.in1d deprecated since NumPy 2.0, use np.isin instead

8743f65

MAINT: fix typos

25aee29

ENH: move thinking ahead to a dedicated function

1dd39b1

ENH: remove unnecessary case distinction when computing item_utilities

22aa356

FIX: load_model without losing hyperparameter values

45982a2

ENH: a call to from_csv builds only one TripDataset and not train/val…

7a5b4fb

…/test in one shot

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Basket models #216

Basket models #216

julesdesir commented Jan 15, 2025 •

edited

Loading

github-actions bot commented Jan 15, 2025 •

edited

Loading

github-actions bot commented Jan 15, 2025 •

edited

Loading

github-actions bot commented Jan 15, 2025 •

edited

Loading

github-actions bot commented Jan 15, 2025 •

edited

Loading

VincentAuriau left a comment

VincentAuriau Jan 21, 2025

VincentAuriau Jan 21, 2025

VincentAuriau Jan 21, 2025

julesdesir Jan 22, 2025 •

edited

Loading

Basket models #216

Are you sure you want to change the base?

Basket models #216

Conversation

julesdesir commented Jan 15, 2025 • edited Loading

Description of the goal of the PR

Changes this PR introduces (fill it before implementation)

Checklist before requesting a review

github-actions bot commented Jan 15, 2025 • edited Loading

github-actions bot commented Jan 15, 2025 • edited Loading

github-actions bot commented Jan 15, 2025 • edited Loading

github-actions bot commented Jan 15, 2025 • edited Loading

VincentAuriau left a comment

Choose a reason for hiding this comment

VincentAuriau Jan 21, 2025

Choose a reason for hiding this comment

VincentAuriau Jan 21, 2025

Choose a reason for hiding this comment

VincentAuriau Jan 21, 2025

Choose a reason for hiding this comment

julesdesir Jan 22, 2025 • edited Loading

Choose a reason for hiding this comment

julesdesir commented Jan 15, 2025 •

edited

Loading

github-actions bot commented Jan 15, 2025 •

edited

Loading

github-actions bot commented Jan 15, 2025 •

edited

Loading

github-actions bot commented Jan 15, 2025 •

edited

Loading

github-actions bot commented Jan 15, 2025 •

edited

Loading

julesdesir Jan 22, 2025 •

edited

Loading