dynamic optimisation sensitivity to start_date

[js190]

Hello,
When I run the DO over various start dates I see a variation in the optimal positions for a given date, even when the delta between dates is small e.g I see significant differences between 2000 and 2005 - this was not what I expected as I presumed the difference in 'warmup' data would be washed out and the positions would converge. These are some example portfolios (i've grouped my Experiment objects in list hence the index):

[nav] In [22]: index = -4
          ...: print(c[index].config().start_date)
          ...: c[index].positions(10)
2015-1-19
Out[22]: 
            BBCOMM  BITCOIN  CORN  DOW  EUROSTX  GASOILINE  GOLD_micro  IRON  RUSSELL  SP500_micro  V2X  VIX  VNKI
2022-01-19       2       -1     1    1      1.0          1         0.0     1      0.0          1.0   -1   -1    -2
2022-01-20       2       -1     1    1      1.0          1         0.0     1      0.0          1.0   -1   -1    -2
2022-01-21       2       -1     1    1      0.0          1         1.0     1     -1.0          0.0   -1   -1    -2
2022-01-24       2       -1     1    1      0.0          1         1.0     1     -1.0          0.0   -1   -1    -2
2022-01-25       2       -1     1    1      0.0          1         1.0     1     -1.0          0.0   -1   -1    -2
2022-01-26       2       -1     1    1      0.0          1         1.0     1     -1.0          0.0   -1   -1    -2
2022-01-27       2       -1     1    1      0.0          1         1.0     1     -1.0          0.0   -1   -1    -2
2022-01-28       2       -1     1    1      0.0          1         1.0     1     -1.0          0.0   -1   -1    -2
2022-01-31       2       -1     1    1      0.0          1         1.0     1     -1.0          0.0   -1   -1    -2
2022-02-01       2       -1     1    1      0.0          1         1.0     1     -1.0          0.0   -1   -1    -2

[nav] In [23]: index = -2
          ...: print(c[index].config().start_date)
          ...: c[index].positions(10
          ...: 
          ...: 
          ...: )
2010-1-19
Out[23]: 
            BBCOMM  BITCOIN  CORN  DOW  EDOLLAR  EUROSTX  GOLD_micro  HEATOIL  IRON  SP500_micro  V2X  VIX  VNKI
2022-01-19       3       -1     1    2        1      1.0         1.0        1     1          1.0   -2   -1    -3
2022-01-20       3       -1     1    2        1      1.0         1.0        1     1          1.0   -2   -1    -3
2022-01-21       3       -1     1    2        1      1.0         1.0        1     1          1.0   -2   -1    -3
2022-01-24       3       -1     1    2        1      1.0         1.0        1     1          1.0   -2   -1    -3
2022-01-25       3       -1     1    2        1      1.0         1.0        1     1          1.0   -2   -1    -3
2022-01-26       3       -1     1    2        1      1.0         1.0        1     1          1.0   -2   -1    -3
2022-01-27       4       -1     1    1        1      0.0         0.0        1     1          0.0   -2   -1    -3
2022-01-28       4       -1     1    1        1      0.0         0.0        1     1          0.0   -2   -1    -3
2022-01-31       4       -1     1    1        1      0.0         0.0        1     1          0.0   -2   -1    -3
2022-02-01       4       -1     1    1        1      0.0         0.0        1     1          0.0   -2   -1    -3

[nav] In [24]: index = -1
          ...: print(c[index].config().start_date)
          ...: c[index].positions(10
          ...: 
          ...: 
          ...: )
2005-1-19
Out[24]: 
            BBCOMM  BITCOIN  CORN  CRUDE_W_mini  DAX  DOW  EDOLLAR  EUROSTX  GOLD_micro  IRON  KOSDAQ  MXP  RUSSELL  SP500_micro  V2X  VIX  VNKI
2022-01-19       4      0.0     1             1  1.0    3        1        1         0.0     1     0.0    1      1.0          2.0   -1   -1    -1
2022-01-20       4      0.0     1             1  1.0    3        1        1         0.0     1     0.0    1      0.0          2.0   -1   -1    -1
2022-01-21       5      0.0     1             1  1.0    3        1        1         0.0     1     0.0    1      0.0          1.0   -1   -1    -1
2022-01-24       5      0.0     1             1  1.0    3        1        1         0.0     1     0.0    1      0.0          1.0   -1   -1    -1
2022-01-25       5      0.0     1             1  1.0    2        1        1         0.0     1     0.0    1      0.0          1.0   -1   -1    -1
2022-01-26       5      0.0     1             1  1.0    1        1        1         0.0     1     0.0    1      0.0          1.0   -1   -1    -1
2022-01-27       5      0.0     1             1  1.0    1        1        1         0.0     1     0.0    1      0.0          1.0   -1   -1    -1
2022-01-28       5      0.0     1             1  1.0    1        1        1         0.0     1     0.0    1      0.0          1.0   -1   -1    -1
2022-01-31       6      0.0     1             1  1.0    1        1        1         0.0     1     0.0    1      0.0          1.0   -1   -1    -1
2022-02-01       5     -1.0     1             1  0.0    1        1        1        -1.0     2    -1.0    1      0.0          0.0   -1   -1    -2

[nav] In [25]: index = -3
          ...: print(c[index].config().start_date)
          ...: c[index].positions(10
          ...: 
          ...: 
          ...: )
2000-1-19
Out[25]: 
            BBCOMM  BITCOIN  CAD  CORN  CRUDE_W_mini  DAX  DOW  EDOLLAR  EUROSTX  IRON  MXP  NIKKEI  SP500_micro  V2X  VIX  VNKI
2022-01-19       4      0.0    1     1             1  1.0    3        1        1     1  0.0     1.0          1.0   -1   -1    -2
2022-01-20       4      0.0    1     1             1  1.0    3        1        1     1  0.0     1.0          1.0   -1   -1    -2
2022-01-21       5      0.0    1     1             1  1.0    3        1        1     1  0.0     1.0          1.0   -1   -1    -2
2022-01-24       5      0.0    1     1             1  1.0    3        1        1     1  0.0     1.0          1.0   -1   -1    -2
2022-01-25       6     -1.0    1     1             1  0.0    1        1        1     1  1.0     0.0          0.0   -1   -1    -2
2022-01-26       6     -1.0    1     1             1  0.0    1        1        1     1  1.0     0.0          0.0   -1   -1    -2
2022-01-27       6     -1.0    1     1             1  0.0    1        1        1     1  1.0     0.0          0.0   -1   -1    -2
2022-01-28       5     -1.0    1     1             1  0.0    1        1        1     1  1.0     0.0          0.0   -1   -1    -2
2022-01-31       6     -1.0    1     1             1  0.0    1        1        1     1  1.0     0.0          0.0   -1   -1    -2
2022-02-01       5     -1.0    1     1             1  0.0    1        1        1     1  1.0     0.0          0.0   -1   -1    -2

[nav] In [26]: 

Curious, right?
Should I just pick my preferred portfolio and hence backtest start date? (joking)

[robcarver17]

What's the correlation of account curves between these different experiments; eg are the differences meaningful?

You could also reduce costs to zero to satisfy yourself that if you do that the differences will probably vanish.

[js190]

Sorry to labour this point, but given that the instrument set is the same across each experiment (I only show above the instruments with a recent position e.g. positions.tail().dropna(how='all', axis=1), how surprised are you about the above? Would you expect the optimisation to pick the same instruments given enough history?

[robcarver17]

Not very surprised, but like I said before, if you change the costs to zero you should get an exact match - would that put your mind at rest?

[js190]

Shadow costs?

[robcarver17]

yes

[js190]

I convinced myself, lovely stuff, thanks.