Reasonable Sharp Ratio?

[emretezel]

Hi Rob,

I managed to run rob_system with 74 instruments over 20 years, getting a sharp ratio of 0.7 and annualised return of 7%. Do you think that is on the low end? My naive expectation was around 1.0 sharp. What should be the next logical step to improve sharp without overfitting, eliminate certain costly instruments? Drop certain rules?

At the moment I am running with the following instruments, any obvious bad instruments to avoid?

  AEX: 0.006287244647997258
  AUD: 0.006580743696722926
  BITCOIN: 0.002820857520245068
  BOBL: 0.029929988690138354
  BONO: 0.011985149130296488
  BRE: 0.007837107386953004
  BRENT: 0.010790270281686203
  BTP: 0.007312795317612531
  BUND: 0.029929988690138354
  BUTTER: 0.01801485306380118
  CAC: 0.023943454825293132
  CAD: 0.012269069606057975
  CANOLA: 0.002290624442453564
  CHEESE: 0.010134239245399345
  CHF: 0.004181786281764522
  COCOA: 0.015539879947280719
  COFFEE: 0.0023613774473263964
  COPPER: 0.0229689245479093
  CORN: 0.0011035065667126516
  COTTON2: 0.003701496275705926
  CRUDE_W: 0.010790270281686203
  DAX: 0.04478697401397009
  DOW: 0.01916880411155967
  DX: 0.01192429538213123
  EDOLLAR: 0.025266868023751044
  EUR: 0.005726325974210044
  EUROSTX: 0.006287240837368174
  FEEDCOW: 0.003436085713095647
  FTSE100: 0.021374626148091217
  GASOILINE: 0.036160239216757845
  GAS_US: 0.002088543186205327
  GBP: 0.010325508332858327
  GOLD: 0.010209255820562603
  HEATOIL: 0.020259017158913435
  IBXEX: 0.011844498871951426
  INR: 0.004349996453035082
  IRON: 0.0019497176137097403
  JGB: 0.06104654385562298
  JPY: 0.00402679694457029
  KOSPI: 0.023100850216745916
  KR10: 0.0052701235858995
  KR3: 0.005270123507420452
  LEANHOG: 0.005618682737594854
  LIVECOW: 0.003436085713095647
  LUMBER: 0.002052889090608102
  MILK: 0.010134239245929242
  MXP: 0.007837107386953002
  NASDAQ: 0.037152915474598744
  NIKKEI: 0.044797238693725414
  NOK: 0.011266358388577046
  NZD: 0.006580743696722926
  OAT: 0.007312795170520792
  OATIES: 0.002290624423784589
  OJ: 0.0014613151340988118
  PALLAD: 0.010550986354601105
  PLAT: 0.00917968517420093
  RICE: 0.001009530584012275
  RUBBER: 0.0024675175581746465
  RUR: 0.013091289495124444
  RUSSELL: 0.0685916226607304
  SEK: 0.005726325974210159
  SILVER: 0.010209255820562603
  SMI: 0.021374647080715722
  SOYBEAN: 0.021385169644295002
  SOYMEAL: 0.021385169644295002
  SOYOIL: 0.0016030058743936547
  SP500: 0.01916880411155967
  STEEL: 0.0010095306112844797
  SUGAR11: 0.00370149627528071
  US10: 0.025266868023751044
  US2: 0.048588541262985625
  WHEAT: 0.0011034958320014685

[robcarver17]

Is that the DO performance? How much capital do you have? What is the performance without DO? What parameters did you use to fit your forecast weights? Have you dropped instruments with more than 0.01 SR in costs as per my advice?Why the last 20 years? What is it like if you go further back?

[emretezel]

It is the DO performance.

Starting capital was 350,000 GBP.

I haven't run the non-DO performance yet, will do.

For parameters fitting forecast weights and instrument weights, I have not overridden anything in the defaults.

I have kept few instruments above 0.01 SR cost, thinking inclusion would be better, in case one of them gets a massive move. For example I kept, COTTON2, BUTTER, MILK, LUMBER, OATIES, CHEESE, RICE, KOSPI, KRWUSD.

I only have 20 years of historical data.

Cheers,
Emre

[robcarver17]

What is the performance without DO? What parameters did you use to fit your forecast weights?

[emretezel]

So I ran the base system without DO, and I get a sharp ratio of 0.8 (vs. 0.7 DO). I used the following code, same instrument set and rules. I had to use myFuturesRawData to make the classic backtest to work.

from systems.provided.futures_chapter15.basesystem import System
from sysdata.sim.db_futures_sim_data import dbFuturesSimData
from sysdata.config.configdata import Config
from syscore.objects import arg_not_supplied

from systems.forecasting import Rules
from systems.basesystem import System
from systems.forecast_combine import ForecastCombine
from systems.forecast_scale_cap import ForecastScaleCap
from systems.provided.rob_system.rawdata import myFuturesRawData
from systems.positionsizing import PositionSizing
from systems.portfolio import Portfolios
from systems.accounts.accounts_stage import Account

data = dbFuturesSimData()
config = Config("private.system.config_no_do.yaml")
system = System([Account(),
                 Portfolios(),
                 PositionSizing(),
                 myFuturesRawData(),
                 ForecastCombine(),
                 ForecastScaleCap(),
                 Rules(arg_not_supplied)],
                data,
                config)

portfolio = system.accounts.portfolio()
portfolio.percent.stats()

Parameters to fit forecast weights are the following for both

forecast_weight_estimate:
  ann_target_SR: 0.5
  apply_cost_weight: false
  ceiling_cost_SR: 9999
  cleaning: true
  correlation_estimate:
    cleaning: false
    ew_lookback: 50000
    floor_at_zero: false
    func: sysquant.estimators.correlation_estimator.correlationEstimator
    min_periods: 10
    using_exponent: true
  cost_multiplier: 2.0
  date_method: expanding
  equalise_SR: false
  equalise_gross: false
  equalise_vols: true
  frequency: W
  func: sysquant.optimisation.generic_optimiser.genericOptimiser
  mean_estimate:
    ew_lookback: 50000
    func: sysquant.estimators.mean_estimator.meanEstimator
    min_periods: 5
    using_exponent: true
  method: handcraft
  pool_gross_returns: true
  rollyears: 20
  shrinkage_SR: 0.9
  shrinkage_corr: 0.5
  vol_estimate:
    ew_lookback: 50000
    func: sysquant.estimators.stdev_estimator.stdevEstimator
    min_periods: 5
    using_exponent: true

the full config file is below.

config_full.txt

Are you able to share a full config file which you would use to estimate parameters under DO?

Best,
Emre

[emretezel]

Btw, I was just being informed in another thread that I should be using system.accounts.optimised_portfolio() for P&L (assume for sharp as well) when using DO, I will refresh SR based on this method.

[emretezel]

Using system.accounts.optimised_portfolio() produced even lower SR.

system.accounts.optimised_portfolio().percent.stats()

[[('min', '-7.098'),
  ('max', '4.936'),
  ('median', '0.04612'),
  ('mean', '0.02831'),
  ('std', '0.7667'),
  ('skew', '-0.6561'),
  ('ann_mean', '7.236'),
  ('ann_std', '12.27'),
  ('sharpe', '0.5899'),
  ('sortino', '0.7366'),
  ('avg_drawdown', '-7.913'),
  ('time_in_drawdown', '0.9455'),
  ('calmar', '0.2904'),
  ('avg_return_to_drawdown', '0.9145'),
  ('avg_loss', '-0.5495'),
  ('avg_gain', '0.5193'),
  ('gaintolossratio', '0.945'),
  ('profitfactor', '1.112'),
  ('hitrate', '0.5407'),
  ('t_stat', '2.821'),
  ('p_value', '0.004796')],
 ('You can also plot / print:',
  ['rolling_ann_std', 'drawdown', 'curve', 'percent'])]

[robcarver17]

I just ran my system for the last 20 years only to get a baseline.

system.accounts.optimised_portfolio().percent.stats()
[[('min', '-12.23'), ('max', '8.433'), ('median', '0.1044'), ('mean', '0.08631'), ('std', '1.258'), ('skew', '-0.5708'), ('ann_mean', '22.06'), ('ann_std', '20.13'), ('sharpe', '1.096'), ('sortino', '1.408'), ('avg_drawdown', '-8.304'), ('time_in_drawdown', '0.9103'), ('calmar', '0.5724'), ('avg_return_to_drawdown', '2.657'), ('avg_loss', '-0.8824'), ('avg_gain', '0.8852'), ('gaintolossratio', '1.003'), ('profitfactor', '1.217'), ('hitrate', '0.5481'), ('t_stat', '4.993'), ('p_value', '6.143e-07')], 

system.accounts.optimised_portfolio().costs.percent.ann_mean
-1.023

What do you get when you try the same thing?

[emretezel]

[[('min', '-7.098'),
  ('max', '4.936'),
  ('median', '0.04612'),
  ('mean', '0.02831'),
  ('std', '0.7667'),
  ('skew', '-0.6561'),
  ('ann_mean', '7.236'),
  ('ann_std', '12.27'),
  ('sharpe', '0.5899'),
  ('sortino', '0.7366'),
  ('avg_drawdown', '-7.913'),
  ('time_in_drawdown', '0.9455'),
  ('calmar', '0.2904'),
  ('avg_return_to_drawdown', '0.9145'),
  ('avg_loss', '-0.5495'),
  ('avg_gain', '0.5193'),
  ('gaintolossratio', '0.945'),
  ('profitfactor', '1.112'),
  ('hitrate', '0.5407'),
  ('t_stat', '2.821'),
  ('p_value', '0.004796')],
 ('You can also plot / print:',
  ['rolling_ann_std', 'drawdown', 'curve', 'percent'])]

system.accounts.optimised_portfolio().costs.percent.ann_mean

<bound method accountCurve.ann_mean of 2000-01-13         NaN
2000-01-14    0.000000
2000-01-17         NaN
2000-01-18         NaN
2000-01-19   -0.002253
                ...   
2022-12-06         NaN
2022-12-07         NaN
2022-12-08         NaN
2022-12-09         NaN
2022-12-12    0.000000
Freq: B, Length: 5978, dtype: float64
 Weighted account curve; use object.stats() to see methods>

Something going wrong.

PS: I am using 15% vol target this probably should not affect SR, and instrument selection as per first post.

Thank you,
Emre

[robcarver17]

My bad should be system.accounts.optimised_portfolio().costs.percent.ann_mean()

[emretezel]

system.accounts.optimised_portfolio().costs.percent.ann_mean()
-0.7968543031057249

[robcarver17]

What if you use more capital, say $500K and set the risk target to 25%?

[emretezel]

Little improvement

[[('min', '-6.539'),
  ('max', '5.573'),
  ('median', '0.04921'),
  ('mean', '0.03114'),
  ('std', '0.7729'),
  ('skew', '-0.6506'),
  ('ann_mean', '7.961'),
  ('ann_std', '12.37'),
  ('sharpe', '0.6438'),
  ('sortino', '0.8036'),
  ('avg_drawdown', '-6.169'),
  ('time_in_drawdown', '0.9401'),
  ('calmar', '0.336'),
  ('avg_return_to_drawdown', '1.29'),
  ('avg_loss', '-0.5541'),
  ('avg_gain', '0.5237'),
  ('gaintolossratio', '0.945'),
  ('profitfactor', '1.123'),
  ('hitrate', '0.5431'),
  ('t_stat', '3.079'),
  ('p_value', '0.002087')],
 ('You can also plot / print:',
  ['rolling_ann_std', 'drawdown', 'curve', 'percent'])]
In [5]:

system.accounts.optimised_portfolio().costs.percent.ann_mean()
Out[5]:
-0.8347338428110839

[robcarver17]

Turn off all the optimisation so forecast and instrument weights are equal, and see what that looks like. Before DO, after DO, costs.

[emretezel]

what would be the easiest way to do this in the config?

[robcarver17]

RTFM

[emretezel]

All optimisations turned off, forecast and inst weights equal, diversification multipliers are 1.0

huge drop in SR from non-DO to DO.

non_od_portfolio = system.accounts.portfolio()
non_od_portfolio.percent.stats()

[[('min', '-0.5684'),
  ('max', '0.3485'),
  ('median', '0'),
  ('mean', '0.001407'),
  ('std', '0.03997'),
  ('skew', '-1.197'),
  ('ann_mean', '0.3598'),
  ('ann_std', '0.6396'),
  ('sharpe', '0.5626'),
  ('sortino', '0.5761'),
  ('avg_drawdown', '-0.6779'),
  ('time_in_drawdown', '0.97'),
  ('calmar', '0.2173'),
  ('avg_return_to_drawdown', '0.5308'),
  ('avg_loss', '-0.02638'),
  ('avg_gain', '0.02751'),
  ('gaintolossratio', '1.043'),
  ('profitfactor', '1.139'),
  ('hitrate', '0.5221'),
  ('t_stat', '2.691'),
  ('p_value', '0.007155')],
 ('You can also plot / print:',
  ['rolling_ann_std', 'drawdown', 'curve', 'percent'])]

non_od_portfolio.costs.percent.ann_mean()
-0.059366512629817965

od_portfolio = system.accounts.optimised_portfolio()
od_portfolio.percent.stats()

[[('min', '-0.7723'),
  ('max', '0.9084'),
  ('median', '0'),
  ('mean', '0.0006401'),
  ('std', '0.06282'),
  ('skew', '-0.5741'),
  ('ann_mean', '0.1636'),
  ('ann_std', '1.005'),
  ('sharpe', '0.1628'),
  ('sortino', '0.171'),
  ('avg_drawdown', '-2.157'),
  ('time_in_drawdown', '0.9246'),
  ('calmar', '0.03386'),
  ('avg_return_to_drawdown', '0.07585'),
  ('avg_loss', '-0.03587'),
  ('avg_gain', '0.03645'),
  ('gaintolossratio', '1.016'),
  ('profitfactor', '1.042'),
  ('hitrate', '0.5063'),
  ('t_stat', '0.7786'),
  ('p_value', '0.4362')],
 ('You can also plot / print:',
  ['rolling_ann_std', 'drawdown', 'curve', 'percent'])]

od_portfolio.costs.percent.ann_mean()
-0.015306573127868501

[emretezel]

When running my backtest, I am setting

capital_multiplier:
   func: syscore.capital.full_compounding

Will this have an impact on the sharp calculation in a negative way?

[emretezel]

Hi Rob,

Just wanted to check whether you had any further thoughts on why using dynamic optimisation I get a much lower sharp ratio. Do you think the system could be over trading?

I have synced the code to version 1.4 commit also on my back test I have removed any instruments whose sharp cost is higher than 0.01. I got some improvement but nowhere close to 1.

I am now using the following 54 instruments in the universe with 350,000 GBP starting capital over 20 years.

AEX, AUD, BITCOIN, BOBL, BONO, BRE, BRENT, BTP, BUND, CAC, CAD, CHF, COFFEE, COPPER, CORN, CRUDE_W, DAX, DOW, EDOLLAR, EUR, EUROSTX, FEEDCOW, GASOILINE, GAS_US, GBP, GOLD, HEATOIL, INR, IRON, JGB, JPY, KOSPI, KR10, KR3, LEANHOG, LIVECOW, MXP, NASDAQ, NIKKEI, NZD, OAT, PALLAD, PLAT, RUBBER, RUSSELL, SILVER, SMI, SOYBEAN, SOYMEAL, SOYOIL, SP500, US10, US2, WHEAT

Are you able to share what would have been your full config file for estimating parameters? Any advise on how to pinpoint the problem (could it a single still very expensive instrument?) would be greatly appreciated.

DO stats
[[('min', '-7.295'),
  ('max', '4.743'),
  ('median', '0.05853'),
  ('mean', '0.04118'),
  ('std', '0.8334'),
  ('skew', '-0.665'),
  ('ann_mean', '10.53'),
  ('ann_std', '13.33'),
  ('sharpe', '0.7895'),
  ('sortino', '0.9877'),
  ('avg_drawdown', '-5.627'),
  ('time_in_drawdown', '0.9271'),
  ('calmar', '0.5452'),
  ('avg_return_to_drawdown', '1.871'),
  ('avg_loss', '-0.5996'),
  ('avg_gain', '0.568'),
  ('gaintolossratio', '0.9474'),
  ('profitfactor', '1.152'),
  ('hitrate', '0.5488'),
  ('t_stat', '3.781'),
  ('p_value', '0.0001578')],
 ('You can also plot / print:',
  ['rolling_ann_std', 'drawdown', 'curve', 'percent'])]

do_portfolio.costs.percent.ann_mean()
-0.742986086025463

Non-DO Stats
[[('min', '-6.703'),
  ('max', '4.726'),
  ('median', '0.04851'),
  ('mean', '0.03973'),
  ('std', '0.6883'),
  ('skew', '-0.7383'),
  ('ann_mean', '10.16'),
  ('ann_std', '11.01'),
  ('sharpe', '0.9226'),
  ('sortino', '1.153'),
  ('avg_drawdown', '-3.941'),
  ('time_in_drawdown', '0.9184'),
  ('calmar', '0.6561'),
  ('avg_return_to_drawdown', '2.578'),
  ('avg_loss', '-0.4873'),
  ('avg_gain', '0.4767'),
  ('gaintolossratio', '0.9783'),
  ('profitfactor', '1.18'),
  ('hitrate', '0.5467'),
  ('t_stat', '4.417'),
  ('p_value', '1.018e-05')],
 ('You can also plot / print:',
  ['rolling_ann_std', 'drawdown', 'curve', 'percent'])]

non_do_portfolio.costs.percent.ann_mean()
-0.6141938226506388

[robcarver17]

My parameters are 'hand crafted' in the traditional sense in that I put them together using a spreadsheet - both instrument and forecast weights.

Have you tried running with equal weighting on forecasts (just removing trading rules that are too expensive) and/or instrument weights?

Turnover and costs both look reasonable.

[mjuhanne]

Please check your shadow cost. I didn't actually see the turnover calculated in your posts but it could be the culprit.

In (https://github.com/robcarver17/pysystemtrade/blob/master/docs/production_strategy_changes.md) it's stated that the default value is 10 but for me that caused an awul lot of trading (something like 20-30 trades per day for 300k portfolio). An that's in real-life trading system and not back testing!

I thought it was just the volatility in February but after the high volume trading sustained, I made some back testing and it seems that the sweet spot is somewhere 20-50 depending on the capital and other parameters. There's also another comment somewhere here by Rob where he tuned the shadow cost using turnover.

[robcarver17]

Default shadow cost is 50 in the current .yaml file, and that's what I'm using.

[emretezel]

So I have looked at under equal weights costs across each rule (all instruments) and across each instrument (all rules). How should I interpret the result to decide what is truly expensive? For example highest costing rule seems breakout10 at -0.10. What is the definition of this figure? Similarly highest costing instrument seems JPY at -0.65.

trading_rule_curves = system.accounts.pandl_for_all_trading_rules()
for rule in trading_rule_curves.nested_account_curves.keys():
    print(rule + " " + str(trading_rule_curves[rule].costs.percent.ann_mean()))

breakout10 -0.1011277012831177
breakout20 -0.044701897131271234
breakout40 -0.02260432604520947
breakout80 -0.012278519494663558
breakout160 -0.007414232412195635
breakout320 -0.005181987527402282
relmomentum10 -0.003342561776504923
relmomentum20 -0.002384832934746072
relmomentum40 -0.0020885460661192387
relmomentum80 -0.001963415962986
mrinasset160 -0.0018917786599448121
carry10 -0.002457653042309768
carry30 -0.002361558874223774
carry60 -0.002312557839275748
carry125 -0.0022769919408855304
assettrend2 -0.012041736967957857
assettrend4 -0.009497189006343788
assettrend8 -0.007849865099929364
assettrend16 -0.00699490678337023
assettrend32 -0.0069526028380789
assettrend64 -0.007150907440279174
normmom2 -0.011526943766260348
normmom4 -0.009143443303269888
normmom8 -0.0075317861325971324
normmom16 -0.006652237889906469
normmom32 -0.0065021609944685786
normmom64 -0.006853000955409359
momentum4 -0.009283762024308409
momentum8 -0.0075113976560877965
momentum16 -0.006475708820194379
momentum32 -0.006093427402320606
momentum64 -0.00619160013238015
relcarry -0.0021088035846509147
skewabs365 -0.003429804150366827
skewabs180 -0.0034293596215414454
skewrv365 -0.0030887098424715563
skewrv180 -0.0030901153988167566
accel16 -0.006466194957772768
accel32 -0.00576017411937076
accel64 -0.005682999605394029
mrwrings4 -0.0025766881729123543

instrument_curves = system.accounts.pandl_across_subsystems()
for asset in instrument_curves.asset_columns:
    print(asset + " " + str(instrument_curves[asset].costs.percent.ann_mean()))

AEX -0.08440350386099339
AUD -0.36556179803162453
BITCOIN -0.30383568482589585
BOBL -0.10625708167339301
BONO -0.2286993125843624
BRE -0.37364539950778725
BRENT -0.03553501089029726
BTP -0.05153222807581382
BUND -0.05276608114277479
CAC -0.03887943543598875
CAD -0.35386879864446774
CHF -0.23347016336199286
COFFEE -0.13265269962118678
COPPER -0.1977042374901781
CORN -0.3470577193565545
CRUDE_W -0.05452866313112008
DAX -0.058646271230363166
DOW -0.024019956441720655
EDOLLAR -0.28157278958119897
EUR -0.25777883684918534
EUROSTX -0.05598919631052409
FEEDCOW -0.19033764866074274
GASOILINE -0.04915687062860507
GAS_US -0.1356582220791765
GBP -0.4454523365541721
GOLD -0.05890102461597758
HEATOIL -0.032252032230918784
INR -0.41557786109938183
IRON -0.2893826677851974
JGB -0.16285089538945252
JPY -0.65362473933354
KOSPI -0.6053016527218288
KR10 -0.09349558779123546
KR3 -0.22811363634890922
LEANHOG -0.11311759550659738
LIVECOW -0.2105057277042062
MXP -0.2121469316479926
NASDAQ -0.011440087868485178
NIKKEI -0.057527646684193585
NZD -0.11355656479608339
OAT -0.04977213973258116
PALLAD -0.15790398244735693
PLAT -0.10645726143410984
RUBBER -0.5876487228593509
RUSSELL -0.03775279472694922
SILVER -0.11386656699532655
SMI -0.12511064071476344
SOYBEAN -0.2601545068189624
SOYMEAL -0.09753292184399022
SOYOIL -0.10170120901640169
SP500 -0.021985073377280125
US10 -0.13916776890775484
US2 -0.14727968856789825
WHEAT -0.15068155859975882

[robcarver17]

No, you're doing it wrong. You need to thoroughly read the backtesting.md document and chapter twelve of Systematic Trading

The system will automatically remove trading rules that are too expensive for a given instrument, by setting post_ceiling_cost_SR - I suggest to 0.13. It will do this even if you are equally weighting.

Then to decide what instruments you should trade (suggested max SR 0.01) you can eithier use the remove markets report or do something like this:

for instrument in system.data.get_instrument_list():
    cost = system.accounts.get_SR_cost_per_trade_for_instrument(instrument)
    if cost>0.01:
        print(instrument)

You have a choice with DO of marking expensive instruments as don't trade (but keeping them in the pre DO portfolio, which means your pre DO costs will be overestimated) or assigning them zero instrument weights (so they won't be in the pre DO portfolio eithier) or dropping them entirely from the backtest.

Note you may end up with a pathological situation where you have an instrument marked as don't trade, but which is so expensive that you can't assign any trading rules to it (exceeds 0.13 limit). The manual fix is to manually assign it forecast weights: I normally just give it the slowest trend and carry rules.

[emretezel]

Thanks Rob, yes I am aware of the post_celing_cost_SR parameter, but I thought the default.yaml version was ok for prod use.

So I have now added the following settings into my strategy config. I had set them in the past based on your blog posts.

forecast_scalar_estimate:
   pool_instruments: True

forecast_weight_estimate:
   apply_cost_weight: True
   ceiling_cost_SR: 0.13
   cost_multiplier: 0.0
   pool_gross_returns: True
   equalise_gross: False

forecast_cost_estimates:
   use_pooled_costs: False
   use_pooled_turnover: True

forecast_correlation_estimate:
   pool_instruments: True

When I now run the backtest with estimations on, I do get the following error message however

2022-06-27 15:38:07 {'type': 'base_system', 'stage': 'combForecast', 'instrument_code': 'AEX'}  Only this set of rules ['accel16', 'accel32', 'accel64', 'assettrend16', 'assettrend2', 'assettrend32', 'assettrend4', 'assettrend64', 'assettrend8', 'breakout10', 'breakout160', 'breakout20', 'breakout320', 'breakout40', 'breakout80', 'carry10', 'carry125', 'carry30', 'carry60', 'momentum16', 'momentum32', 'momentum4', 'momentum64', 'momentum8', 'mrinasset160', 'mrwrings4', 'normmom16', 'normmom2', 'normmom32', 'normmom4', 'normmom64', 'normmom8', 'relcarry', 'relmomentum10', 'relmomentum20', 'relmomentum40', 'relmomentum80', 'skewabs180', 'skewabs365', 'skewrv180', 'skewrv365'] is cheap enough to trade for AEX
/Users/emre/opt/pysystemtrade/systems/accounts/account_buffering_subsystem.py:51: SyntaxWarning: "is" with a literal. Did you mean "=="?
  if buffer_method is missing_data or buffer_method is 'none':
/Users/emre/opt/pysystemtrade/systems/accounts/account_buffering_subsystem.py:51: SyntaxWarning: "is" with a literal. Did you mean "=="?
  if buffer_method is missing_data or buffer_method is 'none':
---------------------------------------------------------------------------
KeyError                                  Traceback (most recent call last)
Input In [3], in <cell line: 1>()
----> 1 do_portfolio = system.accounts.optimised_portfolio()

File ~/opt/pysystemtrade/systems/system_cache.py:723, in stage_access_cache_decorator.<locals>.decorate.<locals>.wrapper(self, *args, **kwargs)
    720 system = self.parent
    721 this_stage = self
--> 723 ans = system.cache.calc_or_cache(
    724     func,
    725     this_stage,
    726     *args,
    727     protected=protected,
    728     not_pickable=not_pickable,
    729     instrument_classify=True,
    730     **kwargs
    731 )
    733 return ans

File ~/opt/pysystemtrade/systems/system_cache.py:573, in systemCache.calc_or_cache(self, func, this_stage, protected, not_pickable, instrument_classify, use_arg_names, *args, **kwargs)
    568 value = self._get_item_from_cache(cache_ref)
    570 if value is MISSING_FROM_CACHE:
    571     # call the function. Note in the original function 'this_stage' was
    572     # 'self'
--> 573     value = func(this_stage, *args, **kwargs)
    574     self.set_item_in_cache(
    575         value, cache_ref, protected=protected, not_pickable=not_pickable
    576     )
    578 return value

File ~/opt/pysystemtrade/systems/provided/dynamic_small_system_optimise/accounts_stage.py:20, in accountForOptimisedStage.optimised_portfolio(self, delayfill)
     17 capital = self.get_notional_capital()
     18 instruments = self.get_instrument_list()
     19 dict_of_pandl_by_instrument = dict(
---> 20     [
     21         (
     22             instrument_code,
     23             self.pandl_for_optimised_instrument(
     24                 instrument_code, delayfill=delayfill
     25             ),
     26         )
     27         for instrument_code in instruments
     28     ]
     29 )
     31 dict_of_pandl_by_instrument = dictOfAccountCurves(dict_of_pandl_by_instrument)
     33 account_curve = accountCurveGroup(
     34     dict_of_pandl_by_instrument, capital=capital, weighted=True
     35 )

File ~/opt/pysystemtrade/systems/provided/dynamic_small_system_optimise/accounts_stage.py:23, in <listcomp>(.0)
     17 capital = self.get_notional_capital()
     18 instruments = self.get_instrument_list()
     19 dict_of_pandl_by_instrument = dict(
     20     [
     21         (
     22             instrument_code,
---> 23             self.pandl_for_optimised_instrument(
     24                 instrument_code, delayfill=delayfill
     25             ),
     26         )
     27         for instrument_code in instruments
     28     ]
     29 )
     31 dict_of_pandl_by_instrument = dictOfAccountCurves(dict_of_pandl_by_instrument)
     33 account_curve = accountCurveGroup(
     34     dict_of_pandl_by_instrument, capital=capital, weighted=True
     35 )

File ~/opt/pysystemtrade/systems/system_cache.py:700, in null_decorator.<locals>.wrapper(*args, **kwargs)
    699 def wrapper(*args, **kwargs):
--> 700     return func(*args, **kwargs)

File ~/opt/pysystemtrade/systems/provided/dynamic_small_system_optimise/accounts_stage.py:49, in accountForOptimisedStage.pandl_for_optimised_instrument(self, instrument_code, delayfill)
     39 @dont_cache
     40 def pandl_for_optimised_instrument(
     41     self, instrument_code: str, delayfill: bool = True
     42 ) -> accountCurve:
     44     self.log.msg(
     45         "Calculating pandl for instrument for %s" % instrument_code,
     46         instrument_code=instrument_code,
     47     )
---> 49     positions = self.get_optimised_position(instrument_code)
     51     instrument_pandl = self._pandl_for_instrument_with_positions(
     52         instrument_code, positions=positions, delayfill=delayfill
     53     )
     55     return instrument_pandl

File ~/opt/pysystemtrade/systems/provided/dynamic_small_system_optimise/accounts_stage.py:83, in accountForOptimisedStage.get_optimised_position(self, instrument_code)
     82 def get_optimised_position(self, instrument_code: str) -> pd.Series:
---> 83     opt_position_df = self.get_optimised_position_df()
     84     return opt_position_df[instrument_code]

File ~/opt/pysystemtrade/systems/provided/dynamic_small_system_optimise/accounts_stage.py:87, in accountForOptimisedStage.get_optimised_position_df(self)
     86 def get_optimised_position_df(self) -> pd.DataFrame:
---> 87     return self.parent.optimisedPositions.get_optimised_position_df()

File ~/opt/pysystemtrade/systems/system_cache.py:723, in stage_access_cache_decorator.<locals>.decorate.<locals>.wrapper(self, *args, **kwargs)
    720 system = self.parent
    721 this_stage = self
--> 723 ans = system.cache.calc_or_cache(
    724     func,
    725     this_stage,
    726     *args,
    727     protected=protected,
    728     not_pickable=not_pickable,
    729     instrument_classify=True,
    730     **kwargs
    731 )
    733 return ans

File ~/opt/pysystemtrade/systems/system_cache.py:573, in systemCache.calc_or_cache(self, func, this_stage, protected, not_pickable, instrument_classify, use_arg_names, *args, **kwargs)
    568 value = self._get_item_from_cache(cache_ref)
    570 if value is MISSING_FROM_CACHE:
    571     # call the function. Note in the original function 'this_stage' was
    572     # 'self'
--> 573     value = func(this_stage, *args, **kwargs)
    574     self.set_item_in_cache(
    575         value, cache_ref, protected=protected, not_pickable=not_pickable
    576     )
    578 return value

File ~/opt/pysystemtrade/systems/provided/dynamic_small_system_optimise/optimised_positions_stage.py:47, in optimisedPositions.get_optimised_position_df(self)
     44 @diagnostic()
     45 def get_optimised_position_df(self) -> pd.DataFrame:
     46     self.log.msg("Optimising positions for small capital: may take a while!")
---> 47     common_index = list(self.common_index())
     48     progress = progressBar(
     49         len(common_index),
     50         suffix="Optimising positions",
     51         show_timings=True,
     52         show_each_time=True
     53     )
     54     previous_optimal_positions = portfolioWeights.allzeros(self.instrument_list())

File ~/opt/pysystemtrade/systems/provided/dynamic_small_system_optimise/optimised_positions_stage.py:281, in optimisedPositions.common_index(self)
    280 def common_index(self):
--> 281     return self.portfolio_stage.common_index()

File ~/opt/pysystemtrade/systems/system_cache.py:723, in stage_access_cache_decorator.<locals>.decorate.<locals>.wrapper(self, *args, **kwargs)
    720 system = self.parent
    721 this_stage = self
--> 723 ans = system.cache.calc_or_cache(
    724     func,
    725     this_stage,
    726     *args,
    727     protected=protected,
    728     not_pickable=not_pickable,
    729     instrument_classify=True,
    730     **kwargs
    731 )
    733 return ans

File ~/opt/pysystemtrade/systems/system_cache.py:573, in systemCache.calc_or_cache(self, func, this_stage, protected, not_pickable, instrument_classify, use_arg_names, *args, **kwargs)
    568 value = self._get_item_from_cache(cache_ref)
    570 if value is MISSING_FROM_CACHE:
    571     # call the function. Note in the original function 'this_stage' was
    572     # 'self'
--> 573     value = func(this_stage, *args, **kwargs)
    574     self.set_item_in_cache(
    575         value, cache_ref, protected=protected, not_pickable=not_pickable
    576     )
    578 return value

File ~/opt/pysystemtrade/systems/portfolio.py:1066, in Portfolios.common_index(self)
   1064 @diagnostic()
   1065 def common_index(self):
-> 1066     portfolio_weights = self.get_original_portfolio_weight_df()
   1067     common_index = portfolio_weights.index
   1069     return common_index

File ~/opt/pysystemtrade/systems/system_cache.py:723, in stage_access_cache_decorator.<locals>.decorate.<locals>.wrapper(self, *args, **kwargs)
    720 system = self.parent
    721 this_stage = self
--> 723 ans = system.cache.calc_or_cache(
    724     func,
    725     this_stage,
    726     *args,
    727     protected=protected,
    728     not_pickable=not_pickable,
    729     instrument_classify=True,
    730     **kwargs
    731 )
    733 return ans

File ~/opt/pysystemtrade/systems/system_cache.py:573, in systemCache.calc_or_cache(self, func, this_stage, protected, not_pickable, instrument_classify, use_arg_names, *args, **kwargs)
    568 value = self._get_item_from_cache(cache_ref)
    570 if value is MISSING_FROM_CACHE:
    571     # call the function. Note in the original function 'this_stage' was
    572     # 'self'
--> 573     value = func(this_stage, *args, **kwargs)
    574     self.set_item_in_cache(
    575         value, cache_ref, protected=protected, not_pickable=not_pickable
    576     )
    578 return value

File ~/opt/pysystemtrade/systems/portfolio.py:1075, in Portfolios.get_original_portfolio_weight_df(self)
   1071 @diagnostic()
   1072 def get_original_portfolio_weight_df(self) -> seriesOfPortfolioWeights:
   1073     instrument_list = self.get_instrument_list()
   1074     weights_as_dict = dict(
-> 1075         [
   1076             (
   1077                 instrument_code,
   1078                 self.get_portfolio_weight_series_from_contract_positions(
   1079                     instrument_code
   1080                 ),
   1081             )
   1082             for instrument_code in instrument_list
   1083         ]
   1084     )
   1086     weights_as_pd = pd.DataFrame(weights_as_dict)
   1087     weights_as_pd = weights_as_pd.ffill()

File ~/opt/pysystemtrade/systems/portfolio.py:1078, in <listcomp>(.0)
   1071 @diagnostic()
   1072 def get_original_portfolio_weight_df(self) -> seriesOfPortfolioWeights:
   1073     instrument_list = self.get_instrument_list()
   1074     weights_as_dict = dict(
   1075         [
   1076             (
   1077                 instrument_code,
-> 1078                 self.get_portfolio_weight_series_from_contract_positions(
   1079                     instrument_code
   1080                 ),
   1081             )
   1082             for instrument_code in instrument_list
   1083         ]
   1084     )
   1086     weights_as_pd = pd.DataFrame(weights_as_dict)
   1087     weights_as_pd = weights_as_pd.ffill()

File ~/opt/pysystemtrade/systems/system_cache.py:723, in stage_access_cache_decorator.<locals>.decorate.<locals>.wrapper(self, *args, **kwargs)
    720 system = self.parent
    721 this_stage = self
--> 723 ans = system.cache.calc_or_cache(
    724     func,
    725     this_stage,
    726     *args,
    727     protected=protected,
    728     not_pickable=not_pickable,
    729     instrument_classify=True,
    730     **kwargs
    731 )
    733 return ans

File ~/opt/pysystemtrade/systems/system_cache.py:573, in systemCache.calc_or_cache(self, func, this_stage, protected, not_pickable, instrument_classify, use_arg_names, *args, **kwargs)
    568 value = self._get_item_from_cache(cache_ref)
    570 if value is MISSING_FROM_CACHE:
    571     # call the function. Note in the original function 'this_stage' was
    572     # 'self'
--> 573     value = func(this_stage, *args, **kwargs)
    574     self.set_item_in_cache(
    575         value, cache_ref, protected=protected, not_pickable=not_pickable
    576     )
    578 return value

File ~/opt/pysystemtrade/systems/portfolio.py:1138, in Portfolios.get_portfolio_weight_series_from_contract_positions(self, instrument_code)
   1134 @diagnostic()
   1135 def get_portfolio_weight_series_from_contract_positions(
   1136     self, instrument_code: str
   1137 ) -> pd.Series:
-> 1138     contract_positions = self.get_notional_position_before_risk_scaling(instrument_code)
   1139     per_contract_value_as_proportion_of_capital = (
   1140         self.get_per_contract_value_as_proportion_of_capital(instrument_code)
   1141     )
   1143     weights_as_proportion_of_capital = get_portfolio_weights_from_contract_positions(
   1144         contract_positions=contract_positions,
   1145         per_contract_value_as_proportion_of_capital=per_contract_value_as_proportion_of_capital,
   1146     )

File ~/opt/pysystemtrade/systems/system_cache.py:723, in stage_access_cache_decorator.<locals>.decorate.<locals>.wrapper(self, *args, **kwargs)
    720 system = self.parent
    721 this_stage = self
--> 723 ans = system.cache.calc_or_cache(
    724     func,
    725     this_stage,
    726     *args,
    727     protected=protected,
    728     not_pickable=not_pickable,
    729     instrument_classify=True,
    730     **kwargs
    731 )
    733 return ans

File ~/opt/pysystemtrade/systems/system_cache.py:573, in systemCache.calc_or_cache(self, func, this_stage, protected, not_pickable, instrument_classify, use_arg_names, *args, **kwargs)
    568 value = self._get_item_from_cache(cache_ref)
    570 if value is MISSING_FROM_CACHE:
    571     # call the function. Note in the original function 'this_stage' was
    572     # 'self'
--> 573     value = func(this_stage, *args, **kwargs)
    574     self.set_item_in_cache(
    575         value, cache_ref, protected=protected, not_pickable=not_pickable
    576     )
    578 return value

File ~/opt/pysystemtrade/systems/portfolio.py:218, in Portfolios.get_notional_position_before_risk_scaling(self, instrument_code)
    214 """
    215 """
    217 # same frequency as subsystem / forecasts
--> 218 notional_position_without_idm = self.get_notional_position_without_idm(
    219     instrument_code
    220 )
    222 ## daily
    223 idm = self.get_instrument_diversification_multiplier()

File ~/opt/pysystemtrade/systems/system_cache.py:723, in stage_access_cache_decorator.<locals>.decorate.<locals>.wrapper(self, *args, **kwargs)
    720 system = self.parent
    721 this_stage = self
--> 723 ans = system.cache.calc_or_cache(
    724     func,
    725     this_stage,
    726     *args,
    727     protected=protected,
    728     not_pickable=not_pickable,
    729     instrument_classify=True,
    730     **kwargs
    731 )
    733 return ans

File ~/opt/pysystemtrade/systems/system_cache.py:573, in systemCache.calc_or_cache(self, func, this_stage, protected, not_pickable, instrument_classify, use_arg_names, *args, **kwargs)
    568 value = self._get_item_from_cache(cache_ref)
    570 if value is MISSING_FROM_CACHE:
    571     # call the function. Note in the original function 'this_stage' was
    572     # 'self'
--> 573     value = func(this_stage, *args, **kwargs)
    574     self.set_item_in_cache(
    575         value, cache_ref, protected=protected, not_pickable=not_pickable
    576     )
    578 return value

File ~/opt/pysystemtrade/systems/portfolio.py:235, in Portfolios.get_notional_position_without_idm(self, instrument_code)
    233 @diagnostic()
    234 def get_notional_position_without_idm(self, instrument_code: str) -> pd.Series:
--> 235     instr_weights = self.get_instrument_weights()
    237     # unknown frequency
    238     subsys_position = self.get_subsystem_position(instrument_code)

File ~/opt/pysystemtrade/systems/system_cache.py:723, in stage_access_cache_decorator.<locals>.decorate.<locals>.wrapper(self, *args, **kwargs)
    720 system = self.parent
    721 this_stage = self
--> 723 ans = system.cache.calc_or_cache(
    724     func,
    725     this_stage,
    726     *args,
    727     protected=protected,
    728     not_pickable=not_pickable,
    729     instrument_classify=True,
    730     **kwargs
    731 )
    733 return ans

File ~/opt/pysystemtrade/systems/system_cache.py:573, in systemCache.calc_or_cache(self, func, this_stage, protected, not_pickable, instrument_classify, use_arg_names, *args, **kwargs)
    568 value = self._get_item_from_cache(cache_ref)
    570 if value is MISSING_FROM_CACHE:
    571     # call the function. Note in the original function 'this_stage' was
    572     # 'self'
--> 573     value = func(this_stage, *args, **kwargs)
    574     self.set_item_in_cache(
    575         value, cache_ref, protected=protected, not_pickable=not_pickable
    576     )
    578 return value

File ~/opt/pysystemtrade/systems/portfolio.py:417, in Portfolios.get_instrument_weights(self)
    407 """
    408 Get the time series of instrument weights, accounting for potentially missing positions, and weights that don't add up.
    409 
   (...)
    412 
    413 """
    415 smooth_weighting = self.config.instrument_weight_ewma_span
    416 daily_unsmoothed_instrument_weights = (
--> 417     self.get_unsmoothed_instrument_weights_fitted_to_position_lengths()
    418 )
    420 # smooth to avoid jumps when they change
    421 instrument_weights = daily_unsmoothed_instrument_weights.ewm(
    422     smooth_weighting
    423 ).mean()

File ~/opt/pysystemtrade/systems/system_cache.py:723, in stage_access_cache_decorator.<locals>.decorate.<locals>.wrapper(self, *args, **kwargs)
    720 system = self.parent
    721 this_stage = self
--> 723 ans = system.cache.calc_or_cache(
    724     func,
    725     this_stage,
    726     *args,
    727     protected=protected,
    728     not_pickable=not_pickable,
    729     instrument_classify=True,
    730     **kwargs
    731 )
    733 return ans

File ~/opt/pysystemtrade/systems/system_cache.py:573, in systemCache.calc_or_cache(self, func, this_stage, protected, not_pickable, instrument_classify, use_arg_names, *args, **kwargs)
    568 value = self._get_item_from_cache(cache_ref)
    570 if value is MISSING_FROM_CACHE:
    571     # call the function. Note in the original function 'this_stage' was
    572     # 'self'
--> 573     value = func(this_stage, *args, **kwargs)
    574     self.set_item_in_cache(
    575         value, cache_ref, protected=protected, not_pickable=not_pickable
    576     )
    578 return value

File ~/opt/pysystemtrade/systems/portfolio.py:433, in Portfolios.get_unsmoothed_instrument_weights_fitted_to_position_lengths(self)
    429 @diagnostic()
    430 def get_unsmoothed_instrument_weights_fitted_to_position_lengths(
    431     self,
    432 ) -> pd.DataFrame:
--> 433     raw_instrument_weights = self.get_unsmoothed_raw_instrument_weights()
    435     instrument_list = list(raw_instrument_weights.columns)
    437     subsystem_positions = [
    438         self.get_subsystem_position(instrument_code)
    439         for instrument_code in instrument_list
    440     ]

File ~/opt/pysystemtrade/systems/system_cache.py:723, in stage_access_cache_decorator.<locals>.decorate.<locals>.wrapper(self, *args, **kwargs)
    720 system = self.parent
    721 this_stage = self
--> 723 ans = system.cache.calc_or_cache(
    724     func,
    725     this_stage,
    726     *args,
    727     protected=protected,
    728     not_pickable=not_pickable,
    729     instrument_classify=True,
    730     **kwargs
    731 )
    733 return ans

File ~/opt/pysystemtrade/systems/system_cache.py:573, in systemCache.calc_or_cache(self, func, this_stage, protected, not_pickable, instrument_classify, use_arg_names, *args, **kwargs)
    568 value = self._get_item_from_cache(cache_ref)
    570 if value is MISSING_FROM_CACHE:
    571     # call the function. Note in the original function 'this_stage' was
    572     # 'self'
--> 573     value = func(this_stage, *args, **kwargs)
    574     self.set_item_in_cache(
    575         value, cache_ref, protected=protected, not_pickable=not_pickable
    576     )
    578 return value

File ~/opt/pysystemtrade/systems/portfolio.py:464, in Portfolios.get_unsmoothed_raw_instrument_weights(self)
    460 self.log.terse("Calculating instrument weights")
    462 if self.use_estimated_instrument_weights():
    463     ## will probably be annnual
--> 464     raw_instrument_weights = self.get_raw_estimated_instrument_weights()
    465 else:
    466     ## will be 2*N
    467     raw_instrument_weights = self.get_raw_fixed_instrument_weights()

File ~/opt/pysystemtrade/systems/system_cache.py:723, in stage_access_cache_decorator.<locals>.decorate.<locals>.wrapper(self, *args, **kwargs)
    720 system = self.parent
    721 this_stage = self
--> 723 ans = system.cache.calc_or_cache(
    724     func,
    725     this_stage,
    726     *args,
    727     protected=protected,
    728     not_pickable=not_pickable,
    729     instrument_classify=True,
    730     **kwargs
    731 )
    733 return ans

File ~/opt/pysystemtrade/systems/system_cache.py:573, in systemCache.calc_or_cache(self, func, this_stage, protected, not_pickable, instrument_classify, use_arg_names, *args, **kwargs)
    568 value = self._get_item_from_cache(cache_ref)
    570 if value is MISSING_FROM_CACHE:
    571     # call the function. Note in the original function 'this_stage' was
    572     # 'self'
--> 573     value = func(this_stage, *args, **kwargs)
    574     self.set_item_in_cache(
    575         value, cache_ref, protected=protected, not_pickable=not_pickable
    576     )
    578 return value

File ~/opt/pysystemtrade/systems/portfolio.py:613, in Portfolios.get_raw_estimated_instrument_weights(self)
    592 """
    593 Estimate the instrument weights
    594 
   (...)
    609 2015-12-12  5.645388e-01  0.217462  0.217999
    610 """
    612 # these will probably be annual
--> 613 optimiser = self.calculation_of_raw_instrument_weights()
    614 weights_of_instruments_with_weights = optimiser.weights()
    616 instrument_weights = self._add_zero_weights_to_instrument_weights_df(
    617     weights_of_instruments_with_weights
    618 )

File ~/opt/pysystemtrade/systems/system_cache.py:723, in stage_access_cache_decorator.<locals>.decorate.<locals>.wrapper(self, *args, **kwargs)
    720 system = self.parent
    721 this_stage = self
--> 723 ans = system.cache.calc_or_cache(
    724     func,
    725     this_stage,
    726     *args,
    727     protected=protected,
    728     not_pickable=not_pickable,
    729     instrument_classify=True,
    730     **kwargs
    731 )
    733 return ans

File ~/opt/pysystemtrade/systems/system_cache.py:573, in systemCache.calc_or_cache(self, func, this_stage, protected, not_pickable, instrument_classify, use_arg_names, *args, **kwargs)
    568 value = self._get_item_from_cache(cache_ref)
    570 if value is MISSING_FROM_CACHE:
    571     # call the function. Note in the original function 'this_stage' was
    572     # 'self'
--> 573     value = func(this_stage, *args, **kwargs)
    574     self.set_item_in_cache(
    575         value, cache_ref, protected=protected, not_pickable=not_pickable
    576     )
    578 return value

File ~/opt/pysystemtrade/systems/portfolio.py:640, in Portfolios.calculation_of_raw_instrument_weights(self)
    637 # which function to use for calculation
    638 weighting_func = resolve_function(weighting_params.pop("func"))
--> 640 returns_pre_processor = self.returns_pre_processor()
    642 self.log.terse("Calculating raw instrument weights")
    644 weight_func = weighting_func(
    645     returns_pre_processor, log=self.log, **weighting_params
    646 )

File ~/opt/pysystemtrade/systems/system_cache.py:723, in stage_access_cache_decorator.<locals>.decorate.<locals>.wrapper(self, *args, **kwargs)
    720 system = self.parent
    721 this_stage = self
--> 723 ans = system.cache.calc_or_cache(
    724     func,
    725     this_stage,
    726     *args,
    727     protected=protected,
    728     not_pickable=not_pickable,
    729     instrument_classify=True,
    730     **kwargs
    731 )
    733 return ans

File ~/opt/pysystemtrade/systems/system_cache.py:573, in systemCache.calc_or_cache(self, func, this_stage, protected, not_pickable, instrument_classify, use_arg_names, *args, **kwargs)
    568 value = self._get_item_from_cache(cache_ref)
    570 if value is MISSING_FROM_CACHE:
    571     # call the function. Note in the original function 'this_stage' was
    572     # 'self'
--> 573     value = func(this_stage, *args, **kwargs)
    574     self.set_item_in_cache(
    575         value, cache_ref, protected=protected, not_pickable=not_pickable
    576     )
    578 return value

File ~/opt/pysystemtrade/systems/portfolio.py:654, in Portfolios.returns_pre_processor(self)
    650 @diagnostic(not_pickable=True)
    651 def returns_pre_processor(self) -> returnsPreProcessor:
    653     instrument_list = self.get_instrument_list(for_instrument_weights=True)
--> 654     pandl_across_subsystems_raw = self.pandl_across_subsystems(
    655         instrument_list=instrument_list
    656     )
    657     pandl_across_subsystems_as_returns_object = returnsForOptimisationWithCosts(
    658         pandl_across_subsystems_raw
    659     )
    660     pandl_across_subsystems = dictOfReturnsForOptimisationWithCosts(
    661         pandl_across_subsystems_as_returns_object
    662     )

File ~/opt/pysystemtrade/systems/system_cache.py:700, in null_decorator.<locals>.wrapper(*args, **kwargs)
    699 def wrapper(*args, **kwargs):
--> 700     return func(*args, **kwargs)

File ~/opt/pysystemtrade/systems/portfolio.py:795, in Portfolios.pandl_across_subsystems(self, instrument_list)
    792 if instrument_list is arg_not_supplied:
    793     instrument_list = self.get_instrument_list()
--> 795 return accounts.pandl_across_subsystems_given_instrument_list(
    796     instrument_list, roundpositions=False
    797 )

File ~/opt/pysystemtrade/systems/system_cache.py:723, in stage_access_cache_decorator.<locals>.decorate.<locals>.wrapper(self, *args, **kwargs)
    720 system = self.parent
    721 this_stage = self
--> 723 ans = system.cache.calc_or_cache(
    724     func,
    725     this_stage,
    726     *args,
    727     protected=protected,
    728     not_pickable=not_pickable,
    729     instrument_classify=True,
    730     **kwargs
    731 )
    733 return ans

File ~/opt/pysystemtrade/systems/system_cache.py:573, in systemCache.calc_or_cache(self, func, this_stage, protected, not_pickable, instrument_classify, use_arg_names, *args, **kwargs)
    568 value = self._get_item_from_cache(cache_ref)
    570 if value is MISSING_FROM_CACHE:
    571     # call the function. Note in the original function 'this_stage' was
    572     # 'self'
--> 573     value = func(this_stage, *args, **kwargs)
    574     self.set_item_in_cache(
    575         value, cache_ref, protected=protected, not_pickable=not_pickable
    576     )
    578 return value

File ~/opt/pysystemtrade/systems/accounts/account_subsystem.py:32, in accountSubsystem.pandl_across_subsystems_given_instrument_list(self, instrument_list, delayfill, roundpositions)
     26 @diagnostic(not_pickable=True)
     27 def pandl_across_subsystems_given_instrument_list(
     28     self, instrument_list: list, delayfill=True, roundpositions=False
     29 ) -> accountCurveGroup:
     31     dict_of_pandl_across_subsystems = dict(
---> 32         [
     33             (
     34                 instrument_code,
     35                 self.pandl_for_subsystem(
     36                     instrument_code,
     37                     delayfill=delayfill,
     38                     roundpositions=roundpositions,
     39                 ),
     40             )
     41             for instrument_code in instrument_list
     42         ]
     43     )
     45     dict_of_pandl_across_subsystems = dictOfAccountCurves(
     46         dict_of_pandl_across_subsystems
     47     )
     49     capital = self.get_notional_capital()

File ~/opt/pysystemtrade/systems/accounts/account_subsystem.py:35, in <listcomp>(.0)
     26 @diagnostic(not_pickable=True)
     27 def pandl_across_subsystems_given_instrument_list(
     28     self, instrument_list: list, delayfill=True, roundpositions=False
     29 ) -> accountCurveGroup:
     31     dict_of_pandl_across_subsystems = dict(
     32         [
     33             (
     34                 instrument_code,
---> 35                 self.pandl_for_subsystem(
     36                     instrument_code,
     37                     delayfill=delayfill,
     38                     roundpositions=roundpositions,
     39                 ),
     40             )
     41             for instrument_code in instrument_list
     42         ]
     43     )
     45     dict_of_pandl_across_subsystems = dictOfAccountCurves(
     46         dict_of_pandl_across_subsystems
     47     )
     49     capital = self.get_notional_capital()

File ~/opt/pysystemtrade/systems/system_cache.py:700, in null_decorator.<locals>.wrapper(*args, **kwargs)
    699 def wrapper(*args, **kwargs):
--> 700     return func(*args, **kwargs)

File ~/opt/pysystemtrade/systems/accounts/account_subsystem.py:97, in accountSubsystem.pandl_for_subsystem(self, instrument_code, delayfill, roundpositions)
     93     pandl = self._pandl_for_subsystem_with_SR_costs(
     94         instrument_code, delayfill=delayfill, roundpositions=roundpositions
     95     )
     96 else:
---> 97     pandl = self._pandl_for_subsystem_with_cash_costs(
     98         instrument_code, delayfill=delayfill, roundpositions=roundpositions
     99     )
    101 return pandl

File ~/opt/pysystemtrade/systems/system_cache.py:723, in stage_access_cache_decorator.<locals>.decorate.<locals>.wrapper(self, *args, **kwargs)
    720 system = self.parent
    721 this_stage = self
--> 723 ans = system.cache.calc_or_cache(
    724     func,
    725     this_stage,
    726     *args,
    727     protected=protected,
    728     not_pickable=not_pickable,
    729     instrument_classify=True,
    730     **kwargs
    731 )
    733 return ans

File ~/opt/pysystemtrade/systems/system_cache.py:573, in systemCache.calc_or_cache(self, func, this_stage, protected, not_pickable, instrument_classify, use_arg_names, *args, **kwargs)
    568 value = self._get_item_from_cache(cache_ref)
    570 if value is MISSING_FROM_CACHE:
    571     # call the function. Note in the original function 'this_stage' was
    572     # 'self'
--> 573     value = func(this_stage, *args, **kwargs)
    574     self.set_item_in_cache(
    575         value, cache_ref, protected=protected, not_pickable=not_pickable
    576     )
    578 return value

File ~/opt/pysystemtrade/systems/accounts/account_subsystem.py:150, in accountSubsystem._pandl_for_subsystem_with_cash_costs(self, instrument_code, delayfill, roundpositions)
    148 raw_costs = self.get_raw_cost_data(instrument_code)
    149 price = self.get_daily_price(instrument_code)
--> 150 positions = self.get_buffered_subsystem_position(instrument_code)
    152 fx = self.get_fx_rate(instrument_code)
    154 value_of_price_point = self.get_value_of_block_price_move(instrument_code)

File ~/opt/pysystemtrade/systems/system_cache.py:723, in stage_access_cache_decorator.<locals>.decorate.<locals>.wrapper(self, *args, **kwargs)
    720 system = self.parent
    721 this_stage = self
--> 723 ans = system.cache.calc_or_cache(
    724     func,
    725     this_stage,
    726     *args,
    727     protected=protected,
    728     not_pickable=not_pickable,
    729     instrument_classify=True,
    730     **kwargs
    731 )
    733 return ans

File ~/opt/pysystemtrade/systems/system_cache.py:573, in systemCache.calc_or_cache(self, func, this_stage, protected, not_pickable, instrument_classify, use_arg_names, *args, **kwargs)
    568 value = self._get_item_from_cache(cache_ref)
    570 if value is MISSING_FROM_CACHE:
    571     # call the function. Note in the original function 'this_stage' was
    572     # 'self'
--> 573     value = func(this_stage, *args, **kwargs)
    574     self.set_item_in_cache(
    575         value, cache_ref, protected=protected, not_pickable=not_pickable
    576     )
    578 return value

File ~/opt/pysystemtrade/systems/accounts/account_buffering_subsystem.py:48, in accountBufferingSubSystemLevel.get_buffered_subsystem_position(self, instrument_code, roundpositions)
     22 @diagnostic()
     23 def get_buffered_subsystem_position(
     24     self, instrument_code: str, roundpositions: bool = True
     25 ) -> pd.Series:
     26     """
     27     Get the buffered position
     28 
   (...)
     45     2015-12-11         1
     46     """
---> 48     optimal_position = self.get_subsystem_position(instrument_code)
     50     buffer_method = self.config.get_element_or_missing_data("buffer_method")
     51     if buffer_method is missing_data or buffer_method is 'none':

File ~/opt/pysystemtrade/systems/accounts/account_inputs.py:73, in accountInputs.get_subsystem_position(self, instrument_code)
     72 def get_subsystem_position(self, instrument_code: str) -> pd.Series:
---> 73     return self.parent.positionSize.get_subsystem_position(instrument_code)

File ~/opt/pysystemtrade/systems/system_cache.py:723, in stage_access_cache_decorator.<locals>.decorate.<locals>.wrapper(self, *args, **kwargs)
    720 system = self.parent
    721 this_stage = self
--> 723 ans = system.cache.calc_or_cache(
    724     func,
    725     this_stage,
    726     *args,
    727     protected=protected,
    728     not_pickable=not_pickable,
    729     instrument_classify=True,
    730     **kwargs
    731 )
    733 return ans

File ~/opt/pysystemtrade/systems/system_cache.py:573, in systemCache.calc_or_cache(self, func, this_stage, protected, not_pickable, instrument_classify, use_arg_names, *args, **kwargs)
    568 value = self._get_item_from_cache(cache_ref)
    570 if value is MISSING_FROM_CACHE:
    571     # call the function. Note in the original function 'this_stage' was
    572     # 'self'
--> 573     value = func(this_stage, *args, **kwargs)
    574     self.set_item_in_cache(
    575         value, cache_ref, protected=protected, not_pickable=not_pickable
    576     )
    578 return value

File ~/opt/pysystemtrade/systems/positionsizing.py:123, in PositionSizing.get_subsystem_position(self, instrument_code)
    121 avg_abs_forecast = self.avg_abs_forecast()
    122 vol_scalar = self.get_volatility_scalar(instrument_code)
--> 123 forecast = self.get_combined_forecast(instrument_code)
    125 vol_scalar = vol_scalar.reindex(forecast.index, method="ffill")
    127 subsystem_position = vol_scalar * forecast / avg_abs_forecast

File ~/opt/pysystemtrade/systems/system_cache.py:700, in null_decorator.<locals>.wrapper(*args, **kwargs)
    699 def wrapper(*args, **kwargs):
--> 700     return func(*args, **kwargs)

File ~/opt/pysystemtrade/systems/positionsizing.py:534, in PositionSizing.get_combined_forecast(self, instrument_code)
    511 @input
    512 def get_combined_forecast(self, instrument_code: str) -> pd.Series:
    513     """
    514     Get the combined forecast from previous module
    515 
   (...)
    531     2015-12-11       2.462610
    532     """
--> 534     return self.comb_forecast_stage.get_combined_forecast(instrument_code)

File ~/opt/pysystemtrade/systems/system_cache.py:723, in stage_access_cache_decorator.<locals>.decorate.<locals>.wrapper(self, *args, **kwargs)
    720 system = self.parent
    721 this_stage = self
--> 723 ans = system.cache.calc_or_cache(
    724     func,
    725     this_stage,
    726     *args,
    727     protected=protected,
    728     not_pickable=not_pickable,
    729     instrument_classify=True,
    730     **kwargs
    731 )
    733 return ans

File ~/opt/pysystemtrade/systems/system_cache.py:573, in systemCache.calc_or_cache(self, func, this_stage, protected, not_pickable, instrument_classify, use_arg_names, *args, **kwargs)
    568 value = self._get_item_from_cache(cache_ref)
    570 if value is MISSING_FROM_CACHE:
    571     # call the function. Note in the original function 'this_stage' was
    572     # 'self'
--> 573     value = func(this_stage, *args, **kwargs)
    574     self.set_item_in_cache(
    575         value, cache_ref, protected=protected, not_pickable=not_pickable
    576     )
    578 return value

File ~/opt/pysystemtrade/systems/forecast_combine.py:83, in ForecastCombine.get_combined_forecast(self, instrument_code)
     46 """
     47 Get a combined forecast, linear combination of individual forecasts
     48 with FDM applied
   (...)
     76 2015-12-11             21
     77 """
     78 self.log.msg(
     79     "Calculating combined forecast for %s" % (instrument_code),
     80     instrument_code=instrument_code,
     81 )
     82 raw_multiplied_combined_forecast = (
---> 83     self.get_raw_combined_forecast_before_mapping(instrument_code)
     84 )
     86 # apply cap and /or any non linear mapping
     87 (
     88     mapping_and_capping_function,
     89     mapping_and_capping_kwargs,
     90 ) = self._get_forecast_mapping_function(instrument_code)

File ~/opt/pysystemtrade/systems/system_cache.py:723, in stage_access_cache_decorator.<locals>.decorate.<locals>.wrapper(self, *args, **kwargs)
    720 system = self.parent
    721 this_stage = self
--> 723 ans = system.cache.calc_or_cache(
    724     func,
    725     this_stage,
    726     *args,
    727     protected=protected,
    728     not_pickable=not_pickable,
    729     instrument_classify=True,
    730     **kwargs
    731 )
    733 return ans

File ~/opt/pysystemtrade/systems/system_cache.py:573, in systemCache.calc_or_cache(self, func, this_stage, protected, not_pickable, instrument_classify, use_arg_names, *args, **kwargs)
    568 value = self._get_item_from_cache(cache_ref)
    570 if value is MISSING_FROM_CACHE:
    571     # call the function. Note in the original function 'this_stage' was
    572     # 'self'
--> 573     value = func(this_stage, *args, **kwargs)
    574     self.set_item_in_cache(
    575         value, cache_ref, protected=protected, not_pickable=not_pickable
    576     )
    578 return value

File ~/opt/pysystemtrade/systems/forecast_combine.py:105, in ForecastCombine.get_raw_combined_forecast_before_mapping(self, instrument_code)
     99 @diagnostic()
    100 def get_raw_combined_forecast_before_mapping(
    101     self, instrument_code: str
    102 ) -> pd.Series:
    103 
    104     # sum
--> 105     raw_combined_forecast = self.get_combined_forecast_without_multiplier(
    106         instrument_code
    107     )
    109     # probably daily frequency
    110     forecast_div_multiplier = self.get_forecast_diversification_multiplier(
    111         instrument_code
    112     )

File ~/opt/pysystemtrade/systems/system_cache.py:723, in stage_access_cache_decorator.<locals>.decorate.<locals>.wrapper(self, *args, **kwargs)
    720 system = self.parent
    721 this_stage = self
--> 723 ans = system.cache.calc_or_cache(
    724     func,
    725     this_stage,
    726     *args,
    727     protected=protected,
    728     not_pickable=not_pickable,
    729     instrument_classify=True,
    730     **kwargs
    731 )
    733 return ans

File ~/opt/pysystemtrade/systems/system_cache.py:573, in systemCache.calc_or_cache(self, func, this_stage, protected, not_pickable, instrument_classify, use_arg_names, *args, **kwargs)
    568 value = self._get_item_from_cache(cache_ref)
    570 if value is MISSING_FROM_CACHE:
    571     # call the function. Note in the original function 'this_stage' was
    572     # 'self'
--> 573     value = func(this_stage, *args, **kwargs)
    574     self.set_item_in_cache(
    575         value, cache_ref, protected=protected, not_pickable=not_pickable
    576     )
    578 return value

File ~/opt/pysystemtrade/systems/forecast_combine.py:132, in ForecastCombine.get_combined_forecast_without_multiplier(self, instrument_code)
    125 @diagnostic()
    126 def get_combined_forecast_without_multiplier(
    127     self, instrument_code: str
    128 ) -> pd.Series:
    129     # We take our list of rule variations from the forecasts, since it
    130     # might be that some rules were omitted in the weight calculation
--> 132     weighted_forecasts = self.get_weighted_forecasts_without_multiplier(
    133         instrument_code
    134     )
    136     # sum
    137     raw_combined_forecast = weighted_forecasts.sum(axis=1)

File ~/opt/pysystemtrade/systems/system_cache.py:723, in stage_access_cache_decorator.<locals>.decorate.<locals>.wrapper(self, *args, **kwargs)
    720 system = self.parent
    721 this_stage = self
--> 723 ans = system.cache.calc_or_cache(
    724     func,
    725     this_stage,
    726     *args,
    727     protected=protected,
    728     not_pickable=not_pickable,
    729     instrument_classify=True,
    730     **kwargs
    731 )
    733 return ans

File ~/opt/pysystemtrade/systems/system_cache.py:573, in systemCache.calc_or_cache(self, func, this_stage, protected, not_pickable, instrument_classify, use_arg_names, *args, **kwargs)
    568 value = self._get_item_from_cache(cache_ref)
    570 if value is MISSING_FROM_CACHE:
    571     # call the function. Note in the original function 'this_stage' was
    572     # 'self'
--> 573     value = func(this_stage, *args, **kwargs)
    574     self.set_item_in_cache(
    575         value, cache_ref, protected=protected, not_pickable=not_pickable
    576     )
    578 return value

File ~/opt/pysystemtrade/systems/forecast_combine.py:150, in ForecastCombine.get_weighted_forecasts_without_multiplier(self, instrument_code)
    147 rule_variation_list = self.get_trading_rule_list(instrument_code)
    148 forecasts = self.get_all_forecasts(instrument_code, rule_variation_list)
--> 150 smoothed_daily_forecast_weights = self.get_forecast_weights(instrument_code)
    151 smoothed_forecast_weights = smoothed_daily_forecast_weights.reindex(
    152     forecasts.index, method="ffill")
    154 weighted_forecasts = smoothed_forecast_weights * forecasts

File ~/opt/pysystemtrade/systems/system_cache.py:723, in stage_access_cache_decorator.<locals>.decorate.<locals>.wrapper(self, *args, **kwargs)
    720 system = self.parent
    721 this_stage = self
--> 723 ans = system.cache.calc_or_cache(
    724     func,
    725     this_stage,
    726     *args,
    727     protected=protected,
    728     not_pickable=not_pickable,
    729     instrument_classify=True,
    730     **kwargs
    731 )
    733 return ans

File ~/opt/pysystemtrade/systems/system_cache.py:573, in systemCache.calc_or_cache(self, func, this_stage, protected, not_pickable, instrument_classify, use_arg_names, *args, **kwargs)
    568 value = self._get_item_from_cache(cache_ref)
    570 if value is MISSING_FROM_CACHE:
    571     # call the function. Note in the original function 'this_stage' was
    572     # 'self'
--> 573     value = func(this_stage, *args, **kwargs)
    574     self.set_item_in_cache(
    575         value, cache_ref, protected=protected, not_pickable=not_pickable
    576     )
    578 return value

File ~/opt/pysystemtrade/systems/forecast_combine.py:164, in ForecastCombine.get_forecast_weights(self, instrument_code)
    160 @diagnostic()
    161 def get_forecast_weights(self, instrument_code: str) -> pd.DataFrame:
    162     # These will be in daily frequency
    163     daily_forecast_weights_fixed_to_forecasts_unsmoothed = (
--> 164         self.get_unsmoothed_forecast_weights(instrument_code)
    165     )
    167     # smooth out weights
    168     forecast_smoothing_ewma_span = self.config.forecast_weight_ewma_span

File ~/opt/pysystemtrade/systems/system_cache.py:723, in stage_access_cache_decorator.<locals>.decorate.<locals>.wrapper(self, *args, **kwargs)
    720 system = self.parent
    721 this_stage = self
--> 723 ans = system.cache.calc_or_cache(
    724     func,
    725     this_stage,
    726     *args,
    727     protected=protected,
    728     not_pickable=not_pickable,
    729     instrument_classify=True,
    730     **kwargs
    731 )
    733 return ans

File ~/opt/pysystemtrade/systems/system_cache.py:573, in systemCache.calc_or_cache(self, func, this_stage, protected, not_pickable, instrument_classify, use_arg_names, *args, **kwargs)
    568 value = self._get_item_from_cache(cache_ref)
    570 if value is MISSING_FROM_CACHE:
    571     # call the function. Note in the original function 'this_stage' was
    572     # 'self'
--> 573     value = func(this_stage, *args, **kwargs)
    574     self.set_item_in_cache(
    575         value, cache_ref, protected=protected, not_pickable=not_pickable
    576     )
    578 return value

File ~/opt/pysystemtrade/systems/forecast_combine.py:229, in ForecastCombine.get_unsmoothed_forecast_weights(self, instrument_code)
    221 self.log.msg(
    222     "Calculating forecast weights for %s" % (instrument_code),
    223     instrument_code=instrument_code,
    224 )
    226 # note these might include missing weights, eg too expensive, or absent
    227 # from fixed weights
    228 # These are monthly to save space, or possibly even only 2 rows long
--> 229 monthly_forecast_weights = self.get_raw_monthly_forecast_weights(
    230     instrument_code
    231 )
    233 # fix to forecast time series
    234 forecast_weights_fixed_to_forecasts = self._fix_weights_to_forecasts(
    235     instrument_code=instrument_code,
    236     monthly_forecast_weights=monthly_forecast_weights,
    237 )

File ~/opt/pysystemtrade/systems/system_cache.py:700, in null_decorator.<locals>.wrapper(*args, **kwargs)
    699 def wrapper(*args, **kwargs):
--> 700     return func(*args, **kwargs)

File ~/opt/pysystemtrade/systems/forecast_combine.py:515, in ForecastCombine.get_raw_monthly_forecast_weights(self, instrument_code)
    511 else:
    512     ## will come back as 2*N data frame
    513     forecast_weights = self.get_raw_fixed_forecast_weights(instrument_code)
--> 515 forecast_weights_cheap_rules_only = self._remove_expensive_rules_from_weights(
    516     instrument_code, forecast_weights
    517 )
    520 return forecast_weights_cheap_rules_only

File ~/opt/pysystemtrade/systems/system_cache.py:700, in null_decorator.<locals>.wrapper(*args, **kwargs)
    699 def wrapper(*args, **kwargs):
--> 700     return func(*args, **kwargs)

File ~/opt/pysystemtrade/systems/forecast_combine.py:253, in ForecastCombine._remove_expensive_rules_from_weights(self, instrument_code, monthly_forecast_weights)
    246 @dont_cache
    247 def _remove_expensive_rules_from_weights(
    248     self, instrument_code, monthly_forecast_weights: pd.DataFrame
    249 ) -> pd.DataFrame:
    251     cheap_rules = self.cheap_trading_rules_post_processing(instrument_code)
--> 253     monthly_forecast_weights_cheap_rules_only = monthly_forecast_weights[
    254         cheap_rules
    255     ]
    257     return monthly_forecast_weights_cheap_rules_only

File ~/opt/anaconda3/envs/pysystemtrade/lib/python3.8/site-packages/pandas/core/frame.py:2806, in DataFrame.__getitem__(self, key)
   2804     if is_iterator(key):
   2805         key = list(key)
-> 2806     indexer = self.loc._get_listlike_indexer(key, axis=1, raise_missing=True)[1]
   2808 # take() does not accept boolean indexers
   2809 if getattr(indexer, "dtype", None) == bool:

File ~/opt/anaconda3/envs/pysystemtrade/lib/python3.8/site-packages/pandas/core/indexing.py:1552, in _NDFrameIndexer._get_listlike_indexer(self, key, axis, raise_missing)
   1549 else:
   1550     keyarr, indexer, new_indexer = ax._reindex_non_unique(keyarr)
-> 1552 self._validate_read_indexer(
   1553     keyarr, indexer, o._get_axis_number(axis), raise_missing=raise_missing
   1554 )
   1555 return keyarr, indexer

File ~/opt/anaconda3/envs/pysystemtrade/lib/python3.8/site-packages/pandas/core/indexing.py:1646, in _NDFrameIndexer._validate_read_indexer(self, key, indexer, axis, raise_missing)
   1644 if not (self.name == "loc" and not raise_missing):
   1645     not_found = list(set(key) - set(ax))
-> 1646     raise KeyError(f"{not_found} not in index")
   1648 # we skip the warning on Categorical/Interval
   1649 # as this check is actually done (check for
   1650 # non-missing values), but a bit later in the
   1651 # code, so we want to avoid warning & then
   1652 # just raising
   1653 if not (ax.is_categorical() or ax.is_interval()):

KeyError: "['relmomentum10', 'normmom2', 'assettrend2', 'breakout10'] not in index"

[robcarver17]

Sorry I don't have time to be an unpaid debugger for other people. I'm trying to help but you're going to have to do some more heavy lifting.

[emretezel]

That's fair, I will look into debugging it.

[emretezel]

Btw, when I do system.accounts.pandl_for_all_trading_rules() does it pick account curves for DO or non-DO?

[robcarver17]

Non DO. It woul dbe impossible to do that for DO, DO 'loses' the linear additivity of trading rules up to instrument p&l.

All account functions are non DO except for the two that explicitally reference it.

[emretezel]

Would this be the right way to look at costs for each instrument under DO?

do_portfolio = system.accounts.optimised_portfolio()
for asset in do_portfolio.asset_columns:
    print(asset + " " + str(do_portfolio[asset].costs.percent.ann_mean()))

AEX -0.00020356551935856053
AUD -0.000635446042690715
BITCOIN -0.00031249999634626964
BOBL -0.00026075447800947855
BONO 0.0
BRE -0.0005327964467866968
BRENT -9.678763256706038e-05
BTP -5.2425830450297654e-05
BUND 0.0
CAC -0.00037028218026837037
CAD -0.001416104849382942
CHF -0.000907634353787289
COFFEE 0.0
COPPER -0.00014642763787386446
CORN -0.0010962256541172137
CRUDE_W -0.00032174300345700463
DAX -6.048406608308716e-05
DOW -6.0211744382484346e-05
EDOLLAR -0.0006583697130961906
EUR -0.0008018728844540964
EUROSTX -0.0005589924721005769
FEEDCOW 0.0
GASOILINE 0.0
GAS_US 0.0
GBP -0.0012647327043282485
GOLD -0.0002767195310690403
HEATOIL 0.0
INR -0.0003968671967561903
IRON 0.0
JGB -0.00011294012075223578
JPY 0.0
KOSPI -0.0020127458658680867
KR10 0.0
KR3 -0.0007307911461539723
LEANHOG 0.0
LIVECOW 0.0
MXP -0.0008213885224748084
NASDAQ -7.435662982348551e-05
NIKKEI -6.140474371452097e-05
NZD -0.0012831550992877206
OAT 0.0
PALLAD 0.0
PLAT -0.0005279328532184598
RUBBER -0.004898332301732623
RUSSELL -9.753331848251837e-05
SILVER -0.001384378369298459
SMI -0.00014604195977575645
SOYBEAN -0.0007057936619245514
SOYMEAL 0.0
SOYOIL -0.00038564822228102474
SP500 -8.74612629927575e-05
US10 0.0
US2 -0.0003959209423044069
WHEAT -0.0002359824677558704

[robcarver17]

That's the contribution of each instrument to the total cost; this could be high because the instrument is expensive to trade, or just because the DO happens to like it. So not informative for finding expensive instruments.

[mjuhanne]

If you haven't yet done it, I would heartily recommend installing Visual Studio Code (or some other IDE) with Python extension and then running your own simulation and different production scripts with few breakpoints here and there. There's so much one can learn about the inner workings of a software just by looking at the call stack, variables etc in real time.

[emretezel]

That’s fair, I will debug.

…

Sent from my iPhone

On 27 Jun 2022, at 18:15, Robert Carver ***@***.***> wrote:  Sorry I don't have time to be an unpaid debugger for other people. I'm trying to help but you're going to have to do some more heavy lifting. — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you authored the thread.