Improve previous process logic

[tgibson11]

Example:

Day 1:
6:00 cron kicks off run_systems and run_strategy_order_generator
run_systems starts immediately; run_strategy_order_generator waits
6:30 run_systems completes, run_strategy_order_generator starts doing stuff
Everything is as expected.

Day 2:
6:00 cron kicks off run_systems and run_strategy_order_generator
Both processes start immediately!

That is a contrived example, but can easily happen in real life because process end-times vary from day to day even under normal circumstances. Then there are unusual circumstances: a process takes much longer to run than usual, or a process is run manually at a much later time than usual. The next day, any dependent processes will run prematurely.

Proposed solution:

1. Check if previous process is actually running; if so, WAIT (easy)
2. Check if previous process is waiting to run (because there can be a series of such dependencies). If so, WAIT. More difficult, because I don't think the database would tell you this right now - would need to make a system call, or make changes to how process control works.
3. Only then check if the process has finished in the last 24 hours.

I think that should cover all reasonable scenarios. It wouldn't handle dumb configurations, like starting a dependent process via cron before starting its previous process.

I'd be happy to work on this change, if you agree it's worth doing.

[TobiasAntiGravity]

You guys are wizards, and I am a hack, and so this will probably sound extremely hackey from your perspectives, but can't help my self; docker makes it easy (as long as nothing crashes) you can do one process per container and start one container after the other is finished. That's what I am doing.

Bonus is - extremely easy to set up the whole system (as long as you have a backup - for testing or redeployment in case of unfortunate events.

My production environment (soon to be anyway) - it's not the main branch yet because documentation is not up to date. If you want to see. (I have end of day processes in one container, but can easily change that

[tgibson11]

I don't think we want to make pysystemtrade dependent on Docker though.

[TobiasAntiGravity]

dependencies makes things more complex and brittle, I can see that!

[tgibson11]

I just realized that it does check if the previous process is currently running, so that makes this less likely to be an issue.

Still, I don't think it recognizes if the previous process is waiting to run.

[TobiasAntiGravity]

FYI @tgibson11 -> I am just about to go live and something unexpected happened; run_system will not run because it is saying that Not starting process:because Previous process still running

Previous process is defined as the run_daily_prices_update. I had not run this (so it was not still running - it had not run at all), because I did an update before importing data to a blank db. so - it seems that it does do a check.

[robcarver17]

I'm a bit reluctant to endorse this. The problem is that the more complex the process configuration, the harder it is to debug things when things go wrong. It's actually hard enough right now to be honest.

Right now the idea is that you make sure you kick of your processes in order, but each has a dependency on the previous process running and finishing, so in practice they will run in sequence. If something is regularly not starting because of an over run of a previous process, then you need to change your timetambling. It's basically cron but with a single dependency per process.

There are probably many good and sophisticated ways to schedule and handle proceses out there, in python and as seperate applications, and I'm reluctant to completely reinvent the wheel. psystemtrade is hopefully modular enough that you can use any of these other tools if you aren't happy with eithier basic cron or the slightly more advanced solution provided.

[tgibson11]

That sounds good to me - keep things simple in pysystemtrade, and use an external tool if the built-in features don't meet your requirements.

[robcarver17]

Converted to discussion as it's not a bug or a nailed down feature request.